Create goal in mix mode with App Direct set to No Interleaved

[DangoNetwork]

Hi, I am trying to create a goal in mix mode with the app direct set to no interleaved. Is there a command I can use to configure that in OS using ipmctl? So far, I can only configure it in the BIOS.

[sscargal]

@DangoNetwork: See the example in the ipmctl-create-goal man page.

Configures the capacity on each PMem module with 20% of the capacity in Memory Mode and the remaining as App Direct capacity that does not use hardware interleaving.

ipmctl create -goal MemoryMode=20 PersistentMemoryType=AppDirectNotInterleaved

You must remove the BIOS configuration first so the OS can manage the PMem modules. Otherwise, the BIOS config takes priority. You'll also need to adjust the MemoryMode% according to your requirements and hardware population.

MemoryMode will interleave across all PMem modules in the socket rather than using specific modules. Any remaining capacity will be used for AppDirect. So if you have 128GB PMem Modules and specify 20% for MemoryMode and the remainder for App Direct, I expect the PMem modules to appear as ~100GB App Direct Not-Interleaved in the OS.

I strongly caution against this approach for performance reasons. It's entirely plausible that the bandwidth used by Memory Mode will dominate, causing indeterministic performance for applications using AppDirect. You have no control over MemoryMode traffic, which the memory controller handles. For this reason, I recommend you use 100% Memory Mode or 100% AppDirect if you want to do benchmarking or run applications in production. If this is purely research, and you don't mind the performance penalty, then go for it.

[DangoNetwork]

Cool, that command did work. However, I found another issue. The show -goal always say There are no goal configs defined in the system. But my goal command did apply after the reboot. Is this normal? Also, does the goal, region, namespace configuration stored on motherboard/BIOS or on the DCPMM itself or within OS?

[sscargal]

The show -goal always say There are no goal configs defined in the system. But my goal command did apply after the reboot. Is this normal?

Yes, that is expected. The ipmctl show -goal shows unapplied goal configurations vs currently applied goals. To show the currently applied goal/config use ipmctl show -memoryresources. You'll get a table similar to the following:

$ ipmctl show -memoryresources
 MemoryType   | DDR         | PMemModule   | Total
==========================================================
 Volatile     | 381.500 GiB | 0.000 GiB    | 381.500 GiB
 AppDirect    | -           | 3024.000 GiB | 3024.000 GiB
 Cache        | 0.000 GiB   | -            | 0.000 GiB
 Inaccessible | 2.500 GiB   | 5.452 GiB    | 7.952 GiB
 Physical     | 384.000 GiB | 3029.452 GiB | 3413.452 GiB

This example shows a 100% AppDirect configuration.

Also, does the goal, region, namespace configuration stored on motherboard/BIOS or on the DCPMM itself or within OS?

The configuration is stored on each PMem module.