Update and document virtual memory management

docs/advanced.md

@@ -82,17 +82,23 @@ To manage the virtual address space for ArgoDSM applications, we acquire large
 amounts of virtual memory. There are currently three ways to acquire the
 underlying memory for ArgoDSM, and exactly one must be selected at compile time:
 
-1. POSIX shared memory objects (`-DARGO_VM_SHM`)
-2. `memfd_create` syscall      (`-DARGO_VM_MEMFD`)
+1. `memfd_create` syscall      (`-DARGO_VM_MEMFD`)
+2. POSIX shared memory objects (`-DARGO_VM_SHM`)
 3. anonymous remappable memory (`-DARGO_VM_ANONYMOUS`)
 
 Each version has its own downsides:
-1. POSIX shared memory objects are size-limited and require `/dev/shm` support.
-2. `memfd_create` requires Linux kernel version of 3.17+ and memory overcommit.
+1. `memfd_create` requires Linux kernel version of 3.17+ and memory overcommit.
+2. POSIX shared memory objects are size-limited and require `/dev/shm` support.
 3. anonymous remappable memory has a runtime overhead, and relies on kernel
    functionality that is deprecated since Linux version 3.16.
 
-For now, the default is to use POSIX shared memory objects.
+The default and best option is to use `ARGO_VM_MEMFD`.
+Only another option if you cannot upgrade your Linux kernel.
+If you use `ARGO_VM_SHM`, make sure you mount `/dev/shm` with a sufficient size,
+it should be larger than your system memory!
+If you cannot mount `/dev/shm` with sufficient size for your program, the only
+remaining option is `ARGO_VM_ANONYMOUS`. It should work, but there is a large
+(20% to 40% in our tests) runtime overhead for using this outdated interface.
 
 
 ## Data Allocation Policies

docs/index.md

@@ -26,7 +26,7 @@ ArgoDSM depends on the
 [C++ QD Locking Library](https://github.com/davidklaftenegger/qd_library).
 This is included as a git submodule and is built automatically by CMake,
 requiring both git and an internet connection. For instructions on how to build
-offline, see [building ArgoDSM offline](#Building-ArgoDSM-offline).
+offline, see [building ArgoDSM offline](building-argodsm-offline).
 
 Additionally, ArgoDSM requires `libnuma` to detect whether it is running on top
 of NUMA systems, and if so how they are structured internally.
@@ -38,12 +38,25 @@ respectively are required.
 
 Note: adjust the below commands to your needs, especially `CMAKE_INSTALL_PREFIX`.
 
+You must activate exactly one of the `ARGO_VM_*` options. See
+[here]({{ site.baseurl }}/advanced.html#virtual-memory-management) for more
+details.
+`ARGO_VM_MEMFD`. If your kernel is too ancient and you cannot it, the next best
+option is `ARGO_VM_SHM`. However, your system administrator must mount `/dev/shm`
+with a tmpfs of sufficient size for your *entire* program (across all nodes).
+This is safe to do, as there will be no actual data stored in the location. If
+you cannot get enough space in `/dev/shm`, you must use `ARGO_VM_ANONYMOUS`,
+which in our experience is 20% to 40% slower.
+
 ``` bash
 git clone https://github.com/etascale/argodsm.git
 cd argodsm && mkdir build && cd build
 cmake -DARGO_BACKEND_MPI=ON              \
       -DARGO_BACKEND_SINGLENODE=ON       \
       -DARGO_TESTS=ON                    \
+      -DARGO_VM_SHM=OFF                  \
+      -DARGO_VM_ANONYMOUS=OFF            \
+      -DARGO_VM_MEMFD=ON                 \
       -DBUILD_DOCUMENTATION=ON           \
       -DCMAKE_CXX_COMPILER=mpic++        \
       -DCMAKE_C_COMPILER=mpicc           \

src/CMakeLists.txt

@@ -28,11 +28,11 @@ endforeach(src)
 
 # exactly one of these must be enabled. below is some code to ensure this.
 option(ARGO_VM_SHM
-	"Handle virtual addresses using POSIX shared memory. Size-limited." ON)
+	"Handle virtual addresses using POSIX shared memory. Size-limited." OFF)
 option(ARGO_VM_ANONYMOUS
 	"Handle virtual addresses using anonymously-mapped memory. Slow." OFF)
 option(ARGO_VM_MEMFD
-	"Handle virtual addresses using an anonymous memory file. Requires kernel 3.17+." OFF)
+	"Handle virtual addresses using an anonymous memory file. Requires kernel 3.17+." ON)
 
 set(ARGO_VM_COUNT 0 CACHE INTERNAL "Check for multiple virtual address handlers")
 unset(vm_libs)