Skip to content

Add memory file option (-e) for fast warm restart (#540)#540

Open
excelle08 wants to merge 1 commit intofacebookresearch:v2-betafrom
excelle08:export-D97244738-to-v2-beta
Open

Add memory file option (-e) for fast warm restart (#540)#540
excelle08 wants to merge 1 commit intofacebookresearch:v2-betafrom
excelle08:export-D97244738-to-v2-beta

Conversation

@excelle08
Copy link
Copy Markdown
Contributor

@excelle08 excelle08 commented Mar 24, 2026

Summary:

Expose memcached's native -e memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:

  • Add --memory-file argument to server args
  • Append -e <path> to memcached command when memory file specified
  • Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
  • Per-instance memory files in autoscale mode (suffix .0, .1, etc.)
  • Auto-expand /dev/shm when tmpfs is smaller than memsize
  • Add memory_file variable to all TaoBench job configurations
  • Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 24, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync bot commented Mar 24, 2026

@excelle08 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D97244738.

excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 24, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from 881451c to 902cec2 Compare March 24, 2026 20:16
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 24, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Differential Revision: D97244738
@meta-codesync meta-codesync bot changed the title Add memory file option (-e) for fast warm restart Add memory file option (-e) for fast warm restart (#540) Mar 24, 2026
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 24, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from 902cec2 to f6bacb8 Compare March 24, 2026 20:17
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 24, 2026
Summary:
Pull Request resolved: facebookresearch#540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from f6bacb8 to 058502b Compare March 24, 2026 20:19
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 24, 2026
Summary:
Pull Request resolved: facebookresearch#540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from 058502b to 80e21e8 Compare March 24, 2026 20:25
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from 80e21e8 to 463da88 Compare March 25, 2026 20:41
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from 463da88 to d2e20c2 Compare March 25, 2026 20:43
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:
Pull Request resolved: facebookresearch#540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from d2e20c2 to b6d3c3b Compare March 25, 2026 20:44
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
excelle08 added a commit to excelle08/DCPerf-1 that referenced this pull request Mar 25, 2026
Summary:
Pull Request resolved: facebookresearch#540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
Summary:
Pull Request resolved: facebookresearch#540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738
@excelle08 excelle08 force-pushed the export-D97244738-to-v2-beta branch from b6d3c3b to 4dc1b94 Compare March 25, 2026 20:51
meta-codesync bot pushed a commit that referenced this pull request Mar 26, 2026
Summary:
Pull Request resolved: #540

Expose memcached's native `-e` memory file option in TaoBench. When specified,
memcached mmaps the given file for slab storage. On graceful shutdown (SIGUSR1),
state is saved to the file. On subsequent runs with the same file, cache data is
pre-loaded, drastically reducing warmup time.

Changes:
- Add `--memory-file` argument to server args
- Append `-e <path>` to memcached command when memory file specified
- Use SIGUSR1 for graceful shutdown (60s grace period) when memory file is in use
- Per-instance memory files in autoscale mode (suffix `.0`, `.1`, etc.)
- Auto-expand `/dev/shm` when tmpfs is smaller than memsize
- Add `memory_file` variable to all TaoBench job configurations
- Update README with memory file documentation

Reviewed By: gandhijayneel

Differential Revision: D97244738

fbshipit-source-id: f3d55af69593957bcbab1a184cc93ed44d5e5659
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant