Skip to content

Extstore in parts for review #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
artikell wants to merge 41 commits into
base: unstable
Choose a base branch
from
Open

Extstore in parts for review #8

artikell wants to merge 41 commits into from

Conversation

@artikell
Copy link
Owner

@artikell artikell commented Jul 3, 2025

No description provided.

artikell pushed a commit that referenced this pull request Aug 4, 2025
@gusakovy
Fix hashtablePauseAutoShrink and use it in hashtableScanDefrag (valke…
ad2c6b0 
…y-io#2257)

**Current state**
During `hashtableScanDefrag`, rehashing is paused to prevent entries
from moving, but the scan callback can still delete entries which
triggers `hashtableShrinkIfNeeded`. For example, the
`expireScanCallback` can delete expired entries.

**Issue**
This can cause the table to be resized and the old memory to be freed
while the scan is still accessing it, resulting in the following memory
access violation:

```
[err]: Sanitizer error: =================================================================
==46774==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000003100 at pc 0x0000004704d3 bp 0x7fffcb062000 sp 0x7fffcb061ff0
READ of size 1 at 0x611000003100 thread T0
    #0 0x4704d2 in isPositionFilled /home/gusakovy/Projects/valkey/src/hashtable.c:422
    #1 0x478b45 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1768
    #2 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729
    #3 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402
    #4 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297
    #5 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269
    #6 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577
    #7 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370
    #8 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513
    valkey-io#9 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543
    valkey-io#10 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291
    valkey-io#11 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139)
    valkey-io#12 0x452e39 in _start (/local/home/gusakovy/Projects/valkey/src/valkey-server+0x452e39)

0x611000003100 is located 0 bytes inside of 256-byte region [0x611000003100,0x611000003200)
freed by thread T0 here:
    #0 0x7f471a34a1e5 in __interceptor_free (/lib64/libasan.so.4+0xd81e5)
    #1 0x4aefbc in zfree_internal /home/gusakovy/Projects/valkey/src/zmalloc.c:400
    #2 0x4aeff5 in valkey_free /home/gusakovy/Projects/valkey/src/zmalloc.c:415
    #3 0x4707d2 in rehashingCompleted /home/gusakovy/Projects/valkey/src/hashtable.c:456
    #4 0x471b5b in resize /home/gusakovy/Projects/valkey/src/hashtable.c:656
    #5 0x475bff in hashtableShrinkIfNeeded /home/gusakovy/Projects/valkey/src/hashtable.c:1272
    #6 0x47704b in hashtablePop /home/gusakovy/Projects/valkey/src/hashtable.c:1448
    #7 0x47716f in hashtableDelete /home/gusakovy/Projects/valkey/src/hashtable.c:1459
    #8 0x480038 in kvstoreHashtableDelete /home/gusakovy/Projects/valkey/src/kvstore.c:847
    valkey-io#9 0x50c12c in dbGenericDeleteWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:490
    valkey-io#10 0x515f28 in deleteExpiredKeyAndPropagateWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:1831
    valkey-io#11 0x516103 in deleteExpiredKeyAndPropagate /home/gusakovy/Projects/valkey/src/db.c:1844
    valkey-io#12 0x6d8642 in activeExpireCycleTryExpire /home/gusakovy/Projects/valkey/src/expire.c:70
    valkey-io#13 0x6d8706 in expireScanCallback /home/gusakovy/Projects/valkey/src/expire.c:139
    valkey-io#14 0x478bd8 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1770
    valkey-io#15 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729
    valkey-io#16 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402
    valkey-io#17 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297
    valkey-io#18 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269
    valkey-io#19 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577
    valkey-io#20 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370
    valkey-io#21 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513
    valkey-io#22 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543
    valkey-io#23 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291
    valkey-io#24 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139)

previously allocated by thread T0 here:
    #0 0x7f471a34a753 in __interceptor_calloc (/lib64/libasan.so.4+0xd8753)
    #1 0x4ae48c in ztrycalloc_usable_internal /home/gusakovy/Projects/valkey/src/zmalloc.c:214
    #2 0x4ae757 in valkey_calloc /home/gusakovy/Projects/valkey/src/zmalloc.c:257
    #3 0x4718fc in resize /home/gusakovy/Projects/valkey/src/hashtable.c:645
    #4 0x475bff in hashtableShrinkIfNeeded /home/gusakovy/Projects/valkey/src/hashtable.c:1272
    #5 0x47704b in hashtablePop /home/gusakovy/Projects/valkey/src/hashtable.c:1448
    #6 0x47716f in hashtableDelete /home/gusakovy/Projects/valkey/src/hashtable.c:1459
    #7 0x480038 in kvstoreHashtableDelete /home/gusakovy/Projects/valkey/src/kvstore.c:847
    #8 0x50c12c in dbGenericDeleteWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:490
    valkey-io#9 0x515f28 in deleteExpiredKeyAndPropagateWithDictIndex /home/gusakovy/Projects/valkey/src/db.c:1831
    valkey-io#10 0x516103 in deleteExpiredKeyAndPropagate /home/gusakovy/Projects/valkey/src/db.c:1844
    valkey-io#11 0x6d8642 in activeExpireCycleTryExpire /home/gusakovy/Projects/valkey/src/expire.c:70
    valkey-io#12 0x6d8706 in expireScanCallback /home/gusakovy/Projects/valkey/src/expire.c:139
    valkey-io#13 0x478bd8 in hashtableScanDefrag /home/gusakovy/Projects/valkey/src/hashtable.c:1770
    valkey-io#14 0x4789c2 in hashtableScan /home/gusakovy/Projects/valkey/src/hashtable.c:1729
    valkey-io#15 0x47e3ca in kvstoreScan /home/gusakovy/Projects/valkey/src/kvstore.c:402
    valkey-io#16 0x6d9040 in activeExpireCycle /home/gusakovy/Projects/valkey/src/expire.c:297
    valkey-io#17 0x4859d2 in databasesCron /home/gusakovy/Projects/valkey/src/server.c:1269
    valkey-io#18 0x486e92 in serverCron /home/gusakovy/Projects/valkey/src/server.c:1577
    valkey-io#19 0x4637dd in processTimeEvents /home/gusakovy/Projects/valkey/src/ae.c:370
    valkey-io#20 0x4643e3 in aeProcessEvents /home/gusakovy/Projects/valkey/src/ae.c:513
    valkey-io#21 0x4647ea in aeMain /home/gusakovy/Projects/valkey/src/ae.c:543
    valkey-io#22 0x4a61fc in main /home/gusakovy/Projects/valkey/src/server.c:7291
    valkey-io#23 0x7f471957c139 in __libc_start_main (/lib64/libc.so.6+0x21139)

SUMMARY: AddressSanitizer: heap-use-after-free /home/gusakovy/Projects/valkey/src/hashtable.c:422 in isPositionFilled
Shadow bytes around the buggy address:
  0x0c227fff85d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff85e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff85f0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c227fff8600: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff8610: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
=>0x0c227fff8620:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff8630: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c227fff8640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8660: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c227fff8670: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==46774==ABORTING
```


**Solution**
Suggested solution is to also pause auto shrinking during
`hashtableScanDefrag`. I noticed that there was already a
`hashtablePauseAutoShrink` method and `pause_auto_shrink` counter, but
it wasn't actually used in `hashtableShrinkIfNeeded` so I fixed that.

**Testing**
I created a simple tcl test that (most of the times) triggers this
error, but it's a little clunky so I didn't add it as part of the PR:

```
start_server {tags {"expire hashtable defrag"}} {
    test {hashtable scan defrag on expiry} {

        r config set hz 100

        set num_keys 20
        for {set i 0} {$i < $num_keys} {incr i} {
            r set "key_$i" "value_$i"
        }

        for {set j 0} {$j < 50} {incr j} {
            set expire_keys 100
            for {set i 0} {$i < $expire_keys} {incr i} {
                # Short expiry time to ensure they expire quickly
                r psetex "expire_key_${i}_${j}" 100 "expire_value_${i}_${j}"
            }

            # Verify keys are set
            set initial_size [r dbsize]
            assert_equal $initial_size [expr $num_keys + $expire_keys]
            
            after 150
            for {set i 0} {$i < 10} {incr i} {
                r get "expire_key_${i}_${j}"
                after 10
            }
        }

        set remaining_keys [r dbsize]
        assert_equal $remaining_keys $num_keys

        # Verify server is still responsive
        assert_equal [r ping] {PONG}
    } {}
}
```
Compiling with ASAN using `make noopt SANITIZER=address valkey-server`
and running the test causes error above. Applying the fix resolves the
issue.

Signed-off-by: Yakov Gusakov <yaakov0015@gmail.com>
@kronwerk kronwerk force-pushed the extstore_in_parts branch 3 times, most recently from 9fb94a7 to 783b796 Compare October 2, 2025 08:29
artikell pushed a commit that referenced this pull request Oct 14, 2025
@hpatro
Fix memory leak with CLIENT LIST/KILL duplicate filters (valkey-io#2362)
155b0bb 
With valkey-io#1401, we introduced additional filters to CLIENT LIST/KILL
subcommand. The intended behavior was to pick the last value of the
filter. However, we introduced memory leak for all the preceding
filters.

Before this change:
```
> CLIENT LIST IP 127.0.0.1 IP 127.0.0.1
id=4 addr=127.0.0.1:37866 laddr=127.0.0.1:6379 fd=10 name= age=0 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=21 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=16989 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=49 tot-net-out=0 tot-cmds=0
```
Leak:
```
Direct leak of 11 byte(s) in 1 object(s) allocated from:
    #0 0x7f2901aa557d in malloc (/lib64/libasan.so.4+0xd857d)
    #1 0x76db76 in ztrymalloc_usable_internal /workplace/harkrisp/valkey/src/zmalloc.c:156
    #2 0x76db76 in zmalloc_usable /workplace/harkrisp/valkey/src/zmalloc.c:200
    #3 0x4c4121 in _sdsnewlen.constprop.230 /workplace/harkrisp/valkey/src/sds.c:113
    #4 0x4dc456 in parseClientFiltersOrReply.constprop.63 /workplace/harkrisp/valkey/src/networking.c:4264
    #5 0x4bb9f7 in clientListCommand /workplace/harkrisp/valkey/src/networking.c:4600
    #6 0x641159 in call /workplace/harkrisp/valkey/src/server.c:3772
    #7 0x6431a6 in processCommand /workplace/harkrisp/valkey/src/server.c:4434
    #8 0x4bfa9b in processCommandAndResetClient /workplace/harkrisp/valkey/src/networking.c:3571
    valkey-io#9 0x4bfa9b in processInputBuffer /workplace/harkrisp/valkey/src/networking.c:3702
    valkey-io#10 0x4bffa3 in readQueryFromClient /workplace/harkrisp/valkey/src/networking.c:3812
    valkey-io#11 0x481015 in callHandler /workplace/harkrisp/valkey/src/connhelpers.h:79
    valkey-io#12 0x481015 in connSocketEventHandler.lto_priv.394 /workplace/harkrisp/valkey/src/socket.c:301
    valkey-io#13 0x7d3fb3 in aeProcessEvents /workplace/harkrisp/valkey/src/ae.c:486
    valkey-io#14 0x7d4d44 in aeMain /workplace/harkrisp/valkey/src/ae.c:543
    valkey-io#15 0x453925 in main /workplace/harkrisp/valkey/src/server.c:7319
    valkey-io#16 0x7f2900cd7139 in __libc_start_main (/lib64/libc.so.6+0x21139)
```

Note: For filter ID / NOT-ID we group all the option and perform
filtering whereas for remaining filters we only pick the last filter
option.

---------

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
@kronwerk kronwerk force-pushed the extstore_in_parts branch 2 times, most recently from 5f7f960 to 7abf0a2 Compare October 17, 2025 15:19
@kronwerk kronwerk force-pushed the extstore_in_parts branch 3 times, most recently from bf0ccfe to 8fd5791 Compare October 25, 2025 19:52
roshkhatri and others added 20 commits November 26, 2025 15:19
@roshkhatri @ranshid
Adds HGETDEL Support to Valkey (valkey-io#2851)
56ab3c4 
Fixes this: valkey-io#2850
Adds support for HGETDEL to Valkey and aligns with Redis 8.0 feature.
Maintains syntax compatibility
Retrieves all the values, and null if fields dont exists and deletes
once retrieved.
```
127.0.0.1:6379> HGETDEL key FIELDS numfields field [ field ... ]
```
```
127.0.0.1:6379> HSET foo field1 bar1 field2 bar2 field3 bar3
(integer) 3
127.0.0.1:6379> HGETDEL foo FIELDS 1 field2
1) "bar2"
127.0.0.1:6379> HGETDEL foo FIELDS 1 field2
1) (nil)
127.0.0.1:6379> HGETALL foo
1) "field1"
2) "bar1"
3) "field3"
4) "bar3"
127.0.0.1:6379>  HGETDEL foo FIELDS 2 field2 field3
1) (nil)
2) "bar3"
127.0.0.1:6379> HGETALL foo
1) "field1"
2) "bar1"
127.0.0.1:6379> HGETDEL foo FIELDS 3 field1 non-exist-field
(error) ERR syntax error
127.0.0.1:6379> HGETDEL foo FIELDS 2 field1 non-exist-field
1) "bar1"
2) (nil)
127.0.0.1:6379> HGETALL foo
(empty array)
127.0.0.1:6379> 
```

---------

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
@zhijun42
Cluster: Optimize slot bitmap iteration from per-bit to 64-bit word s…
faac14a 
…can (valkey-io#2781)

In functions `clusterSendFailoverAuthIfNeeded` and
`clusterProcessPacket`, we iterate through **every slot bit**
sequentially in the form of `for (int j = 0; j < CLUSTER_SLOTS; j++)`,
performing 16384 checks even when only a few bits were set, and thus
causing unnecessary loop overhead.

This is particularly wasteful in function
`clusterSendFailoverAuthIfNeeded` where we need to ensure the sender's
claimed slots all have up-to-date config epoch. Usually healthy senders
would meet such condition, and thus we normally need to exhaust the for
loop of 16384 checks.

The proposed new implementation loads 64 bits (8 byte word) at a time
and skips empty words completely, therefore only performing 256 checks.

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
@enjoy-binbin
Fix discarded-qualifiers warnings reported by fedorarawhide (valkey-i…
d16788e 
…o#2874)

fedorarawhide CI reports these warnings:
```
networking.c: In function 'afterErrorReply':
networking.c:821:30: error: initialization discards 'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
  821 |             char *spaceloc = memchr(s, ' ', len < 32 ? len : 32);
```

Signed-off-by: Binbin <binloveplay1314@qq.com>
@zuiderkwast
Fix persisting missing make variables (valkey-io#2881)
761fba2 
Persist USE_FAST_FLOAT and PROG_SUFFIX to prevent a complete rebuild
next time someone types make or make test without specifying variables.

Fixes valkey-io#2880

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@enjoy-binbin
Add psync offset range information to the failed psync log (valkey-io…
a087cc1 
…#2877)

Although we can infer this infomartion from the replica logs,
i think this still would be useful to see this information directly
in the primary logs.

So now we can see the psync offset range when psync fails and then
we can analyze and adjust the value of repl-backlog-size.

Signed-off-by: Binbin <binloveplay1314@qq.com>
@enjoy-binbin
Handle failed psync log when there is no replication backlog (valkey-…
4a0e20b 
…io#2886)

This crash was introduced in valkey-io#2877, we will crash when there is no
replication backlog.

Signed-off-by: Binbin <binloveplay1314@qq.com>
@dvkashapov
Make all ACL categories explicit in JSON files (valkey-io#2887)
825d19f 
Resolves valkey-io#417

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
@zhijun42
Refactor TCL reference to support running tests with CMake (valkey-io…
3fd0942 
…#2816)

Historically, Valkey’s TCL test suite expected all binaries
(src/valkey-server, src/valkey-cli, src/valkey-benchmark, etc.) to exist
under the src/ directory. This PR enables Valkey TCL tests to run
seamlessly after a CMake build — no manual symlinks or make build
required.

The test framework accepts a new environment variable `VALKEY_BIN_DIR`
to look for the binaries.

CMake will copy all TCL test entrypoints (runtest, runtest-cluster,
etc.) into the CMake build dir (e.g. `cmake-build-debug`) and insert
`VALKEY_BIN_DIR` into these. Now we can either do
./cmake-build-debug/runtest at the project root or ./runtest at the
Cmake dir to run all tests.

A new CMake post-build target prints a friendly reminder after
successful builds, guiding developers on how to run tests with their
CMake binaries:

```
Hint: It is a good idea to run tests with your CMake-built binaries ;)
      ./cmake-build-debug/runtest

Build finished
```

A helper TCL script `tests/support/set_executable_path.tcl` is added to
support this change, which gets called by all test entrypoints:
`runtest`, `runtest-cluster`, `runtest-sentinel`.

---------

Signed-off-by: Zhijun <dszhijun@gmail.com>
@zhijun42
Build: Support make test when PROG_SUFFIX is used (valkey-io#2885)
853d111 
Closes valkey-io#2883

Support a new environment variable `VALKEY_PROG_SUFFIX` in the test
framework, which can be used for running tests if the binaries are
compiled with a program suffix. For example, if the binaries are
compiled using `make PROG_SUFFIX=-alt` to produce binaries named
valkey-server-alt, valkey-cli-alt, etc., run the tests against these
binaries using `VALKEY_PROG_SUFFIX=-alt ./runtest` or simply using `make
test`.

Now the test with the make variable `PROG_SUFFIX` works well.

```
% make PROG_SUFFIX="-alt"
		...
		...
		CC trace/trace_aof.o
    LINK valkey-server-alt
    INSTALL valkey-sentinel-alt
    CC valkey-cli.o
    CC serverassert.o
    CC cli_common.o
    CC cli_commands.o
    LINK valkey-cli-alt
    CC valkey-benchmark.o
    LINK valkey-benchmark-alt
    INSTALL valkey-check-rdb-alt
    INSTALL valkey-check-aof-alt

Hint: It's a good idea to run 'make test' ;)
% 
% make test                                                                
cd src && /Library/Developer/CommandLineTools/usr/bin/make test
    CC Makefile.dep
    CC release.o
    LINK valkey-server-alt
    INSTALL valkey-check-aof-alt
    INSTALL valkey-check-rdb-alt
    LINK valkey-cli-alt
    LINK valkey-benchmark-alt
Cleanup: may take some time... OK
Starting test server at port 21079
[ready]: 39435
Testing unit/pubsub
```

Signed-off-by: Zhijun <dszhijun@gmail.com>
@JimB123 @ranshid
Refactor of LFU/LRU code for modularity (valkey-io#2857)
1b5f245 
General cleanup on LRU/LFU code. Improve modularity and maintainability.

Specifically:
* Consolidates the mathematical logic for LRU/LFU into `lrulfu.c`, with
an API in `lrulfu.h`. Knowledge of the LRU/LFU implementation was
previously spread out across `db.c`, `evict.c`, `object.c`, `server.c`,
and `server.h`.
* Separates knowledge of the LRU from knowledge of the object containing
the LRU value. `lrulfu.c` knows about the LRU/LFU algorithms, without
knowing about the `robj`. `object.c` knows about the `robj` without
knowing about the details of the LRU/LFU algorithms.
* Eliminated `server.lruclock`, instead using `server.unixtime`. This
also eliminates the periodic need to call `mstime()` to maintain the lru
clock.
* Fixed a minor computation bug in the old `LFUTimeElapsed` function
(off by 1 after rollover).
* Eliminate specific IF checks for rollover, using defined behavior for
unsigned rollover instead.
* Fixed a bug in `debug.c` which would perform LFU modification on an
LRU value.

---------

Signed-off-by: Jim Brunner <brunnerj@amazon.com>
Co-authored-by: Ran Shidlansik <ranshid@amazon.com>
@roshkhatri
[Test Fix] flaky benchmark test for warmup (valkey-io#2890)
2e8fba4 
Fixes: valkey-io#2859
Increased the warmup to 2 sec so we can verify that it runs more number
of commands than the actual benchmark.

Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
@MuradSh
Replace strcmp with byte-by-byte comparison in valkey-check-aof (valk…
c8341d4 
…ey-io#2809)

Fixes valkey-io#2792

Replace strcmp with byte-by-byte comparison to avoid accidental
heap-buffer-overflow errors.

Signed-off-by: murad shahmammadli <shmurad@amazon.com>
Co-authored-by: murad shahmammadli <shmurad@amazon.com>
@hpatro
Send duplicate multi meet packet only for node which supports it (val…
8ec9381 
…key-io#2840)

This prevents crashes on the older nodes in mixed clusters where some
nodes are running 8.0 or older. Mixed clusters often exist temporarily
during rolling upgrades.

Fixes: valkey-io#2341 

Signed-off-by: Harkrishn Patro <harkrisp@amazon.com>
@rainsupreme @zuiderkwast
Add safe iterator tracking in hashtable to prevents invalid access on…
70196ee 
… hashtable delete (valkey-io#2807)

This makes it safe to delete hashtable while a safe iterator is
iterating it. This currently isn't possible, but this improvement is
required for fork-less replication valkey-io#1754 which is being actively
worked on.

We discussed these issues in valkey-io#2611 which guards against a different but
related issue: calling hashtableNext again after it has already returned
false.

I implemented a singly linked list that hashtable uses to track its
current safe iterators. It is used to invalidate all associated safe
iterators when the hashtable is released. A singly linked list is
acceptable because the list length is always very small - typically zero
and no more than a handful.

Also, renames the internal functions:

    hashtableReinitIterator -> hashtableRetargetIterator
    hashtableResetIterator -> hashtableCleanupIterator

---------

Signed-off-by: Rain Valentine <rsg000@gmail.com>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@kronwerk
new configs
47b6fb2 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
1. EXTERNAL_DATA LOADED
98c754d 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
2. MODULE LOAD
c08471f 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
3. EXTERNAL_DATA STATS
f66e630 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
4. EXTERNAL_DATA INIT
4126461 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
5. EXTERNAL_DATA DROP
6b4b352 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
kronwerk added 17 commits December 4, 2025 15:49
@kronwerk
6. key read
43a2a67 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
7.0. fix CI tests
7b0176c 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
7.1. remove value from filter set
2f0e388 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
7.2. add single-sharded cluster test (just in case)
516d4b6 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
7.3. add keys command option and processing
2940788 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
7.4. add scan command option and processing
137e414 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
merge storage and filter modules into a single external data module
8e6df24 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
refactor get\scan\keys flags and behaviour a bit
1a8174b 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
remove iterator id
d8ab945 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
8. set key
aabc62b 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
9. del key
a0b3f3f 
Signed-off-by: kronwerk <kronwerk@users.noreply.github.com>
@kronwerk
0. add some tests and fixes
e16b2e3
@kronwerk
0. refactor test module to fix non-blocking test
ba388ad
@kronwerk
0. merge setro\dropro for filter\storage into common methods
5eff73a
@kronwerk
10. expire and evict to external storage
b73f777
@kronwerk
11. flushall/flushdb support
0547884
@kronwerk
12. swapdb support
22d5a63
@kronwerk kronwerk force-pushed the extstore_in_parts branch 2 times, most recently from 6d18a58 to 113aff2 Compare December 8, 2025 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.