diff --git a/.gitignore b/.gitignore index fafc1b6..8f2501b 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ target/ -/*.sqlite* \ No newline at end of file +/*.sqlite* +rawkakani_db.pem \ No newline at end of file diff --git a/CONNECTION_ISSUES_EXPLANATION.md b/CONNECTION_ISSUES_EXPLANATION.md new file mode 100644 index 0000000..3d6850f --- /dev/null +++ b/CONNECTION_ISSUES_EXPLANATION.md @@ -0,0 +1,188 @@ +# Why PostgreSQL Connection Issues Are Happening + +## Summary of the Issue + +Based on your logs from Dec 28, 2025, you're experiencing PostgreSQL connection failures that occur when: + +1. **PostgreSQL server process crashes or restarts** +2. **Connection pool tries to use dead connections** +3. **PostgreSQL cannot create relation-cache files** + +## Root Causes + +### 1. **PostgreSQL Server Process Crash** (Primary Cause) + +**Log Evidence:** +``` +WARNING: terminating connection because of crash of another server process +``` + +**What This Means:** +- Another PostgreSQL backend process crashed (not your DenoKV process) +- PostgreSQL automatically terminates all connections when a backend process crashes +- This is a **safety mechanism** to prevent data corruption + +**Why This Happens:** +- **Memory issues**: PostgreSQL process ran out of memory (OOM killer) +- **Disk I/O errors**: Storage problems causing process crashes +- **PostgreSQL bugs**: Rare but possible in certain versions +- **Resource exhaustion**: CPU/memory limits reached +- **System instability**: Hardware or OS issues + +**How to Diagnose:** +```bash +# Check PostgreSQL logs for crash details +sudo tail -100 /var/log/postgresql/postgresql-*.log | grep -i "crash\|fatal\|panic" + +# Check system logs for OOM kills +sudo dmesg | grep -i "out of memory\|killed process" + +# Check PostgreSQL process status +sudo systemctl status postgresql +``` + +### 2. **Connection Pool Using Dead Connections** + +**Log Evidence:** +``` +WARN deadpool.postgres] Connection error: connection closed +INFO deadpool.postgres] Connection could not be recycled: Connection closed +``` + +**What This Means:** +- The connection pool (deadpool) had connections that were **already dead** +- When PostgreSQL crashed, it closed all connections +- deadpool tried to reuse these dead connections +- deadpool detected they were closed and tried to recycle them +- But recycling failed because the connection was already terminated + +**Why This Happens:** +- **No connection health checks**: The pool doesn't validate connections before use +- **Stale connections**: Connections remain in pool after server crash +- **No automatic recovery**: Pool doesn't automatically recreate dead connections + +**The Fix (Already Implemented):** +- Added connection validation before use (`SELECT 1` query) +- Added retry logic with exponential backoff +- Added automatic connection recreation on failure + +### 3. **Relation-Cache Initialization File Errors** + +**Log Evidence:** +``` +WARNING: could not create relation-cache initialization file "base/16385/pg_internal.init" +WARNING: could not create relation-cache initialization file "global/pg_internal.init" +``` + +**What This Means:** +- PostgreSQL tries to create cache files for faster query planning +- These files are **optional performance optimizations** +- Failure to create them is **not critical** - PostgreSQL works without them +- This is a **warning**, not an error + +**Why This Happens:** +- **File system permissions**: PostgreSQL user doesn't have write access +- **Disk space issues**: No space to create cache files +- **Read-only file system**: Database directory mounted read-only +- **PostgreSQL recovery mode**: Server in recovery and can't write cache + +**Impact:** +- **Minimal**: Queries work but may be slightly slower +- **No data loss**: This doesn't affect data integrity +- **Can be ignored**: This is a non-critical warning + +## Why It Happened on Dec 28 (11 Days After Startup) + +The server started successfully on **Dec 17** and ran fine for 11 days. Then on **Dec 28**, you saw these errors. This suggests: + +1. **PostgreSQL server restarted/crashed** on Dec 28 +2. **All existing connections were terminated** by PostgreSQL +3. **Connection pool had stale connections** that were no longer valid +4. **Application tried to use dead connections** → errors occurred + +## What Happens Now (After Our Fixes) + +With the improvements we've implemented: + +1. **Connection Validation**: Every connection is tested with `SELECT 1` before use +2. **Automatic Retry**: Transient errors trigger automatic retries (up to 3 attempts) +3. **Exponential Backoff**: Retries wait progressively longer (100ms, 200ms, 400ms) +4. **Better Error Detection**: We detect transient vs permanent errors +5. **Connection Recreation**: Dead connections are automatically replaced + +**Result**: The application will now automatically recover from PostgreSQL crashes without user intervention. + +## Recommendations + +### 1. **Investigate PostgreSQL Crashes** + +Find out why PostgreSQL crashed: + +```bash +# Check PostgreSQL error log +sudo tail -200 /var/log/postgresql/postgresql-*.log + +# Check for OOM kills +sudo dmesg | grep -i "killed process.*postgres" + +# Check system resources +free -h +df -h +``` + +### 2. **Monitor PostgreSQL Health** + +Set up monitoring for: +- PostgreSQL process crashes +- Memory usage +- Disk space +- Connection counts + +### 3. **Configure PostgreSQL for Stability** + +```sql +-- Increase shared_buffers if you have enough RAM +ALTER SYSTEM SET shared_buffers = '256MB'; + +-- Set connection limits +ALTER SYSTEM SET max_connections = 100; + +-- Enable connection timeouts +ALTER SYSTEM SET idle_in_transaction_session_timeout = '10min'; +``` + +### 4. **Set Up Automatic Restart** + +Ensure PostgreSQL auto-restarts on crash: + +```bash +# For systemd +sudo systemctl enable postgresql +sudo systemctl edit postgresql +# Add: +# [Service] +# Restart=always +# RestartSec=5 +``` + +### 5. **Fix Relation-Cache Warnings (Optional)** + +If you want to eliminate the warnings: + +```bash +# Check PostgreSQL data directory permissions +sudo ls -la /var/lib/postgresql/*/base/ + +# Ensure PostgreSQL user can write +sudo chown -R postgres:postgres /var/lib/postgresql/ +sudo chmod 700 /var/lib/postgresql/*/base/ +``` + +## Conclusion + +**The connection issues are caused by:** +1. PostgreSQL server process crashing (primary) +2. Connection pool not detecting dead connections (secondary - now fixed) +3. PostgreSQL cache file warnings (cosmetic - can be ignored) + +**The application will now handle these gracefully** with automatic retries and connection recovery. However, you should still investigate why PostgreSQL is crashing to prevent future issues. diff --git a/Cargo.lock b/Cargo.lock index 25cef96..5d00852 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -79,7 +79,7 @@ version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5ca11d4be1bab0c8bc8734a9aa7bf4ee8316d462a08c6ac5052f888fef5b494b" dependencies = [ - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -89,7 +89,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f0699d10d2f4d628a98ee7b57b289abbc98ff3bad977cb3152709d4bf2330628" dependencies = [ "anstyle", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -624,6 +624,12 @@ version = "3.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f30e7476521f6f8af1a1c4c0b8cc94f0bee37d91763d0ca2665f299b6cd8aec" +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + [[package]] name = "bytes" version = "1.5.0" @@ -770,6 +776,40 @@ dependencies = [ "typenum", ] +[[package]] +name = "deadpool" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "421fe0f90f2ab22016f32a9881be5134fdd71c65298917084b0c7477cbc3856e" +dependencies = [ + "async-trait", + "deadpool-runtime", + "num_cpus", + "retain_mut", + "tokio", +] + +[[package]] +name = "deadpool-postgres" +version = "0.10.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "836a24a9d49deefe610b8b60c767a7412e9a931d79a89415cd2d2d71630ca8d7" +dependencies = [ + "deadpool", + "log", + "tokio", + "tokio-postgres", +] + +[[package]] +name = "deadpool-runtime" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "092966b41edc516079bdf31ec78a2e0588d1d0c08f78b91d8307215928642b2b" +dependencies = [ + "tokio", +] + [[package]] name = "deno_error" version = "0.7.0" @@ -810,6 +850,7 @@ dependencies = [ "clap", "constant_time_eq", "deno_error", + "denokv_postgres", "denokv_proto", "denokv_remote", "denokv_sqlite", @@ -823,7 +864,7 @@ dependencies = [ "log", "num-bigint", "prost", - "rand", + "rand 0.8.5", "reqwest", "rusqlite", "serde", @@ -835,6 +876,30 @@ dependencies = [ "v8_valueserializer", ] +[[package]] +name = "denokv_postgres" +version = "0.13.0" +dependencies = [ + "async-stream", + "async-trait", + "bytes", + "chrono", + "clap", + "deadpool-postgres", + "deno_error", + "denokv_proto", + "futures", + "log", + "rand 0.8.5", + "rusqlite", + "serde", + "serde_json", + "thiserror 2.0.8", + "tokio", + "tokio-postgres", + "uuid", +] + [[package]] name = "denokv_proto" version = "0.13.0" @@ -864,7 +929,7 @@ dependencies = [ "http 1.1.0", "log", "prost", - "rand", + "rand 0.8.5", "reqwest", "serde", "serde_json", @@ -888,7 +953,7 @@ dependencies = [ "hex", "log", "num-bigint", - "rand", + "rand 0.8.5", "rusqlite", "serde_json", "thiserror 2.0.8", @@ -975,9 +1040,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7c18ee0ed65a5f1f81cac6b1d213b69c35fa47d4252ad41f1486dbd8226fe36e" dependencies = [ "libc", - "windows-sys", + "windows-sys 0.48.0", ] +[[package]] +name = "fallible-iterator" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7" + [[package]] name = "fallible-iterator" version = "0.3.0" @@ -1139,7 +1210,19 @@ checksum = "fe9006bed769170c11f845cf00c7c1e9092aeb3f268e007c3e760ac68008070f" dependencies = [ "cfg-if", "libc", - "wasi", + "wasi 0.11.0+wasi-snapshot-preview1", +] + +[[package]] +name = "getrandom" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26145e563e54f2cadc477553f1ec5ee650b00862f0a58bcd12cbdc5f0ea2d2f4" +dependencies = [ + "cfg-if", + "libc", + "r-efi", + "wasi 0.14.7+wasi-0.2.4", ] [[package]] @@ -1472,7 +1555,7 @@ checksum = "cb0889898416213fab133e1d33a0e5858a48177452750691bde3666d0fdbaf8b" dependencies = [ "hermit-abi", "rustix", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -1507,9 +1590,20 @@ checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646" [[package]] name = "libc" -version = "0.2.150" +version = "0.2.176" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "58f929b4d672ea937a23a1ab494143d968337a5f47e56d0815df1e0890ddf174" + +[[package]] +name = "libredox" +version = "0.1.10" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "89d92a4743f9a61002fae18374ed11e7973f530cb3a3255fb354818118b2203c" +checksum = "416f7e718bdb06000964960ffa43b4335ad4012ae8b99060261aa4a8088d5ccb" +dependencies = [ + "bitflags 2.6.0", + "libc", + "redox_syscall 0.5.17", +] [[package]] name = "libsqlite3-sys" @@ -1588,8 +1682,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3dce281c5e46beae905d4de1870d8b1509a9142b62eedf18b443b011ca8343d0" dependencies = [ "libc", - "wasi", - "windows-sys", + "wasi 0.11.0+wasi-snapshot-preview1", + "windows-sys 0.48.0", ] [[package]] @@ -1689,9 +1783,9 @@ checksum = "4c42a9226546d68acdd9c0a280d17ce19bfe27a46bf68784e4066115788d008e" dependencies = [ "cfg-if", "libc", - "redox_syscall", + "redox_syscall 0.4.1", "smallvec", - "windows-targets", + "windows-targets 0.48.5", ] [[package]] @@ -1710,6 +1804,25 @@ dependencies = [ "indexmap 2.1.0", ] +[[package]] +name = "phf" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf" +dependencies = [ + "phf_shared", + "serde", +] + +[[package]] +name = "phf_shared" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266" +dependencies = [ + "siphasher", +] + [[package]] name = "pin-project" version = "1.1.3" @@ -1748,6 +1861,35 @@ version = "0.3.27" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "26072860ba924cbfa98ea39c8c19b4dd6a4a25423dbdf219c1eca91aa0cf6964" +[[package]] +name = "postgres-protocol" +version = "0.6.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fbef655056b916eb868048276cfd5d6a7dea4f81560dfd047f97c8c6fe3fcfd4" +dependencies = [ + "base64 0.22.0", + "byteorder", + "bytes", + "fallible-iterator 0.2.0", + "hmac", + "md-5", + "memchr", + "rand 0.9.2", + "sha2", + "stringprep", +] + +[[package]] +name = "postgres-types" +version = "0.2.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77a120daaabfcb0e324d5bf6e411e9222994cb3795c79943a0ef28ed27ea76e4" +dependencies = [ + "bytes", + "fallible-iterator 0.2.0", + "postgres-protocol", +] + [[package]] name = "powerfmt" version = "0.2.0" @@ -1841,6 +1983,12 @@ dependencies = [ "proc-macro2", ] +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + [[package]] name = "rand" version = "0.8.5" @@ -1848,8 +1996,18 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404" dependencies = [ "libc", - "rand_chacha", - "rand_core", + "rand_chacha 0.3.1", + "rand_core 0.6.4", +] + +[[package]] +name = "rand" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1" +dependencies = [ + "rand_chacha 0.9.0", + "rand_core 0.9.3", ] [[package]] @@ -1859,7 +2017,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" dependencies = [ "ppv-lite86", - "rand_core", + "rand_core 0.6.4", +] + +[[package]] +name = "rand_chacha" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb" +dependencies = [ + "ppv-lite86", + "rand_core 0.9.3", ] [[package]] @@ -1868,7 +2036,16 @@ version = "0.6.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" dependencies = [ - "getrandom", + "getrandom 0.2.11", +] + +[[package]] +name = "rand_core" +version = "0.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "99d9a13982dcf210057a8a78572b2217b667c3beacbf3a0d8b454f6f82837d38" +dependencies = [ + "getrandom 0.3.3", ] [[package]] @@ -1880,6 +2057,15 @@ dependencies = [ "bitflags 1.3.2", ] +[[package]] +name = "redox_syscall" +version = "0.5.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5407465600fb0548f1442edf71dd20683c6ed326200ace4b1ef0763521bb3b77" +dependencies = [ + "bitflags 2.6.0", +] + [[package]] name = "regex" version = "1.10.2" @@ -1946,6 +2132,12 @@ dependencies = [ "winreg", ] +[[package]] +name = "retain_mut" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4389f1d5789befaf6029ebd9f7dac4af7f7e3d61b69d4f30e2ac02b57e7712b0" + [[package]] name = "ring" version = "0.16.20" @@ -1968,11 +2160,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fb0205304757e5d899b9c2e448b867ffd03ae7f988002e47cd24954391394d0b" dependencies = [ "cc", - "getrandom", + "getrandom 0.2.11", "libc", "spin 0.9.8", "untrusted 0.9.0", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -1982,7 +2174,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "165ca6e57b20e1351573e3729b958bc62f0e48025386970b6e4d29e7a7e71f3f" dependencies = [ "bitflags 2.6.0", - "fallible-iterator", + "fallible-iterator 0.3.0", "fallible-streaming-iterator", "hashlink", "libsqlite3-sys", @@ -2014,7 +2206,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -2068,7 +2260,7 @@ version = "0.1.22" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0c3733bf4cf7ea0880754e19cb5a462007c4a8c1914bff372ccc95b464f1df88" dependencies = [ - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -2206,6 +2398,12 @@ dependencies = [ "libc", ] +[[package]] +name = "siphasher" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56199f7ddabf13fe5074ce809e7d3f42b42ae711800501b5b16ea82ad029c39d" + [[package]] name = "slab" version = "0.4.9" @@ -2238,7 +2436,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7b5fac59a5cb5dd637972e5fca70daf0523c9067fcdc4842f053dae04a18f8e9" dependencies = [ "libc", - "windows-sys", + "windows-sys 0.48.0", +] + +[[package]] +name = "socket2" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "233504af464074f9d066d7b5416c5f9b894a5862a6506e306f7b816cdd6f1807" +dependencies = [ + "libc", + "windows-sys 0.59.0", ] [[package]] @@ -2253,6 +2461,17 @@ version = "0.9.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6980e8d7511241f8acf4aebddbb1ff938df5eebe98691418c4468d0b72a96a67" +[[package]] +name = "stringprep" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b4df3d392d81bd458a8a621b8bffbd2302a12ffe288a9d931670948749463b1" +dependencies = [ + "unicode-bidi", + "unicode-normalization", + "unicode-properties", +] + [[package]] name = "strsim" version = "0.10.0" @@ -2290,9 +2509,9 @@ checksum = "7ef1adac450ad7f4b3c28589471ade84f25f731a7a0fe30d71dfa9f60fd808e5" dependencies = [ "cfg-if", "fastrand 2.0.1", - "redox_syscall", + "redox_syscall 0.4.1", "rustix", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -2405,7 +2624,7 @@ dependencies = [ "signal-hook-registry", "socket2 0.5.5", "tokio-macros", - "windows-sys", + "windows-sys 0.48.0", ] [[package]] @@ -2419,6 +2638,32 @@ dependencies = [ "syn", ] +[[package]] +name = "tokio-postgres" +version = "0.7.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a156efe7fff213168257853e1dfde202eed5f487522cbbbf7d219941d753d853" +dependencies = [ + "async-trait", + "byteorder", + "bytes", + "fallible-iterator 0.2.0", + "futures-channel", + "futures-util", + "log", + "parking_lot", + "percent-encoding", + "phf", + "pin-project-lite", + "postgres-protocol", + "postgres-types", + "rand 0.9.2", + "socket2 0.6.0", + "tokio", + "tokio-util", + "whoami", +] + [[package]] name = "tokio-rustls" version = "0.23.4" @@ -2552,6 +2797,12 @@ dependencies = [ "tinyvec", ] +[[package]] +name = "unicode-properties" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e70f2a8b45122e719eb623c01822704c4e0907e7e426a05927e1a1cfff5b75d0" + [[package]] name = "untrusted" version = "0.7.1" @@ -2593,7 +2844,7 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "88ad59a7560b41a70d191093a945f0b87bc1deeda46fb237479708a1d6b6cdfc" dependencies = [ - "getrandom", + "getrandom 0.2.11", "serde", ] @@ -2645,6 +2896,30 @@ version = "0.11.0+wasi-snapshot-preview1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423" +[[package]] +name = "wasi" +version = "0.14.7+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "883478de20367e224c0090af9cf5f9fa85bed63a95c1abf3afc5c083ebc06e8c" +dependencies = [ + "wasip2", +] + +[[package]] +name = "wasip2" +version = "1.0.1+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0562428422c63773dad2c345a1882263bbf4d65cf3f42e90921f787ef5ad58e7" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasite" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8dad83b4f25e74f184f64c43b150b91efe7647395b42289f38e50566d82855b" + [[package]] name = "wasm-bindgen" version = "0.2.88" @@ -2744,6 +3019,17 @@ dependencies = [ "untrusted 0.9.0", ] +[[package]] +name = "whoami" +version = "1.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d4a4db5077702ca3015d3d02d74974948aba2ad9e12ab7df718ee64ccd7e97d" +dependencies = [ + "libredox", + "wasite", + "web-sys", +] + [[package]] name = "winapi" version = "0.3.9" @@ -2781,7 +3067,16 @@ version = "0.48.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9" dependencies = [ - "windows-targets", + "windows-targets 0.48.5", +] + +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets 0.52.6", ] [[package]] @@ -2790,13 +3085,29 @@ version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c" dependencies = [ - "windows_aarch64_gnullvm", - "windows_aarch64_msvc", - "windows_i686_gnu", - "windows_i686_msvc", - "windows_x86_64_gnu", - "windows_x86_64_gnullvm", - "windows_x86_64_msvc", + "windows_aarch64_gnullvm 0.48.5", + "windows_aarch64_msvc 0.48.5", + "windows_i686_gnu 0.48.5", + "windows_i686_msvc 0.48.5", + "windows_x86_64_gnu 0.48.5", + "windows_x86_64_gnullvm 0.48.5", + "windows_x86_64_msvc 0.48.5", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm 0.52.6", + "windows_aarch64_msvc 0.52.6", + "windows_i686_gnu 0.52.6", + "windows_i686_gnullvm", + "windows_i686_msvc 0.52.6", + "windows_x86_64_gnu 0.52.6", + "windows_x86_64_gnullvm 0.52.6", + "windows_x86_64_msvc 0.52.6", ] [[package]] @@ -2805,42 +3116,90 @@ version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8" +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + [[package]] name = "windows_aarch64_msvc" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc" +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + [[package]] name = "windows_i686_gnu" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e" +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + [[package]] name = "windows_i686_msvc" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406" +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + [[package]] name = "windows_x86_64_gnu" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e" +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + [[package]] name = "windows_x86_64_gnullvm" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc" +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + [[package]] name = "windows_x86_64_msvc" version = "0.48.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538" +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + [[package]] name = "winreg" version = "0.52.0" @@ -2848,9 +3207,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a277a57398d4bfa075df44f501a17cfdf8542d224f0d36095a2adc7aee4ef0a5" dependencies = [ "cfg-if", - "windows-sys", + "windows-sys 0.48.0", ] +[[package]] +name = "wit-bindgen" +version = "0.46.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f17a85883d4e6d00e8a97c586de764dabcc06133f7f1d55dce5cdc070ad7fe59" + [[package]] name = "wtf8" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index ce6326b..ddd2d03 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,5 +1,5 @@ [workspace] -members = ["denokv", "proto", "remote", "sqlite", "timemachine"] +members = ["denokv", "proto", "remote", "sqlite", "postgres", "timemachine"] resolver = "2" [workspace.package] @@ -11,6 +11,7 @@ edition = "2021" [workspace.dependencies] denokv_proto = { version = "0.13.0", path = "./proto" } denokv_sqlite = { version = "0.13.0", path = "./sqlite" } +denokv_postgres = { version = "0.13.0", path = "./postgres" } denokv_remote = { version = "0.13.0", path = "./remote" } denokv_timemachine = { version = "0.13.0", path = "./timemachine" } diff --git a/DIAGNOSIS_DISK_FULL.md b/DIAGNOSIS_DISK_FULL.md new file mode 100644 index 0000000..4ca8335 --- /dev/null +++ b/DIAGNOSIS_DISK_FULL.md @@ -0,0 +1,156 @@ +# CRITICAL: Disk Full Issue - Root Cause of PostgreSQL Crashes + +## Immediate Problem + +Your root filesystem is **100% full**: +``` +/dev/mapper/rocky-lvroot 8.8G 8.8G 20K 100% / +``` + +**This is almost certainly the cause of your PostgreSQL crashes!** + +When a disk is full: +- PostgreSQL cannot write to WAL (Write-Ahead Log) +- PostgreSQL cannot create temporary files +- PostgreSQL cannot write relation-cache files (explains your warnings) +- PostgreSQL processes can crash when they can't write + +## Immediate Actions Required + +### 1. Find What's Using Disk Space + +```bash +# Find largest directories +sudo du -h --max-depth=1 / | sort -hr | head -20 + +# Check PostgreSQL data directory size +sudo du -sh /var/lib/pgsql/* 2>/dev/null || sudo du -sh /var/lib/postgresql/* 2>/dev/null + +# Check log files +sudo du -sh /var/log/* | sort -hr | head -10 + +# Check for large files +sudo find / -type f -size +100M 2>/dev/null | head -20 +``` + +### 2. Find PostgreSQL Logs + +PostgreSQL logs might be in different locations: + +```bash +# Check PostgreSQL configuration for log location +sudo -u postgres psql -c "SHOW log_directory;" +sudo -u postgres psql -c "SHOW log_filename;" + +# Common locations: +ls -lh /var/lib/pgsql/*/data/log/ 2>/dev/null +ls -lh /var/lib/postgresql/*/log/ 2>/dev/null +ls -lh /var/log/postgresql/ 2>/dev/null +journalctl -u postgresql* -n 100 +``` + +### 3. Free Up Disk Space Immediately + +**Option A: Clean up log files** +```bash +# Check log sizes +sudo du -sh /var/log/* + +# Clean old logs (be careful!) +sudo journalctl --vacuum-time=7d # Keep only 7 days +sudo find /var/log -name "*.log" -mtime +30 -delete # Delete logs older than 30 days +``` + +**Option B: Clean PostgreSQL logs** +```bash +# Find PostgreSQL log directory +PG_LOG_DIR=$(sudo -u postgres psql -t -c "SHOW log_directory;" | xargs) +echo "PostgreSQL logs at: $PG_LOG_DIR" + +# Clean old PostgreSQL logs (keep last 7 days) +sudo find "$PG_LOG_DIR" -name "*.log" -mtime +7 -delete +``` + +**Option C: Clean package cache** +```bash +sudo dnf clean all +sudo yum clean all 2>/dev/null +``` + +**Option D: Remove old kernels** +```bash +# List installed kernels +rpm -qa kernel + +# Remove old kernels (keep current + 1 backup) +sudo dnf remove --oldinstallonly +``` + +**Option E: Check for large temporary files** +```bash +sudo find /tmp -type f -size +100M -ls +sudo find /var/tmp -type f -size +100M -ls +``` + +### 4. Expand Disk Space (Long-term Solution) + +If you're on Rocky Linux with LVM: + +```bash +# Check available space in volume group +sudo vgs + +# If you have free space in VG, extend the logical volume +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot +sudo resize2fs /dev/mapper/rocky-lvroot # For ext4 +# OR +sudo xfs_growfs /dev/mapper/rocky-lvroot # For xfs +``` + +### 5. Prevent Future Issues + +**Set up log rotation:** +```bash +# Edit PostgreSQL log rotation +sudo vi /etc/logrotate.d/postgresql +``` + +**Monitor disk space:** +```bash +# Add to crontab +echo "0 * * * * df -h | grep -E '100%|9[0-9]%' && echo 'WARNING: Disk space low' | mail -s 'Disk Alert' admin@example.com" | sudo crontab - +``` + +## Why This Caused Your Crashes + +1. **WAL writes fail** → PostgreSQL cannot commit transactions +2. **Temp file creation fails** → Queries that need temp files crash +3. **Relation-cache writes fail** → You see those warnings +4. **Process crashes** → PostgreSQL backend processes die +5. **Connection termination** → All connections get killed + +## After Freeing Space + +1. Restart PostgreSQL to ensure it's healthy: + ```bash + sudo systemctl restart postgresql + ``` + +2. Verify PostgreSQL is running: + ```bash + sudo systemctl status postgresql + sudo -u postgres psql -c "SELECT version();" + ``` + +3. Check your DenoKV service: + ```bash + sudo systemctl status denokv + sudo journalctl -u denokv -n 50 + ``` + +## Prevention + +1. **Set up disk monitoring alerts** +2. **Configure log rotation for PostgreSQL** +3. **Regular cleanup of old logs** +4. **Consider expanding disk or adding storage** diff --git a/EXPAND_DISK_GUIDE.md b/EXPAND_DISK_GUIDE.md new file mode 100644 index 0000000..65d585d --- /dev/null +++ b/EXPAND_DISK_GUIDE.md @@ -0,0 +1,174 @@ +# Guide: Expand Disk and Free Space + +## Important: Expanding Disk Won't Free Space Automatically + +**Expanding the disk adds more space, but doesn't clean up existing files.** You need to: +1. **First**: Free up space immediately (so PostgreSQL can work) +2. **Then**: Expand the disk (for long-term capacity) + +## Step 1: Free Up Space IMMEDIATELY (Do This First!) + +### Quick Cleanup Commands + +```bash +# 1. Clean system journal logs (usually the biggest culprit) +sudo journalctl --vacuum-time=3d # Keep only 3 days of logs + +# 2. Clean package cache +sudo dnf clean all + +# 3. Remove old kernels (keep only current + 1) +sudo dnf remove --oldinstallonly --setopt installonly_limit=2 + +# 4. Check PostgreSQL logs size +sudo du -sh /var/lib/pgsql/data/log/* 2>/dev/null +# If large, clean old PostgreSQL logs: +sudo find /var/lib/pgsql/data/log -name "*.log" -mtime +7 -delete + +# 5. Check what's using space +sudo du -h --max-depth=1 / | sort -hr | head -15 +``` + +### After Cleanup, Restart PostgreSQL + +```bash +sudo systemctl restart postgresql +sudo systemctl status postgresql +``` + +## Step 2: Check Current Disk Setup + +```bash +# Check current disk usage +df -h + +# Check LVM setup (you're using LVM based on rocky-lvroot) +sudo pvs # Physical volumes +sudo vgs # Volume groups +sudo lvs # Logical volumes + +# Check if there's unallocated space in the volume group +sudo vgdisplay rocky +``` + +## Step 3: Expand the Disk + +### Option A: If You Have Unallocated Space in Volume Group + +If `vgdisplay` shows free space: + +```bash +# Extend the logical volume to use all free space +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot + +# Resize the filesystem (check which one you have first) +# For ext4: +sudo resize2fs /dev/mapper/rocky-lvroot + +# OR for xfs (more common on Rocky Linux): +sudo xfs_growfs / + +# Verify +df -h +``` + +### Option B: If You Need to Add a New Disk/Partition + +1. **Add new disk to the server** (via cloud provider console or physical disk) + +2. **Create physical volume:** + ```bash + # Find the new disk + lsblk + # Example: /dev/sdb or /dev/nvme1n1 + + # Create physical volume + sudo pvcreate /dev/sdb # Replace with your disk + ``` + +3. **Extend volume group:** + ```bash + sudo vgextend rocky /dev/sdb + ``` + +4. **Extend logical volume:** + ```bash + sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot + ``` + +5. **Resize filesystem:** + ```bash + # For xfs: + sudo xfs_growfs / + + # OR for ext4: + sudo resize2fs /dev/mapper/rocky-lvroot + ``` + +### Option C: Expand Existing Disk (Cloud Provider) + +If you're on AWS/Azure/GCP, you can expand the disk in the cloud console: + +1. **Stop the instance** (or take snapshot first) +2. **Increase disk size** in cloud provider console +3. **Start the instance** +4. **Extend the partition:** + ```bash + # Check current partition + sudo fdisk -l /dev/nvme0n1 + + # Use growpart (if available) + sudo growpart /dev/nvme0n1 3 # Adjust partition number + + # Then extend LVM + sudo pvresize /dev/nvme0n1p3 + sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot + sudo xfs_growfs / # or resize2fs + ``` + +## Step 4: Verify and Monitor + +```bash +# Check disk space +df -h + +# Check PostgreSQL is working +sudo -u postgres psql -c "SELECT version();" + +# Check DenoKV service +sudo systemctl status denokv +``` + +## Prevention: Set Up Automatic Cleanup + +```bash +# Create log rotation for PostgreSQL +sudo vi /etc/logrotate.d/postgresql-custom +``` + +Add: +``` +/var/lib/pgsql/data/log/*.log { + daily + rotate 7 + compress + delaycompress + missingok + notifempty + create 0640 postgres postgres + sharedscripts +} +``` + +## Quick Reference Commands + +```bash +# Check everything +df -h && echo "---" && sudo vgs && echo "---" && sudo lvs + +# Free space immediately +sudo journalctl --vacuum-time=3d && sudo dnf clean all + +# Expand if you have free space in VG +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot && sudo xfs_growfs / +``` diff --git a/FIX_DISK_SIZE.md b/FIX_DISK_SIZE.md new file mode 100644 index 0000000..1b2b0f1 --- /dev/null +++ b/FIX_DISK_SIZE.md @@ -0,0 +1,170 @@ +# Fix: 512GB Disk Not Recognized - Azure VM + +## Problem +- Azure shows 512GB disk +- System shows only 8.8GB (100% full) +- The partition/LVM hasn't been extended to use the full disk + +## Step 1: Check Current Disk Status + +Run these commands to see what's happening: + +```bash +# Check actual disk size (should show 512GB) +lsblk + +# Check partition table +sudo fdisk -l /dev/sda # or /dev/nvme0n1 depending on your setup + +# Check LVM status +sudo pvs +sudo vgs +sudo lvs + +# Check current filesystem +df -h +``` + +## Step 2: Identify the Disk Device + +For Azure VMs, it's usually: +- **Standard VMs**: `/dev/sda` or `/dev/sdb` +- **NVMe VMs**: `/dev/nvme0n1` or `/dev/nvme1n1` + +Check with: +```bash +lsblk -o NAME,SIZE,TYPE,MOUNTPOINT +``` + +## Step 3: Extend the Partition (If Needed) + +### For Standard Disk (/dev/sda): + +```bash +# Check current partition +sudo fdisk -l /dev/sda + +# Use growpart to extend partition (usually partition 3 for LVM) +sudo growpart /dev/sda 3 + +# If growpart not available, install it: +sudo dnf install cloud-utils-growpart +``` + +### For NVMe Disk (/dev/nvme0n1): + +```bash +# Check current partition +sudo fdisk -l /dev/nvme0n1 + +# Extend partition (usually partition 3) +sudo growpart /dev/nvme0n1 3 +``` + +## Step 4: Resize Physical Volume + +After extending the partition, resize the LVM physical volume: + +```bash +# For standard disk +sudo pvresize /dev/sda3 + +# OR for NVMe +sudo pvresize /dev/nvme0n1p3 +``` + +## Step 5: Extend Logical Volume + +```bash +# Extend to use all available space +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot +``` + +## Step 6: Resize Filesystem + +```bash +# For xfs (most common on Rocky Linux) +sudo xfs_growfs / + +# OR for ext4 +sudo resize2fs /dev/mapper/rocky-lvroot +``` + +## Step 7: Verify + +```bash +# Should now show ~512GB available +df -h + +# Verify LVM +sudo vgs +sudo lvs +``` + +## Complete Command Sequence + +```bash +# 1. Check disk +lsblk +sudo fdisk -l /dev/sda # or /dev/nvme0n1 + +# 2. Install growpart if needed +sudo dnf install -y cloud-utils-growpart + +# 3. Extend partition (adjust device and partition number) +sudo growpart /dev/sda 3 # or /dev/nvme0n1 3 + +# 4. Resize physical volume +sudo pvresize /dev/sda3 # or /dev/nvme0n1p3 + +# 5. Extend logical volume +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot + +# 6. Resize filesystem +sudo xfs_growfs / # or resize2fs for ext4 + +# 7. Verify +df -h +``` + +## Troubleshooting + +### If growpart fails: +```bash +# Check partition number +sudo fdisk -l /dev/sda | grep -E "^/dev" + +# Manually extend using fdisk (advanced - be careful!) +# This requires deleting and recreating the partition +# Only do this if you know what you're doing +``` + +### If pvresize fails: +```bash +# Check if physical volume exists +sudo pvs + +# Check partition type +sudo blkid /dev/sda3 +``` + +### If you see "device is busy": +```bash +# Unmount if possible (usually can't for root) +# Or reboot after extending partition, then continue with pvresize +``` + +## After Fixing: Free Up Space + +Even after expanding, you should still clean up: + +```bash +# Clean logs +sudo journalctl --vacuum-time=7d + +# Clean package cache +sudo dnf clean all + +# Restart PostgreSQL +sudo systemctl restart postgresql +``` diff --git a/POSTGRES_500_DIAGNOSIS.md b/POSTGRES_500_DIAGNOSIS.md new file mode 100644 index 0000000..f46b112 --- /dev/null +++ b/POSTGRES_500_DIAGNOSIS.md @@ -0,0 +1,204 @@ +# Postgres 500 Error Diagnosis Guide + +## What I Found + +After reviewing the Postgres implementation, here's what could be causing your 500 errors: + +### Error Flow +1. Postgres errors occur in `postgres/backend.rs:atomic_write()` +2. Errors are converted to `JsErrorBox` in `postgres/lib.rs:109-113` +3. `JsErrorBox` → `ApiError::InternalServerError` in `denokv/main.rs:754-758` +4. Returns HTTP 500 with generic message + +**⚠️ Previously**: Errors were NOT logged, making debugging impossible +**✅ Now Fixed**: Errors are now logged with `log::error!()` before returning 500 + +## Immediate Actions to Take + +### 1. Check Server Logs (MOST IMPORTANT) + +The server now logs detailed error messages. Check your server logs: + +```bash +# If running directly +RUST_LOG=error ./denokv --postgres-url serve --access-token + +# If running in Docker +docker logs 2>&1 | grep -i error + +# For more detail +RUST_LOG=debug ./denokv --postgres-url serve --access-token +``` + +Look for messages like: +- `Database error: ` +- `atomic_write failed: ` + +### 2. Check Postgres Server Logs + +```bash +# Find Postgres log location +sudo -u postgres psql -c "SHOW log_directory;" +sudo -u postgres psql -c "SHOW log_filename;" + +# View recent errors +sudo tail -f /var/log/postgresql/postgresql-*.log | grep -i error +``` + +### 3. Check Connection Pool Status + +```sql +-- Connect to your database +psql -h -U -d + +-- Check active connections +SELECT count(*) as active_connections, + count(*) FILTER (WHERE state = 'active') as active_queries, + count(*) FILTER (WHERE state = 'idle in transaction') as idle_in_transaction +FROM pg_stat_activity +WHERE datname = current_database(); + +-- Check max connections +SHOW max_connections; +``` + +**If you see many `idle in transaction` connections**, you have a connection leak. + +### 4. Check for Deadlocks + +```sql +-- Check for locks +SELECT + locktype, + relation::regclass, + mode, + granted, + pid, + pg_stat_activity.query +FROM pg_locks +JOIN pg_stat_activity ON pg_locks.pid = pg_stat_activity.pid +WHERE NOT granted +ORDER BY pid; +``` + +### 5. Check Transaction Timeouts + +```sql +-- Check timeout settings +SHOW statement_timeout; +SHOW idle_in_transaction_session_timeout; +SHOW lock_timeout; +``` + +### 6. Verify Schema is Correct + +```sql +-- Check if tables exist +SELECT tablename FROM pg_tables +WHERE schemaname = 'public' +AND tablename IN ('kv_store', 'queue_messages', 'queue_running'); + +-- Check table structure +\d kv_store +\d queue_messages +\d queue_running + +-- Check indexes +\di +``` + +## Most Likely Causes (Based on Code Review) + +### 1. **Connection Pool Exhaustion** (Most Likely) +- **Symptom**: Errors increase with load +- **Check**: Active connections vs. pool size +- **Fix**: Increase `max_connections` in PostgresConfig or check for leaks + +### 2. **Transaction Deadlocks** +- **Symptom**: Intermittent errors, especially with concurrent writes +- **Check**: `pg_locks` table for blocking queries +- **Fix**: Add retry logic or optimize transaction scope + +### 3. **Transaction Timeout** +- **Symptom**: Errors after specific duration +- **Check**: `statement_timeout` and `idle_in_transaction_session_timeout` +- **Fix**: Increase timeouts or optimize slow queries + +### 4. **Schema Issues** +- **Symptom**: Consistent errors on specific operations +- **Check**: Table existence and structure +- **Fix**: Ensure `initialize_schema()` was called successfully + +### 5. **Connection Failures** +- **Symptom**: Intermittent "connection refused" or "connection reset" +- **Check**: Postgres server status, network connectivity +- **Fix**: Verify Postgres is running, check firewall, verify credentials + +## Code Locations to Review + +1. **Atomic Write Implementation**: `postgres/backend.rs:170-271` + - Transaction creation: line 175 + - Checks: lines 178-193 + - Mutations: lines 200-251 + - Enqueues: lines 255-266 + - Commit: line 268 + +2. **Error Handling**: `postgres/lib.rs:105-116` + - Connection acquisition: line 109 + - Error conversion: lines 110, 113 + +3. **Error Types**: `postgres/error.rs` + - All error variants and their conversions + +## Quick Diagnostic Script + +Run this to get a comprehensive view: + +```bash +#!/bin/bash +echo "=== Connection Pool Status ===" +psql -h -U -d -c " +SELECT + count(*) as total_connections, + count(*) FILTER (WHERE state = 'active') as active, + count(*) FILTER (WHERE state = 'idle') as idle, + count(*) FILTER (WHERE state = 'idle in transaction') as idle_in_tx +FROM pg_stat_activity +WHERE datname = current_database(); +" + +echo -e "\n=== Locks ===" +psql -h -U -d -c " +SELECT count(*) as blocked_locks +FROM pg_locks +WHERE NOT granted; +" + +echo -e "\n=== Table Status ===" +psql -h -U -d -c " +SELECT tablename, + pg_size_pretty(pg_total_relation_size(quote_ident(tablename)::regclass)) as size +FROM pg_tables +WHERE schemaname = 'public' +AND tablename IN ('kv_store', 'queue_messages', 'queue_running'); +" + +echo -e "\n=== Recent Errors (from Postgres logs) ===" +sudo tail -100 /var/log/postgresql/postgresql-*.log | grep -i error | tail -10 +``` + +## Next Steps + +1. **Enable error logging** on your server (already done in code) +2. **Check server logs** for the actual error messages +3. **Run diagnostic queries** above to identify the issue +4. **Check Postgres server logs** for database-level errors +5. **Monitor connection pool** usage over time + +## What Was Changed + +I've improved error logging in the codebase: +- Added `log::error!()` in `atomic_write_endpoint` to log errors before conversion +- Added `log::error!()` in `From` implementation to log all database errors + +This means you'll now see detailed error messages in your server logs, which will tell you exactly what's failing. diff --git a/POSTGRES_TROUBLESHOOTING.md b/POSTGRES_TROUBLESHOOTING.md new file mode 100644 index 0000000..004f5fc --- /dev/null +++ b/POSTGRES_TROUBLESHOOTING.md @@ -0,0 +1,264 @@ +# Troubleshooting 500 Errors with Postgres Backend + +## Error Flow + +When using the Postgres backend, errors flow like this: + +1. **PostgresError** (in `postgres/error.rs`) → converted to **JsErrorBox** (in `postgres/lib.rs:109-113`) +2. **JsErrorBox** → converted to **ApiError::InternalServerError** (in `denokv/main.rs:754-758`) +3. **ApiError::InternalServerError** → HTTP 500 response + +**⚠️ IMPORTANT**: Unlike SQLite errors, Postgres errors are NOT currently logged before being converted to InternalServerError. This makes debugging harder. + +## Common Postgres-Specific Causes + +### 1. Connection Pool Exhaustion + +**Symptoms:** +- Errors under high load +- Errors become more frequent as load increases +- May see "connection pool timeout" or "no connections available" + +**Diagnosis:** +```sql +-- Check active connections +SELECT count(*) FROM pg_stat_activity WHERE datname = 'your_database'; + +-- Check connection pool settings +SHOW max_connections; +``` + +**Solutions:** +- Increase `max_connections` in PostgresConfig +- Check for connection leaks (connections not being returned to pool) +- Increase Postgres server's `max_connections` setting +- Use connection pooling at the Postgres level (PgBouncer) + +### 2. Transaction Deadlocks + +**Symptoms:** +- Intermittent 500 errors +- Errors occur with concurrent writes to same keys +- May see "deadlock detected" in Postgres logs + +**Diagnosis:** +```sql +-- Check for locks +SELECT * FROM pg_locks WHERE NOT granted; + +-- Check for blocking queries +SELECT + blocked_locks.pid AS blocked_pid, + blocking_locks.pid AS blocking_pid, + blocked_activity.usename AS blocked_user, + blocking_activity.usename AS blocking_user, + blocked_activity.query AS blocked_statement, + blocking_activity.query AS blocking_statement +FROM pg_catalog.pg_locks blocked_locks +JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid +JOIN pg_catalog.pg_locks blocking_locks + ON blocking_locks.locktype = blocked_locks.locktype + AND blocking_locks.database IS NOT DISTINCT FROM blocked_locks.database + AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation + AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page + AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple + AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid + AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid + AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid + AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid + AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid + AND blocking_locks.pid != blocked_locks.pid +JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid +WHERE NOT blocked_locks.granted; +``` + +**Solutions:** +- Ensure transactions are kept short +- Use appropriate isolation levels +- Add retry logic for deadlock errors +- Consider using advisory locks for critical sections + +### 3. Transaction Timeout + +**Symptoms:** +- Errors after a specific duration +- Long-running transactions fail + +**Diagnosis:** +- Check `statement_timeout` setting in PostgresConfig +- Check Postgres server's `statement_timeout` and `idle_in_transaction_session_timeout` + +**Solutions:** +- Increase timeout values if needed +- Optimize slow queries +- Break large transactions into smaller ones + +### 4. Schema/Table Issues + +**Symptoms:** +- Consistent errors on specific operations +- "relation does not exist" or "column does not exist" errors + +**Diagnosis:** +```sql +-- Check if tables exist +SELECT tablename FROM pg_tables WHERE schemaname = 'public'; + +-- Check table structure +\d kv_store +\d queue_messages +\d queue_running + +-- Check indexes +\di +``` + +**Solutions:** +- Ensure schema is initialized: `backend.initialize_schema().await?` +- Check for missing migrations +- Verify table structure matches expected schema + +### 5. Data Type Mismatches + +**Symptoms:** +- Errors on specific mutations (Sum/Min/Max) +- "invalid input syntax" errors + +**Diagnosis:** +- Check value encoding in database: +```sql +SELECT key, value_encoding, length(value) FROM kv_store WHERE key = $1; +``` + +**Solutions:** +- Ensure mutations match value types (Sum/Min/Max only work with U64) +- Check for data corruption + +### 6. Serialization Errors + +**Symptoms:** +- Errors when enqueueing messages +- "invalid json" errors + +**Diagnosis:** +- Check `keys_if_undelivered` and `backoff_schedule` serialization +- Look for invalid JSON in queue_messages table + +**Solutions:** +- Verify enqueue payload structure +- Check JSON serialization of complex types + +### 7. Connection Failures + +**Symptoms:** +- Intermittent connection errors +- "connection refused" or "connection reset" + +**Diagnosis:** +```bash +# Check Postgres is running +pg_isready -h localhost -p 5432 + +# Check network connectivity +telnet 5432 + +# Check Postgres logs +tail -f /var/log/postgresql/postgresql-*.log +``` + +**Solutions:** +- Verify Postgres server is running +- Check network connectivity +- Verify connection string is correct +- Check firewall rules +- Verify authentication credentials + +### 8. Query Performance Issues + +**Symptoms:** +- Slow responses leading to timeouts +- Errors under load + +**Diagnosis:** +```sql +-- Check for slow queries +SELECT pid, now() - pg_stat_activity.query_start AS duration, query +FROM pg_stat_activity +WHERE (now() - pg_stat_activity.query_start) > interval '5 seconds'; + +-- Check index usage +SELECT schemaname, tablename, indexname, idx_scan, idx_tup_read, idx_tup_fetch +FROM pg_stat_user_indexes +WHERE schemaname = 'public'; +``` + +**Solutions:** +- Add missing indexes +- Analyze and optimize slow queries +- Consider partitioning for large tables +- Update table statistics: `ANALYZE kv_store;` + +## How to Get Detailed Error Information + +### Enable Debug Logging + +```bash +RUST_LOG=debug ./denokv --postgres-url serve --access-token +``` + +### Check Postgres Server Logs + +```bash +# On most Linux systems +tail -f /var/log/postgresql/postgresql-*.log + +# Or check the configured log location +SHOW log_directory; +SHOW log_filename; +``` + +### Enable Postgres Query Logging + +Add to `postgresql.conf`: +``` +log_statement = 'all' +log_duration = on +log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h ' +``` + +### Monitor Active Queries + +```sql +-- See what queries are running +SELECT pid, usename, application_name, client_addr, state, query, query_start +FROM pg_stat_activity +WHERE datname = current_database() +AND state != 'idle'; +``` + +## Code Locations to Check + +1. **Error Conversion**: `postgres/lib.rs:109-113` - where PostgresError → JsErrorBox +2. **Atomic Write**: `postgres/backend.rs:170-271` - main atomic_write implementation +3. **Transaction Handling**: `postgres/backend.rs:175` - transaction creation +4. **Error Types**: `postgres/error.rs` - all PostgresError variants + +## Recommended Improvements + +1. **Add Error Logging**: Log Postgres errors before converting to InternalServerError +2. **Add Metrics**: Track connection pool usage, transaction durations, error rates +3. **Add Retry Logic**: Retry on transient errors (deadlocks, connection failures) +4. **Better Error Messages**: Include more context in error responses (for debugging) + +## Quick Diagnostic Checklist + +- [ ] Check Postgres server is running and accessible +- [ ] Verify connection string is correct +- [ ] Check connection pool size vs. actual connections +- [ ] Review Postgres server logs for errors +- [ ] Check for deadlocks in pg_locks +- [ ] Verify schema is initialized correctly +- [ ] Check table/index existence +- [ ] Monitor query performance +- [ ] Check for long-running transactions +- [ ] Verify data types match expectations diff --git a/ROCKY_LINUX_SETUP.md b/ROCKY_LINUX_SETUP.md new file mode 100644 index 0000000..703edd3 --- /dev/null +++ b/ROCKY_LINUX_SETUP.md @@ -0,0 +1,157 @@ +# Rocky Linux Setup for DenoKV PostgreSQL Testing + +This document describes how to set up a Rocky Linux environment for testing DenoKV PostgreSQL integration. + +## Prerequisites + +- Rocky Linux 8 or 9 +- Internet connection +- Non-root user with sudo privileges + +## Quick Setup + +Run the setup script: + +```bash +chmod +x setup-rocky-linux.sh +./setup-rocky-linux.sh +``` + +## What the Setup Script Does + +1. **Updates system packages** - Ensures all packages are up to date +2. **Installs development tools** - Installs essential development packages +3. **Installs PostgreSQL development libraries** - Required for PostgreSQL backend compilation +4. **Installs Docker and Docker Compose** - For running PostgreSQL test container +5. **Installs Rust** - Required for building the project +6. **Installs additional dependencies** - OpenSSL and pkg-config for Rust compilation +7. **Clones the repository** - Downloads the DenoKV source code +8. **Builds the project** - Compiles all components +9. **Creates test script** - Generates a script to run PostgreSQL integration tests + +## Running Tests + +After setup, you can run the PostgreSQL integration tests: + +```bash +./test-postgres-integration.sh +``` + +## Running the Production Server + +To start the DenoKV server for remote access, you need to set up authentication: + +### 1. Set Required Environment Variables + +**Option A: Automatic Token Generation (Recommended)** +The server script will automatically generate a secure token if none is provided: + +```bash +# Required: PostgreSQL connection URL +export DENO_KV_POSTGRES_URL="postgresql://user:password@host:port/database" + +# Optional: Additional configuration +export DENO_KV_DATABASE_TYPE="postgres" # Default: postgres +export DENO_KV_NUM_WORKERS="4" # Default: 4 +``` + +**Option B: Manual Token Generation** +Generate a token manually using the utility script: + +```bash +./generate-access-token.sh +``` + +**Option C: Set Your Own Token** +```bash +# Required: Access token for authentication (minimum 12 characters) +export DENO_KV_ACCESS_TOKEN="your-secure-access-token-here" + +# Required: PostgreSQL connection URL +export DENO_KV_POSTGRES_URL="postgresql://user:password@host:port/database" + +# Optional: Additional configuration +export DENO_KV_DATABASE_TYPE="postgres" # Default: postgres +export DENO_KV_NUM_WORKERS="4" # Default: 4 +``` + +### 2. Start the Server + +```bash +./start-denokv-server.sh +``` + +The server will start on `0.0.0.0:4512` and be accessible remotely. + +**Important:** Make sure port 4512 is open in your firewall: +```bash +# Check if port is open +sudo firewall-cmd --list-ports + +# If not open, add it: +sudo firewall-cmd --permanent --add-port=4512/tcp +sudo firewall-cmd --reload +``` + +### 3. Client Authentication + +When connecting from a Deno application, use the access token in the Authorization header: + +```typescript +const kv = await Deno.openKv("http://your-server:4512", { + accessToken: "your-secure-access-token-here" +}); +``` + +**Important Security Notes:** +- The access token must be at least 12 characters long +- Use a strong, randomly generated token for production +- Keep the access token secure and don't commit it to version control +- The server validates tokens using constant-time comparison for security + +## Manual Steps After Setup + +1. **Log out and log back in** - This ensures Docker group membership takes effect +2. **Verify Docker access** - Run `docker ps` to confirm Docker is accessible +3. **Run tests** - Execute the test script to verify everything works + +## Troubleshooting + +### Docker Permission Issues +If you get permission denied errors with Docker: +```bash +sudo usermod -aG docker $USER +# Then log out and log back in +``` + +### Rust Not Found +If Rust commands are not found: +```bash +source ~/.cargo/env +``` + +### PostgreSQL Connection Issues +Make sure the PostgreSQL container is running: +```bash +docker-compose -f docker-compose.test.yml ps +``` + +## Project Structure + +- `denokv/` - Main DenoKV project +- `postgres/` - PostgreSQL backend implementation +- `docker-compose.test.yml` - PostgreSQL test container configuration +- `test-postgres.sh` - Original test script +- `test-postgres-integration.sh` - Enhanced test script for Rocky Linux + +## Environment Variables + +The test script sets the following environment variable: +- `POSTGRES_URL=postgresql://postgres:password@localhost:5432/denokv_test` + +## Cleanup + +To stop and remove the PostgreSQL test container: +```bash +docker-compose -f docker-compose.test.yml down +``` \ No newline at end of file diff --git a/TROUBLESHOOTING_500_ERRORS.md b/TROUBLESHOOTING_500_ERRORS.md new file mode 100644 index 0000000..2dadc9e --- /dev/null +++ b/TROUBLESHOOTING_500_ERRORS.md @@ -0,0 +1,222 @@ +# Troubleshooting 500 Internal Server Errors on `/v2/atomic_write` + +## Overview + +When the `/v2/atomic_write` endpoint returns a 500 Internal Server Error, it means the server encountered an unexpected error while processing the atomic write operation. The error message "An internal server error occurred." is generic, but the actual error details are logged by the server. + +## How Errors Are Handled + +Looking at the code in `denokv/main.rs`, the following error types are converted to `InternalServerError` (500 status): + +1. **SQLite Errors** (`SqliteBackendError::SqliteError`): + - Database corruption + - Disk I/O errors (disk full, permissions) + - Locking issues (database locked, timeout) + - Transaction failures + - SQL syntax errors (shouldn't happen in normal operation) + +2. **Generic Backend Errors** (`SqliteBackendError::GenericError`): + - Any other unexpected backend error + +3. **Postgres Errors** (if using Postgres backend): + - Connection pool exhaustion + - Transaction failures + - Database connection errors + - Query execution errors + +4. **JavaScript Error Box Errors** (`JsErrorBox`): + - Any error from the JS error handling layer + +## How to Diagnose + +### 1. Check Server Logs + +The server logs detailed error information using `log::error!()`. To see these logs: + +**If running directly:** +```bash +RUST_LOG=error ./denokv --sqlite-path /data/denokv.sqlite serve --access-token +``` + +**If running in Docker:** +```bash +docker logs 2>&1 | grep -i error +``` + +**For more detailed logging:** +```bash +RUST_LOG=debug ./denokv --sqlite-path /data/denokv.sqlite serve --access-token +``` + +The logs will show messages like: +- `Sqlite error: ` +- `Generic error: ` + +### 2. Common Causes and Solutions + +#### A. Database Locking Issues (SQLite) + +**Symptoms:** +- Intermittent 500 errors +- Errors occur under high concurrency +- Logs show "database is locked" or timeout errors + +**Solutions:** +- Check if multiple processes are accessing the database +- Ensure the database file has proper permissions +- Consider using WAL mode (should be enabled by default) +- Increase SQLite timeout if needed +- Check for long-running transactions + +#### B. Disk Space Issues + +**Symptoms:** +- Consistent 500 errors +- Logs show "disk I/O error" or "no space left on device" + +**Solutions:** +```bash +# Check disk space +df -h /data + +# Check database file size +ls -lh /data/denokv.sqlite +``` + +- Free up disk space +- Consider database cleanup/compaction +- Move database to a location with more space + +#### C. Database Corruption + +**Symptoms:** +- Consistent 500 errors +- Logs show "database disk image is malformed" or similar + +**Solutions:** +```bash +# Check database integrity +sqlite3 /data/denokv.sqlite "PRAGMA integrity_check;" + +# If corrupted, restore from backup +``` + +- Restore from a known good backup +- If no backup, attempt recovery using SQLite tools + +#### D. Connection Pool Exhaustion (Postgres) + +**Symptoms:** +- Errors under high load +- Logs show connection timeout or pool exhaustion + +**Solutions:** +- Increase connection pool size in Postgres configuration +- Check for connection leaks +- Monitor active connections: `SELECT count(*) FROM pg_stat_activity;` + +#### E. Transaction Failures + +**Symptoms:** +- Intermittent errors +- May occur with specific operations + +**Solutions:** +- Check for constraint violations +- Verify data types match expected formats +- Check for deadlocks (Postgres) + +#### F. Resource Exhaustion + +**Symptoms:** +- Errors under high load +- System resource limits reached + +**Solutions:** +```bash +# Check system resources +free -h +ulimit -a +``` + +- Increase system limits +- Add more memory/CPU +- Optimize queries/operations + +### 3. Enable Debug Logging + +To get more detailed information about what's happening: + +```bash +RUST_LOG=debug ./denokv --sqlite-path /data/denokv.sqlite serve --access-token +``` + +This will show: +- Detailed error stack traces +- Transaction details +- Database operation logs + +### 4. Check Database Health + +**For SQLite:** +```bash +sqlite3 /data/denokv.sqlite < # Checkout specific version + +# NPM package +cd npm/napi +npm run build # Build native bindings +npm run test # Run tests +npm run format # Format code + +# Docker +docker build -t denokv . +docker run -p 4512:4512 -v ./data:/data denokv --sqlite-path /data/db serve --access-token token +``` + +## Learning Resources + +- **Deno KV Documentation**: https://deno.com/kv +- **KV Connect Protocol**: `proto/kv-connect.md` +- **Rust Async Book**: https://rust-lang.github.io/async-book/ +- **Axum Framework**: https://docs.rs/axum/ +- **SQLite Documentation**: https://www.sqlite.org/docs.html +- **NAPI-RS**: https://napi.rs/ + +## Notes for Future Self + +- **Always test with real server** - Unit tests aren't enough +- **Understand the protocol** - KV Connect is the foundation +- **SQLite is single-writer** - Design around this constraint +- **Versionstamps are critical** - They provide ordering guarantees +- **Authentication is simple** - Just Bearer tokens +- **Error handling matters** - Users depend on clear error messages +- **Performance is important** - This is a database, not just a toy + +Remember: This is a production database system. Changes affect real users and their data. Test thoroughly, understand the implications, and always consider backward compatibility. \ No newline at end of file diff --git a/check_postgres_space.sh b/check_postgres_space.sh new file mode 100644 index 0000000..c008b65 --- /dev/null +++ b/check_postgres_space.sh @@ -0,0 +1,57 @@ +#!/bin/bash +# Script to find which PostgreSQL files are using disk space + +echo "=== PostgreSQL Disk Space Analysis ===" +echo "" + +# Find PostgreSQL data directory +PG_DATA_DIR="/var/lib/pgsql/data" +if [ ! -d "$PG_DATA_DIR" ]; then + PG_DATA_DIR="/var/lib/postgresql/*/data" +fi + +echo "1. PostgreSQL data directory size:" +sudo du -sh $PG_DATA_DIR 2>/dev/null || echo "Could not find PostgreSQL data directory" +echo "" + +echo "2. Top 20 largest files/directories in PostgreSQL data:" +sudo du -h --max-depth=2 $PG_DATA_DIR 2>/dev/null | sort -hr | head -20 +echo "" + +echo "3. PostgreSQL log files size:" +sudo du -sh $PG_DATA_DIR/log/* 2>/dev/null || sudo du -sh $PG_DATA_DIR/pg_log/* 2>/dev/null || echo "No log files found" +echo "" + +echo "4. Individual log files (sorted by size):" +sudo find $PG_DATA_DIR -name "*.log" -type f -exec du -h {} \; 2>/dev/null | sort -hr | head -10 +echo "" + +echo "5. WAL (Write-Ahead Log) files size:" +sudo du -sh $PG_DATA_DIR/pg_wal 2>/dev/null || echo "WAL directory not found" +echo "" + +echo "6. Check for old WAL files:" +sudo ls -lh $PG_DATA_DIR/pg_wal/* 2>/dev/null | tail -20 || echo "No WAL files found" +echo "" + +echo "7. Check PostgreSQL log configuration:" +sudo -u postgres psql -c "SHOW log_directory;" 2>/dev/null || echo "Could not connect to PostgreSQL" +sudo -u postgres psql -c "SHOW log_filename;" 2>/dev/null || echo "" +echo "" + +echo "8. System journal logs (often the biggest culprit):" +sudo journalctl --disk-usage +echo "" + +echo "9. Top 10 largest directories on root filesystem:" +sudo du -h --max-depth=1 / 2>/dev/null | sort -hr | head -10 +echo "" + +echo "=== Summary ===" +echo "To clean PostgreSQL logs (keep last 7 days):" +echo " sudo find $PG_DATA_DIR/log -name '*.log' -mtime +7 -delete" +echo " sudo find $PG_DATA_DIR/pg_log -name '*.log' -mtime +7 -delete" +echo "" +echo "To clean system journal (keep last 3 days):" +echo " sudo journalctl --vacuum-time=3d" +echo "" diff --git a/denokv/Cargo.toml b/denokv/Cargo.toml index e4ae1c6..55361f9 100644 --- a/denokv/Cargo.toml +++ b/denokv/Cargo.toml @@ -28,6 +28,7 @@ clap.workspace = true constant_time_eq.workspace = true denokv_proto.workspace = true denokv_sqlite.workspace = true +denokv_postgres.workspace = true denokv_timemachine.workspace = true env_logger.workspace = true futures.workspace = true diff --git a/denokv/config.rs b/denokv/config.rs index 09a6562..3310aca 100644 --- a/denokv/config.rs +++ b/denokv/config.rs @@ -6,7 +6,15 @@ use clap::Parser; pub struct Config { /// The path to the SQLite database KV will persist to. #[clap(long, env = "DENO_KV_SQLITE_PATH")] - pub sqlite_path: String, + pub sqlite_path: Option, + + /// PostgreSQL connection URL for the database. + #[clap(long, env = "DENO_KV_POSTGRES_URL")] + pub postgres_url: Option, + + /// Database type to use (sqlite or postgres). + #[clap(long, env = "DENO_KV_DATABASE_TYPE", default_value = "sqlite")] + pub database_type: String, #[command(subcommand)] pub subcommand: SubCmd, diff --git a/denokv/main.rs b/denokv/main.rs index 1fb95e2..83bdd71 100644 --- a/denokv/main.rs +++ b/denokv/main.rs @@ -39,6 +39,7 @@ use denokv_proto::time::utc_now; use denokv_proto::AtomicWrite; use denokv_proto::Consistency; use denokv_proto::ConvertError; +use denokv_proto::Database; use denokv_proto::DatabaseMetadata; use denokv_proto::EndpointInfo; use denokv_proto::MetadataExchangeRequest; @@ -49,6 +50,8 @@ use denokv_sqlite::Sqlite; use denokv_sqlite::SqliteBackendError; use denokv_sqlite::SqliteConfig; use denokv_sqlite::SqliteNotifier; +use denokv_postgres::Postgres; +use denokv_postgres::PostgresConfig; use denokv_timemachine::backup_source_s3::DatabaseBackupSourceS3; use denokv_timemachine::backup_source_s3::DatabaseBackupSourceS3Config; use denokv_timemachine::time_travel::TimeTravelControl; @@ -80,9 +83,55 @@ mod config; const SYNC_INTERVAL_BASE_MS: u64 = 10000; const SYNC_INTERVAL_JITTER_MS: u64 = 5000; +#[derive(Clone)] +enum DatabaseBackend { + Sqlite(Sqlite), + Postgres(Postgres), +} + +impl DatabaseBackend { + async fn snapshot_read( + &self, + requests: Vec, + options: SnapshotReadOptions, + ) -> Result, deno_error::JsErrorBox> { + match self { + DatabaseBackend::Sqlite(sqlite) => sqlite.snapshot_read(requests, options).await.map_err(deno_error::JsErrorBox::from_err), + DatabaseBackend::Postgres(postgres) => postgres.snapshot_read(requests, options).await, + } + } + + async fn atomic_write( + &self, + write: AtomicWrite, + ) -> Result, deno_error::JsErrorBox> { + match self { + DatabaseBackend::Sqlite(sqlite) => sqlite.atomic_write(write).await.map_err(deno_error::JsErrorBox::from_err), + DatabaseBackend::Postgres(postgres) => postgres.atomic_write(write).await, + } + } + + fn watch( + &self, + keys: Vec>, + ) -> std::pin::Pin, deno_error::JsErrorBox>> + Send>> { + match self { + DatabaseBackend::Sqlite(sqlite) => sqlite.watch(keys), + DatabaseBackend::Postgres(postgres) => postgres.watch(keys), + } + } + + fn close(&self) { + match self { + DatabaseBackend::Sqlite(sqlite) => sqlite.close(), + DatabaseBackend::Postgres(postgres) => postgres.close(), + } + } +} + #[derive(Clone)] struct AppState { - sqlite: Sqlite, + database: DatabaseBackend, access_token: &'static str, } @@ -135,7 +184,9 @@ async fn run_pitr( run_sync(config, &options.replica, false, None).await?; } PitrSubCmd::List(options) => { - let db = rusqlite::Connection::open(&config.sqlite_path)?; + let sqlite_path = config.sqlite_path.as_ref() + .ok_or_else(|| anyhow::anyhow!("SQLite path is required for PITR operations"))?; + let db = rusqlite::Connection::open(sqlite_path)?; let mut ttc = TimeTravelControl::open(db)?; let start = if let Some(start) = &options.start { @@ -174,7 +225,9 @@ async fn run_pitr( } } PitrSubCmd::Info => { - let db = rusqlite::Connection::open(&config.sqlite_path)?; + let sqlite_path = config.sqlite_path.as_ref() + .ok_or_else(|| anyhow::anyhow!("SQLite path is required for PITR operations"))?; + let db = rusqlite::Connection::open(sqlite_path)?; let mut ttc = TimeTravelControl::open(db)?; let current_versionstamp = ttc.get_current_versionstamp()?; @@ -184,7 +237,9 @@ async fn run_pitr( ); } PitrSubCmd::Checkout(options) => { - let db = rusqlite::Connection::open(&config.sqlite_path)?; + let sqlite_path = config.sqlite_path.as_ref() + .ok_or_else(|| anyhow::anyhow!("SQLite path is required for PITR operations"))?; + let db = rusqlite::Connection::open(sqlite_path)?; let mut ttc = TimeTravelControl::open(db)?; let versionstamp = hex::decode(&options.versionstamp) .ok() @@ -211,26 +266,43 @@ async fn run_serve( anyhow::bail!("Access token must be at minimum 12 chars long."); } - let path = Path::new(&config.sqlite_path); - let read_only = options.read_only || options.sync_from_s3; - let sqlite_config = SqliteConfig { - batch_timeout: options - .atomic_write_batch_timeout_ms - .map(std::time::Duration::from_millis), - num_workers: options.num_workers, + let database = match config.database_type.as_str() { + "sqlite" => { + let sqlite_path = config.sqlite_path.as_ref() + .ok_or_else(|| anyhow::anyhow!("SQLite path is required when using sqlite database type"))?; + let path = Path::new(sqlite_path); + let read_only = options.read_only || options.sync_from_s3; + let sqlite_config = SqliteConfig { + batch_timeout: options + .atomic_write_batch_timeout_ms + .map(std::time::Duration::from_millis), + num_workers: options.num_workers, + }; + let sqlite = open_sqlite(path, read_only, sqlite_config.clone())?; + info!( + "Opened{} SQLite database at {}. Batch timeout: {:?}", + if read_only { " read only" } else { "" }, + path.to_string_lossy(), + sqlite_config.batch_timeout, + ); + DatabaseBackend::Sqlite(sqlite) + } + "postgres" => { + let postgres_url = config.postgres_url.as_ref() + .ok_or_else(|| anyhow::anyhow!("PostgreSQL URL is required when using postgres database type"))?; + let postgres_config = PostgresConfig::new(postgres_url.clone()) + .with_max_connections(options.num_workers.max(10)); + let postgres = Postgres::new(postgres_config).await?; + info!("Opened PostgreSQL database at {}", postgres_url); + DatabaseBackend::Postgres(postgres) + } + _ => anyhow::bail!("Invalid database type: {}. Must be 'sqlite' or 'postgres'", config.database_type), }; - let sqlite = open_sqlite(path, read_only, sqlite_config.clone())?; - info!( - "Opened{} database at {}. Batch timeout: {:?}", - if read_only { " read only" } else { "" }, - path.to_string_lossy(), - sqlite_config.batch_timeout, - ); let access_token = options.access_token.as_str(); let state = AppState { - sqlite, + database, access_token, }; @@ -293,7 +365,9 @@ async fn run_sync( let s3_config = s3_config.load().await; let s3_client = aws_sdk_s3::Client::new(&s3_config); - let db = rusqlite::Connection::open(&config.sqlite_path)?; + let sqlite_path = config.sqlite_path.as_ref() + .ok_or_else(|| anyhow::anyhow!("SQLite path is required for sync operations"))?; + let db = rusqlite::Connection::open(sqlite_path)?; let mut ttc = TimeTravelControl::open(db)?; let s3_config = DatabaseBackupSourceS3Config { bucket: options @@ -459,7 +533,7 @@ async fn snapshot_read_endpoint( consistency: Consistency::Strong, }; - let result_ranges = state.sqlite.snapshot_read(requests, options).await?; + let result_ranges = state.database.snapshot_read(requests, options).await?; let res = result_ranges.into(); Ok(Protobuf(res)) @@ -472,7 +546,11 @@ async fn atomic_write_endpoint( ) -> Result, ApiError> { let atomic_write: AtomicWrite = atomic_write.try_into()?; - let res = state.sqlite.atomic_write(atomic_write).await?; + let res = state.database.atomic_write(atomic_write).await + .map_err(|e| { + log::error!("atomic_write failed: {}", e); + e + })?; Ok(Protobuf(res.into())) } @@ -483,7 +561,7 @@ async fn watch_endpoint( ) -> Result { let keys = watch.try_into()?; - let watcher = state.sqlite.watch(keys); + let watcher = state.database.watch(keys); let data_stream = watcher.map_ok(|outs| { let output = pb::WatchOutput::from(outs); @@ -677,6 +755,13 @@ impl From for ApiError { } } +impl From for ApiError { + fn from(err: deno_error::JsErrorBox) -> ApiError { + log::error!("Database error: {}", err); + ApiError::InternalServerError + } +} + struct Protobuf(T); impl IntoResponse for Protobuf { diff --git a/docker-compose.test.yml b/docker-compose.test.yml new file mode 100644 index 0000000..6e9ed25 --- /dev/null +++ b/docker-compose.test.yml @@ -0,0 +1,21 @@ +version: '3.8' + +services: + postgres: + image: postgres:15 + environment: + POSTGRES_DB: denokv_test + POSTGRES_USER: postgres + POSTGRES_PASSWORD: password + ports: + - "5432:5432" + volumes: + - postgres_data:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U postgres"] + interval: 5s + timeout: 5s + retries: 5 + +volumes: + postgres_data: \ No newline at end of file diff --git a/expand_disk.sh b/expand_disk.sh new file mode 100644 index 0000000..7877b7f --- /dev/null +++ b/expand_disk.sh @@ -0,0 +1,153 @@ +#!/bin/bash +# Script to expand 512GB disk on Azure VM +# Run this on the database server: 102.37.137.29 + +set -e # Exit on error + +echo "=== Expanding 512GB Disk ===" +echo "" + +# Step 1: Fix GPT table +echo "Step 1: Fixing GPT partition table..." +echo "Automatically answering prompts: Y, Y, w, Y" +echo "" + +# Use echo to pipe answers to gdisk prompts +# Order: Y (fix secondary header), Y (proceed), w (write), Y (confirm) +echo -e "Y\nY\nw\nY" | sudo gdisk /dev/nvme0n1 + +echo "" +echo "Step 1 complete: GPT table fixed" +echo "" + +# Step 2: Install growpart if needed +echo "Step 2: Installing growpart if needed..." +if ! command -v growpart &> /dev/null; then + sudo dnf install -y cloud-utils-growpart || sudo yum install -y cloud-utils-growpart + echo "growpart installed" +else + echo "growpart already installed" +fi +echo "" + +# Step 3: Extend partition 4 +echo "Step 3: Extending partition 4 to use full disk..." +sudo growpart /dev/nvme0n1 4 +echo "Partition extended" +echo "" + +# Step 4: Resize physical volume +echo "Step 4: Resizing LVM physical volume..." +sudo pvresize /dev/nvme0n1p4 +echo "Physical volume resized" +echo "" + +# Step 5: Extend logical volume +echo "Step 5: Extending logical volume to use all available space..." +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot +echo "Logical volume extended" +echo "" + +# Step 6: Resize filesystem +echo "Step 6: Resizing filesystem..." +sudo xfs_growfs / +echo "Filesystem resized" +echo "" + +# Step 7: Verify +echo "Step 7: Verifying disk space..." +echo "" +df -h +echo "" +echo "=== Disk expansion complete! ===" +echo "" +echo "Current disk status:" +sudo pvs +sudo vgs +sudo lvs + + +what does this mean [Pasted text #1 +45 lines] │ + └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ + + You're in the gdisk interactive prompt. The output means: + • GPT partition table is present and valid + • It's waiting for a command + + When you typed y, it showed the help menu because y isn't a command here. + To fix the GPT table and write it, type w (write), then confirm with Y when prompted. + + Run this command instead (non-interactive): + + + echo -e "w\ny" | sudo gdisk /dev/nvme0n1 + + This sends: + • w = write table to disk + • y = confirm + + Or if gdisk asks about fixing the secondary header first, use: + + echo -e "y\ny\nw\ny" | sudo gdisk /dev/nvme0n1 + + This sends: + • y = fix secondary header placement + • y = confirm fix + • w = write table + • y = confirm write + + After that, continue with the rest: + + 1 │# Install growpart + 2 │sudo dnf install -y cloud-utils-growpart + 3 │ + 4 │# Extend partition + 5 │sudo growpart /dev/nvme0n1 4 + 6 │ + 7 │# Resize physical volume + 8 │sudo pvresize /dev/nvme0n1p4 + 9 │ + 10 │# Extend logical volume + 11 │sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot + 12 │ + 13 │# Resize filesystem + 14 │sudo xfs_growfs / + 15 │ + 16 │# Verify + 17 │df -h + + Try the echo -e command above; it should work non-interactively. + + Next Steps: + + 1. Free up some space first (you were at 100%): + + + 1 │ sudo journalctl --vacuum-time=3d + 2 │ sudo dnf clean all + + 2. Restart PostgreSQL to ensure it recognizes the new space: + + + 1 │ sudo systemctl restart postgresql + 2 │ sudo systemctl status postgresql + + 3. Verify PostgreSQL is working: + + + sudo -u postgres psql -c "SELECT version();" + + 4. Check your DenoKV service: + + + sudo systemctl status denokv + + PostgreSQL will now have plenty of space for: + • Data files + • WAL files + • Log files + • Temporary files + + The connection recovery code we added earlier will also help handle any future transient connection issues. You're all set. + + diff --git a/fix-postgres-auth.sh b/fix-postgres-auth.sh new file mode 100755 index 0000000..507dc50 --- /dev/null +++ b/fix-postgres-auth.sh @@ -0,0 +1,87 @@ +#!/bin/bash + +# PostgreSQL Authentication Fix Script for Rocky Linux + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' + +print_status() { echo -e "${BLUE}[INFO]${NC} $1"; } +print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; } +print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; } +print_error() { echo -e "${RED}[ERROR]${NC} $1"; } + +echo "🔧 PostgreSQL Authentication Fix Script" +echo "=======================================" +echo "" + +# Check if PostgreSQL is running +if ! systemctl is-active --quiet postgresql; then + print_status "Starting PostgreSQL service..." + sudo systemctl start postgresql + sleep 2 +fi + +# Find pg_hba.conf +PG_HBA_PATHS=( + "/var/lib/pgsql/data/pg_hba.conf" + "/var/lib/postgresql/data/pg_hba.conf" + "/etc/postgresql/*/main/pg_hba.conf" +) + +PG_HBA_PATH="" +for path in "${PG_HBA_PATHS[@]}"; do + if [ -f "$path" ] || ls $path 2>/dev/null; then + PG_HBA_PATH="$path" + break + fi +done + +if [ -z "$PG_HBA_PATH" ]; then + print_error "Could not find pg_hba.conf file" + print_status "Trying to find PostgreSQL data directory..." + sudo -u postgres psql -c "SHOW data_directory;" 2>/dev/null || true + exit 1 +fi + +print_status "Found pg_hba.conf at: $PG_HBA_PATH" + +# Backup the original file +print_status "Creating backup of pg_hba.conf..." +sudo cp "$PG_HBA_PATH" "$PG_HBA_PATH.backup.$(date +%Y%m%d_%H%M%S)" + +# Update authentication methods +print_status "Updating authentication methods..." +sudo sed -i 's/local all all ident/local all all md5/g' "$PG_HBA_PATH" +sudo sed -i 's/local all all peer/local all all md5/g' "$PG_HBA_PATH" +sudo sed -i 's/local all all trust/local all all md5/g' "$PG_HBA_PATH" + +# Add explicit entry for denokv user if not present +if ! grep -q "denokv" "$PG_HBA_PATH"; then + print_status "Adding explicit entry for denokv user..." + echo "local denokv denokv md5" | sudo tee -a "$PG_HBA_PATH" +fi + +# Reload PostgreSQL configuration +print_status "Reloading PostgreSQL configuration..." +sudo systemctl reload postgresql + +# Test connection +print_status "Testing database connection..." +if PGPASSWORD='denokv_password' psql -h localhost -U denokv -d denokv -c "SELECT 1;" >/dev/null 2>&1; then + print_success "Database connection test successful!" +else + print_warning "Database connection test failed" + print_status "You may need to restart PostgreSQL: sudo systemctl restart postgresql" +fi + +print_success "PostgreSQL authentication fix completed!" +echo "" +print_status "If you still have issues, try:" +echo " sudo systemctl restart postgresql" +echo " ./manage-services.sh restart" \ No newline at end of file diff --git a/fix_disk_remote.sh b/fix_disk_remote.sh new file mode 100644 index 0000000..94428db --- /dev/null +++ b/fix_disk_remote.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# Script to fix 512GB disk not being recognized on remote server +# Run this on the database server: 102.37.137.29 + +echo "=== Step 1: Check Current Disk Status ===" +lsblk +echo "" +echo "=== Step 2: Check Partition Table ===" +sudo fdisk -l /dev/sda 2>/dev/null || sudo fdisk -l /dev/nvme0n1 2>/dev/null +echo "" +echo "=== Step 3: Check LVM Status ===" +sudo pvs +sudo vgs +sudo lvs +echo "" +echo "=== Step 4: Check Filesystem Usage ===" +df -h +echo "" +echo "=== If disk shows 512GB but partition is small, continue below ===" +echo "" +echo "Installing growpart if needed..." +sudo dnf install -y cloud-utils-growpart 2>/dev/null || sudo yum install -y cloud-utils-growpart 2>/dev/null +echo "" +echo "=== Identify the disk device ===" +echo "Run: lsblk to see which device (sda or nvme0n1)" +echo "Then run the appropriate commands below:" +echo "" +echo "For /dev/sda:" +echo " sudo growpart /dev/sda 3" +echo " sudo pvresize /dev/sda3" +echo " sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot" +echo " sudo xfs_growfs /" +echo "" +echo "For /dev/nvme0n1:" +echo " sudo growpart /dev/nvme0n1 3" +echo " sudo pvresize /dev/nvme0n1p3" +echo " sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot" +echo " sudo xfs_growfs /" diff --git a/free_postgres_space.sh b/free_postgres_space.sh new file mode 100644 index 0000000..3237af8 --- /dev/null +++ b/free_postgres_space.sh @@ -0,0 +1,120 @@ +#!/bin/bash +# Script to safely free up PostgreSQL-related disk space + +echo "=== Freeing Up PostgreSQL Disk Space ===" +echo "" + +# Find PostgreSQL data directory +PG_DATA_DIR="/var/lib/pgsql/data" +if [ ! -d "$PG_DATA_DIR" ]; then + PG_DATA_DIR="/var/lib/postgresql/*/data" +fi + +# Show current disk usage +echo "Current disk usage:" +df -h / +echo "" + +# 1. Clean old PostgreSQL log files (keep last 7 days) +echo "1. Cleaning PostgreSQL log files older than 7 days..." +LOG_COUNT=$(sudo find $PG_DATA_DIR/log -name "*.log" -mtime +7 2>/dev/null | wc -l) +if [ "$LOG_COUNT" -gt 0 ]; then + echo " Found $LOG_COUNT log files to clean" + sudo find $PG_DATA_DIR/log -name "*.log" -mtime +7 -delete 2>/dev/null + echo " PostgreSQL log files cleaned" +else + echo " No old log files found in $PG_DATA_DIR/log" +fi + +# Try alternative log location +LOG_COUNT2=$(sudo find $PG_DATA_DIR/pg_log -name "*.log" -mtime +7 2>/dev/null | wc -l) +if [ "$LOG_COUNT2" -gt 0 ]; then + echo " Found $LOG_COUNT2 log files to clean in pg_log" + sudo find $PG_DATA_DIR/pg_log -name "*.log" -mtime +7 -delete 2>/dev/null + echo " PostgreSQL pg_log files cleaned" +fi +echo "" + +# 2. Clean system journal logs (keep last 3 days) +echo "2. Cleaning system journal logs (keeping last 3 days)..." +BEFORE=$(sudo journalctl --disk-usage 2>/dev/null | grep -oP '\d+\.\d+[GM]' || echo "0") +sudo journalctl --vacuum-time=3d +AFTER=$(sudo journalctl --disk-usage 2>/dev/null | grep -oP '\d+\.\d+[GM]' || echo "0") +echo " Journal logs cleaned (before: $BEFORE, after: $AFTER)" +echo "" + +# 3. Clean package cache +echo "3. Cleaning package cache..." +sudo dnf clean all 2>/dev/null || sudo yum clean all 2>/dev/null +echo " Package cache cleaned" +echo "" + +# 4. Show new disk usage +echo "New disk usage:" +df -h / +echo "" + +# 5. Show space freed +echo "=== Space Cleanup Complete ===" +echo "" +echo "To see what's still using space:" +echo " sudo du -h --max-depth=1 / | sort -hr | head -10" + + + + + Run these commands on your server to find which PostgreSQL files are using space: + + Check What's Using Space + + + 1 │# 1. Check PostgreSQL data directory size + 2 │sudo du -sh /var/lib/pgsql/data + 3 │ + 4 │# 2. Find largest PostgreSQL files/directories + 5 │sudo du -h --max-depth=2 /var/lib/pgsql/data | sort -hr | head -20 + 6 │ + 7 │# 3. Check PostgreSQL log files + 8 │sudo du -sh /var/lib/pgsql/data/log/* 2>/dev/null + 9 │sudo du -sh /var/lib/pgsql/data/pg_log/* 2>/dev/null + 10 │ + 11 │# 4. List log files by size + 12 │sudo find /var/lib/pgsql/data -name "*.log" -type f -exec du -h {} \; | sort -hr | head -10 + 13 │ + 14 │# 5. Check WAL files (Write-Ahead Log) + 15 │sudo du -sh /var/lib/pgsql/data/pg_wal + 16 │sudo ls -lh /var/lib/pgsql/data/pg_wal/* | tail -20 + 17 │ + 18 │# 6. Check system journal (often the biggest) + 19 │sudo journalctl --disk-usage + 20 │ + 21 │# 7. Top 10 largest directories on root + 22 │sudo du -h --max-depth=1 / | sort -hr | head -10 + + + Free Up Space Safely + + + 1 │# 1. Clean old PostgreSQL logs (keep last 7 days) + 2 │sudo find /var/lib/pgsql/data/log -name "*.log" -mtime +7 -delete + 3 │sudo find /var/lib/pgsql/data/pg_log -name "*.log" -mtime +7 -delete + 4 │ + 5 │# 2. Clean system journal (keep last 3 days) - usually frees the most space + 6 │sudo journalctl --vacuum-time=3d + 7 │ + 8 │# 3. Clean package cache + 9 │sudo dnf clean all + 10 │ + 11 │# 4. Check space freed + 12 │df -h / + + + Quick One-Liner to See Biggest PostgreSQL Files + + + sudo du -h /var/lib/pgsql/data | sort -hr | head -20 + + Start with the check commands to see what's using space, then run the cleanup commands. The system journal (journalctl) is often the largest consumer. + After freeing space, restart PostgreSQL: + + sudo systemctl restart postgresq \ No newline at end of file diff --git a/manage-services.sh b/manage-services.sh new file mode 100755 index 0000000..ad5528e --- /dev/null +++ b/manage-services.sh @@ -0,0 +1,139 @@ +#!/bin/bash + +# Service management script for DenoKV on Rocky Linux + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' + +print_status() { echo -e "${BLUE}[INFO]${NC} $1"; } +print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; } +print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; } +print_error() { echo -e "${RED}[ERROR]${NC} $1"; } + +case "${1:-help}" in + start) + print_status "Starting all services..." + + # Start PostgreSQL service + print_status "Starting PostgreSQL service..." + sudo systemctl start postgresql + + # Wait for PostgreSQL + until sudo -u postgres pg_isready; do + echo "Waiting for PostgreSQL..." + sleep 2 + done + print_success "PostgreSQL service started" + + # Start DenoKV server + print_status "Starting DenoKV server..." + source ~/.cargo/env + source .env 2>/dev/null || true + + if pgrep -f "denokv.*serve" > /dev/null; then + print_warning "DenoKV server is already running" + else + nohup cargo run --release -- serve --addr 0.0.0.0:4512 > denokv.log 2>&1 & + sleep 2 + if pgrep -f "denokv.*serve" > /dev/null; then + print_success "DenoKV server started" + else + print_error "Failed to start DenoKV server" + fi + fi + ;; + + stop) + print_status "Stopping DenoKV server..." + + # Stop DenoKV server only + if pgrep -f "denokv.*serve" > /dev/null; then + pkill -f "denokv.*serve" + print_success "DenoKV server stopped" + else + print_warning "DenoKV server was not running" + fi + + print_status "PostgreSQL service remains running (persistent)" + ;; + + restart) + $0 stop + sleep 2 + $0 start + ;; + + stop-postgres) + print_status "Stopping PostgreSQL service..." + sudo systemctl stop postgresql + print_success "PostgreSQL service stopped" + print_warning "Note: DenoKV server will not work without PostgreSQL" + ;; + + start-postgres) + print_status "Starting PostgreSQL service..." + sudo systemctl start postgresql + until sudo -u postgres pg_isready; do + echo "Waiting for PostgreSQL..." + sleep 2 + done + print_success "PostgreSQL service started" + ;; + + status) + print_status "Service Status:" + echo "" + + # Check PostgreSQL service + if systemctl is-active --quiet postgresql; then + print_success "PostgreSQL Service: Running" + else + print_warning "PostgreSQL Service: Stopped" + fi + + # Check DenoKV server + if pgrep -f "denokv.*serve" > /dev/null; then + print_success "DenoKV Server: Running (PID: $(pgrep -f 'denokv.*serve'))" + else + print_warning "DenoKV Server: Stopped" + fi + + # Check port 4512 + if netstat -tlnp 2>/dev/null | grep -q ":4512 "; then + print_success "Port 4512: Open" + else + print_warning "Port 4512: Closed" + fi + ;; + + logs) + if [ -f "denokv.log" ]; then + tail -f denokv.log + else + print_warning "No log file found" + fi + ;; + + *) + echo "DenoKV Service Manager" + echo "Usage: $0 {start|stop|restart|status|logs|start-postgres|stop-postgres}" + echo "" + echo "Commands:" + echo " start - Start DenoKV server (PostgreSQL must be running)" + echo " stop - Stop DenoKV server only (PostgreSQL stays running)" + echo " restart - Restart DenoKV server only" + echo " status - Show service status" + echo " logs - Show DenoKV server logs" + echo " start-postgres - Start PostgreSQL service" + echo " stop-postgres - Stop PostgreSQL service (use with caution)" + echo "" + echo "Note: PostgreSQL runs as a persistent system service" + echo " DenoKV server can be started/stopped independently" + ;; +esac \ No newline at end of file diff --git a/postgres/Cargo.toml b/postgres/Cargo.toml new file mode 100644 index 0000000..3e6de52 --- /dev/null +++ b/postgres/Cargo.toml @@ -0,0 +1,31 @@ +[package] +name = "denokv_postgres" +version = "0.13.0" +edition = "2021" +license = "MIT" +repository = "https://github.com/denoland/denokv" +authors = ["rawkakani"] + +[lib] +name = "denokv_postgres" +path = "lib.rs" + +[dependencies] +denokv_proto = { workspace = true } +async-trait = { workspace = true } +tokio = { workspace = true } +tokio-postgres = "0.7" +deadpool-postgres = "0.10" +serde = { workspace = true } +serde_json = { workspace = true } +chrono = { workspace = true } +deno_error = { workspace = true } +futures = { workspace = true } +async-stream = { workspace = true } +bytes = { workspace = true } +uuid = { workspace = true } +rand = { workspace = true } +log = { workspace = true } +thiserror = { workspace = true } +clap = { workspace = true } +rusqlite = { workspace = true } \ No newline at end of file diff --git a/postgres/backend.rs b/postgres/backend.rs new file mode 100644 index 0000000..34be755 --- /dev/null +++ b/postgres/backend.rs @@ -0,0 +1,629 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use std::collections::HashMap; +use std::num::NonZeroU32; + +use chrono::{DateTime, Utc}; +use deadpool_postgres::{Client, Pool}; +use denokv_proto::{ + AtomicWrite, Check, CommitResult, Enqueue, KvEntry, KvValue, Mutation, MutationKind, + ReadRange, Versionstamp, +}; +use serde_json::Value; +use tokio_postgres::Row; + +use crate::error::{PostgresError, PostgresResult}; +use crate::message_handle::PostgresMessageHandle; + +/// PostgreSQL backend implementation +pub struct PostgresBackend { + pub pool: Pool, +} + +impl PostgresBackend { + pub fn new(pool: Pool) -> Self { + Self { pool } + } + + /// Initialize the database schema + pub async fn initialize_schema(&self) -> PostgresResult<()> { + let conn = self.pool.get().await?; + + + // Create the main KV table + conn.execute( + r#" + CREATE TABLE IF NOT EXISTS kv_store ( + key BYTEA PRIMARY KEY, + value BYTEA NOT NULL, + value_encoding INTEGER NOT NULL, + versionstamp BYTEA NOT NULL, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + expires_at BIGINT + ) + "#, + &[], + ).await?; + + // Create indexes for performance + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_kv_versionstamp ON kv_store(versionstamp)", + &[], + ).await?; + + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_kv_expires_at ON kv_store(expires_at) WHERE expires_at IS NOT NULL", + &[], + ).await?; + + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_kv_updated_at ON kv_store(updated_at)", + &[], + ).await?; + + // Create queue tables + conn.execute( + r#" + CREATE TABLE IF NOT EXISTS queue_messages ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + payload BYTEA NOT NULL, + deadline BIGINT NOT NULL, + keys_if_undelivered BYTEA[] NOT NULL, + backoff_schedule INTEGER[], + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + retry_count INTEGER DEFAULT 0 + ) + "#, + &[], + ).await?; + + conn.execute( + r#" + CREATE TABLE IF NOT EXISTS queue_running ( + message_id UUID PRIMARY KEY REFERENCES queue_messages(id), + deadline BIGINT NOT NULL, + started_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() + ) + "#, + &[], + ).await?; + + // Create indexes for queue + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_queue_deadline ON queue_messages(deadline)", + &[], + ).await?; + + conn.execute( + "CREATE INDEX IF NOT EXISTS idx_queue_running_deadline ON queue_running(deadline)", + &[], + ).await?; + + // Monotonic version counter — matches SQLite's data_version table. + // The single row is locked by UPDATE during atomic_write, which + // serializes all writers without needing SERIALIZABLE isolation. + conn.execute( + r#" + CREATE TABLE IF NOT EXISTS data_version ( + k INTEGER PRIMARY KEY DEFAULT 0, + version BIGINT NOT NULL DEFAULT 0 + ) + "#, + &[], + ).await?; + + // Seed the row if it doesn't exist + conn.execute( + "INSERT INTO data_version (k, version) VALUES (0, 0) ON CONFLICT DO NOTHING", + &[], + ).await?; + + Ok(()) + } + + /// Read a range of keys, excluding expired entries. + pub async fn read_range( + &self, + conn: &Client, + request: &ReadRange, + ) -> PostgresResult> { + let now_ms = crate::time::utc_now().timestamp_millis(); + let query = if request.reverse { + r#" + SELECT key, value, value_encoding, versionstamp + FROM kv_store + WHERE key >= $1 AND key < $2 + AND (expires_at IS NULL OR expires_at > $4) + ORDER BY key DESC + LIMIT $3 + "# + } else { + r#" + SELECT key, value, value_encoding, versionstamp + FROM kv_store + WHERE key >= $1 AND key < $2 + AND (expires_at IS NULL OR expires_at > $4) + ORDER BY key ASC + LIMIT $3 + "# + }; + + let rows = conn.query(query, &[ + &request.start, + &request.end, + &(request.limit.get() as i64), + &now_ms, + ]).await?; + + let mut entries = Vec::new(); + for row in rows { + let key: Vec = row.get("key"); + let value: Vec = row.get("value"); + let encoding: i32 = row.get("value_encoding"); + let versionstamp: Vec = row.get("versionstamp"); + + let kv_value = match encoding { + 1 => KvValue::V8(value), + 2 => { + let mut buf = [0; 8]; + buf.copy_from_slice(&value); + KvValue::U64(u64::from_le_bytes(buf)) + } + 3 => KvValue::Bytes(value), + _ => return Err(PostgresError::InvalidData(format!("Unknown encoding: {}", encoding))), + }; + + let mut versionstamp_array = [0; 10]; + versionstamp_array.copy_from_slice(&versionstamp); + + entries.push(KvEntry { + key, + value: kv_value, + versionstamp: versionstamp_array, + }); + } + + Ok(entries) + } + + /// Perform an atomic write operation. + /// + /// Concurrency is handled by a monotonic version counter in the + /// `data_version` table (matching the SQLite backend design). + /// `UPDATE ... RETURNING` takes an exclusive row lock on the counter, + /// which serializes all writers under plain READ COMMITTED isolation — + /// no SERIALIZABLE needed, no aborted transactions to retry. + pub async fn atomic_write( + &self, + conn: &mut Client, + write: AtomicWrite, + ) -> PostgresResult> { + let tx = conn.transaction().await?; + + // Lock the version counter first — this serializes all writers. + // The row lock is held until tx.commit() / rollback. + let new_version: i64 = tx.query_one( + "UPDATE data_version SET version = version + 1 WHERE k = 0 RETURNING version", + &[], + ).await?.get(0); + + // Perform checks — treat expired keys as non-existent + let now_ms = crate::time::utc_now().timestamp_millis(); + for check in &write.checks { + let row = tx.query_opt( + "SELECT versionstamp FROM kv_store WHERE key = $1 AND (expires_at IS NULL OR expires_at > $2)", + &[&check.key, &now_ms], + ).await?; + + let current_versionstamp = row.map(|r| r.get::<_, Vec>("versionstamp")); + + if let Some(expected) = &check.versionstamp { + if current_versionstamp.as_ref().map(|v| v.as_slice()) != Some(expected.as_slice()) { + return Ok(None); // Check failed + } + } else if current_versionstamp.is_some() { + return Ok(None); // Expected key to not exist, but it does + } + } + + // Convert version to 10-byte versionstamp (matches SQLite format) + let versionstamp = version_to_versionstamp(new_version); + + // Perform mutations + for mutation in &write.mutations { + match &mutation.kind { + MutationKind::Set(value) => { + let (value_bytes, encoding) = self.encode_value(value); + let expires_at = mutation.expire_at; + + tx.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, expires_at, updated_at) + VALUES ($1, $2, $3, $4, $5, NOW()) + ON CONFLICT (key) DO UPDATE SET + value = EXCLUDED.value, + value_encoding = EXCLUDED.value_encoding, + versionstamp = EXCLUDED.versionstamp, + expires_at = EXCLUDED.expires_at, + updated_at = NOW() + "#, + &[&mutation.key, &value_bytes, &(encoding as i32), &versionstamp.as_slice(), &expires_at.map(|dt| dt.timestamp_millis())], + ).await?; + } + MutationKind::Delete => { + tx.execute( + "DELETE FROM kv_store WHERE key = $1", + &[&mutation.key], + ).await?; + } + MutationKind::Sum { value, .. } => { + self.handle_sum_mutation(&tx, &mutation.key, value, &versionstamp).await?; + } + MutationKind::Min(value) => { + self.handle_min_mutation(&tx, &mutation.key, value, &versionstamp).await?; + } + MutationKind::Max(value) => { + self.handle_max_mutation(&tx, &mutation.key, value, &versionstamp).await?; + } + MutationKind::SetSuffixVersionstampedKey(value) => { + let mut new_key = mutation.key.clone(); + new_key.extend_from_slice(&versionstamp); + + let (value_bytes, encoding) = self.encode_value(value); + let expires_at = mutation.expire_at; + + tx.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, expires_at, updated_at) + VALUES ($1, $2, $3, $4, $5, NOW()) + "#, + &[&new_key, &value_bytes, &(encoding as i32), &versionstamp.as_slice(), &expires_at.map(|dt| dt.timestamp_millis())], + ).await?; + } + } + } + + // Handle enqueues + for enqueue in &write.enqueues { + let keys_json = serde_json::to_string(&enqueue.keys_if_undelivered)?; + let backoff_json = enqueue.backoff_schedule.as_ref().map(|b| serde_json::to_string(b)).transpose()?; + + tx.execute( + r#" + INSERT INTO queue_messages (payload, deadline, keys_if_undelivered, backoff_schedule) + VALUES ($1, $2, $3, $4) + "#, + &[&enqueue.payload, &enqueue.deadline.timestamp_millis(), &keys_json, &backoff_json], + ).await?; + } + + tx.commit().await?; + Ok(Some(CommitResult { versionstamp })) + } + + /// Handle sum mutation + async fn handle_sum_mutation( + &self, + tx: &tokio_postgres::Transaction<'_>, + key: &[u8], + value: &KvValue, + versionstamp: &Versionstamp, + ) -> PostgresResult<()> { + let (value_bytes, encoding) = self.encode_value(value); + + if encoding != 2 { + return Err(PostgresError::InvalidData("Sum operation only supports U64 values".to_string())); + } + + let sum_value = match value { + KvValue::U64(v) => *v as i64, + _ => return Err(PostgresError::InvalidData("Sum operation only supports U64 values".to_string())), + }; + + // First, try to get the current value + let current_row = tx.query_opt( + "SELECT value FROM kv_store WHERE key = $1 AND value_encoding = 2", + &[&key], + ).await?; + + let new_value = if let Some(row) = current_row { + // Parse current value as i64 and add sum_value + let current_bytes: Vec = row.get(0); + if current_bytes.len() == 8 { + let mut bytes_array = [0u8; 8]; + bytes_array.copy_from_slice(¤t_bytes); + let current_int = i64::from_le_bytes(bytes_array); + current_int + sum_value + } else { + sum_value + } + } else { + sum_value + }; + + let new_value_bytes = new_value.to_le_bytes().to_vec(); + + tx.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, updated_at) + VALUES ($1, $2, 2, $3, NOW()) + ON CONFLICT (key) DO UPDATE SET + value = $2, + versionstamp = EXCLUDED.versionstamp, + updated_at = NOW() + WHERE kv_store.value_encoding = 2 + "#, + &[&key, &new_value_bytes, &versionstamp.as_slice()], + ).await?; + + Ok(()) + } + + /// Handle min mutation + async fn handle_min_mutation( + &self, + tx: &tokio_postgres::Transaction<'_>, + key: &[u8], + value: &KvValue, + versionstamp: &Versionstamp, + ) -> PostgresResult<()> { + let (value_bytes, encoding) = self.encode_value(value); + + if encoding != 2 { + return Err(PostgresError::InvalidData("Min operation only supports U64 values".to_string())); + } + + let min_value = match value { + KvValue::U64(v) => *v as i64, + _ => return Err(PostgresError::InvalidData("Min operation only supports U64 values".to_string())), + }; + + // First, try to get the current value + let current_row = tx.query_opt( + "SELECT value FROM kv_store WHERE key = $1 AND value_encoding = 2", + &[&key], + ).await?; + + let new_value = if let Some(row) = current_row { + // Parse current value as i64 and take minimum + let current_bytes: Vec = row.get(0); + if current_bytes.len() == 8 { + let mut bytes_array = [0u8; 8]; + bytes_array.copy_from_slice(¤t_bytes); + let current_int = i64::from_le_bytes(bytes_array); + current_int.min(min_value) + } else { + min_value + } + } else { + min_value + }; + + let new_value_bytes = new_value.to_le_bytes().to_vec(); + + tx.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, updated_at) + VALUES ($1, $2, 2, $3, NOW()) + ON CONFLICT (key) DO UPDATE SET + value = $2, + versionstamp = EXCLUDED.versionstamp, + updated_at = NOW() + WHERE kv_store.value_encoding = 2 + "#, + &[&key, &new_value_bytes, &versionstamp.as_slice()], + ).await?; + + Ok(()) + } + + /// Handle max mutation + async fn handle_max_mutation( + &self, + tx: &tokio_postgres::Transaction<'_>, + key: &[u8], + value: &KvValue, + versionstamp: &Versionstamp, + ) -> PostgresResult<()> { + let (value_bytes, encoding) = self.encode_value(value); + + if encoding != 2 { + return Err(PostgresError::InvalidData("Max operation only supports U64 values".to_string())); + } + + let max_value = match value { + KvValue::U64(v) => *v as i64, + _ => return Err(PostgresError::InvalidData("Max operation only supports U64 values".to_string())), + }; + + // First, try to get the current value + let current_row = tx.query_opt( + "SELECT value FROM kv_store WHERE key = $1 AND value_encoding = 2", + &[&key], + ).await?; + + let new_value = if let Some(row) = current_row { + // Parse current value as i64 and take maximum + let current_bytes: Vec = row.get(0); + if current_bytes.len() == 8 { + let mut bytes_array = [0u8; 8]; + bytes_array.copy_from_slice(¤t_bytes); + let current_int = i64::from_le_bytes(bytes_array); + current_int.max(max_value) + } else { + max_value + } + } else { + max_value + }; + + let new_value_bytes = new_value.to_le_bytes().to_vec(); + + tx.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, updated_at) + VALUES ($1, $2, 2, $3, NOW()) + ON CONFLICT (key) DO UPDATE SET + value = $2, + versionstamp = EXCLUDED.versionstamp, + updated_at = NOW() + WHERE kv_store.value_encoding = 2 + "#, + &[&key, &new_value_bytes, &versionstamp.as_slice()], + ).await?; + + Ok(()) + } + + /// Dequeue the next message from the queue + pub async fn dequeue_next_message( + &self, + conn: &mut Client, + ) -> PostgresResult> { + let tx = conn.transaction().await?; + + // Find the next message to process + let row = tx.query_opt( + r#" + SELECT id, payload, deadline, keys_if_undelivered, backoff_schedule + FROM queue_messages + WHERE deadline <= NOW() + AND id NOT IN (SELECT message_id FROM queue_running) + ORDER BY deadline ASC + LIMIT 1 + FOR UPDATE SKIP LOCKED + "#, + &[], + ).await?; + + if let Some(row) = row { + let id_str: String = row.get("id"); + let id = uuid::Uuid::parse_str(&id_str)?; + let payload: Vec = row.get("payload"); + let deadline_str: String = row.get("deadline"); + let deadline_naive = chrono::NaiveDateTime::parse_from_str(&deadline_str, "%Y-%m-%d %H:%M:%S%.f") + .map_err(|e| PostgresError::InvalidData(format!("Invalid deadline format: {}", e)))?; + let deadline: DateTime = DateTime::from_naive_utc_and_offset(deadline_naive, Utc); + let keys_json: String = row.get("keys_if_undelivered"); + let keys_if_undelivered: Vec> = serde_json::from_str(&keys_json)?; + let backoff_json: Option = row.get("backoff_schedule"); + let backoff_schedule: Option> = if let Some(json) = backoff_json { + Some(serde_json::from_str(&json)?) + } else { + None + }; + + // Move to running table + tx.execute( + r#" + INSERT INTO queue_running (message_id, deadline, started_at, updated_at) + VALUES ($1, $2, NOW(), NOW()) + "#, + &[&id_str, &deadline_str], + ).await?; + + tx.commit().await?; + + Ok(Some(PostgresMessageHandle { + id, + payload: Some(payload), + pool: self.pool.clone(), + })) + } else { + Ok(None) + } + } + + /// Delete all expired keys. Returns the number of rows removed. + pub async fn collect_expired(&self) -> PostgresResult { + let conn = self.pool.get().await?; + let now_ms = crate::time::utc_now().timestamp_millis(); + let deleted = conn.execute( + "DELETE FROM kv_store WHERE expires_at IS NOT NULL AND expires_at <= $1", + &[&now_ms], + ).await?; + Ok(deleted) + } + + /// Requeue messages stuck in queue_running past their deadline. + /// This recovers from dead workers that never finished their messages. + /// Returns the number of messages requeued. + pub async fn queue_cleanup(&self) -> PostgresResult { + let mut conn = self.pool.get().await?; + let tx = conn.transaction().await?; + let now_ms = crate::time::utc_now().timestamp_millis(); + + // Find running messages past their deadline (dead worker recovery) + let rows = tx.query( + "SELECT message_id FROM queue_running WHERE deadline <= $1 LIMIT 100", + &[&now_ms], + ).await?; + + let mut requeued = 0u64; + for row in &rows { + let message_id: String = row.get("message_id"); + + // Fetch the original message to get backoff info + let msg_row = tx.query_opt( + r#"SELECT backoff_schedule, retry_count + FROM queue_messages WHERE id = $1"#, + &[&message_id], + ).await?; + + // Remove from running table + tx.execute("DELETE FROM queue_running WHERE message_id = $1", &[&message_id]).await?; + + if let Some(msg) = msg_row { + let backoff_json: Option = msg.get("backoff_schedule"); + let retry_count: i32 = msg.get("retry_count"); + let backoff_schedule: Vec = backoff_json + .and_then(|j| serde_json::from_str(&j).ok()) + .unwrap_or_default(); + + if !backoff_schedule.is_empty() { + let delay_ms = backoff_schedule[0] as i64; + let new_deadline = now_ms + delay_ms; + let remaining = serde_json::to_string(&backoff_schedule[1..]) + .unwrap_or_else(|_| "[]".to_string()); + + tx.execute( + r#"UPDATE queue_messages + SET deadline = $1, backoff_schedule = $2, retry_count = $3 + WHERE id = $4"#, + &[&new_deadline, &remaining, &(retry_count + 1), &message_id], + ).await?; + requeued += 1; + } else { + // No retries left — delete the message + tx.execute("DELETE FROM queue_messages WHERE id = $1", &[&message_id]).await?; + } + } + } + + tx.commit().await?; + Ok(requeued) + } + + /// Encode a value for storage + fn encode_value(&self, value: &KvValue) -> (Vec, i32) { + match value { + KvValue::V8(v) => (v.clone(), 1), + KvValue::Bytes(v) => (v.clone(), 3), + KvValue::U64(v) => { + let mut buf = [0; 8]; + buf.copy_from_slice(&v.to_le_bytes()); + (buf.to_vec(), 2) + } + } + } +} + +/// Convert a monotonic i64 version to a 10-byte versionstamp. +/// Matches the SQLite backend format: 8-byte big-endian version + 2 zero bytes. +fn version_to_versionstamp(version: i64) -> Versionstamp { + let mut versionstamp = [0u8; 10]; + versionstamp[..8].copy_from_slice(&version.to_be_bytes()); + versionstamp +} \ No newline at end of file diff --git a/postgres/config.rs b/postgres/config.rs new file mode 100644 index 0000000..98485a0 --- /dev/null +++ b/postgres/config.rs @@ -0,0 +1,59 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use serde::{Deserialize, Serialize}; + +/// Configuration for PostgreSQL backend +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct PostgresConfig { + /// PostgreSQL connection URL + /// Example: postgresql://user:password@localhost:5432/denokv + pub url: String, + + /// Maximum number of connections in the pool + pub max_connections: usize, + + /// Connection timeout in seconds + pub connection_timeout: u64, + + /// Statement timeout in seconds + pub statement_timeout: u64, +} + +impl Default for PostgresConfig { + fn default() -> Self { + Self { + url: "postgresql://postgres:password@localhost:5432/denokv".to_string(), + max_connections: 10, + connection_timeout: 30, + statement_timeout: 60, + } + } +} + +impl PostgresConfig { + /// Create a new PostgreSQL configuration + pub fn new(url: String) -> Self { + Self { + url, + ..Default::default() + } + } + + /// Set the maximum number of connections + pub fn with_max_connections(mut self, max_connections: usize) -> Self { + self.max_connections = max_connections; + self + } + + /// Set the connection timeout + pub fn with_connection_timeout(mut self, timeout: u64) -> Self { + self.connection_timeout = timeout; + self + } + + /// Set the statement timeout + pub fn with_statement_timeout(mut self, timeout: u64) -> Self { + self.statement_timeout = timeout; + self + } +} \ No newline at end of file diff --git a/postgres/error.rs b/postgres/error.rs new file mode 100644 index 0000000..c94aea8 --- /dev/null +++ b/postgres/error.rs @@ -0,0 +1,89 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use deno_error::{JsErrorBox, JsErrorClass}; +use thiserror::Error; + +/// PostgreSQL-specific errors +#[derive(Error, Debug)] +pub enum PostgresError { + #[error("Invalid configuration: {0}")] + InvalidConfig(String), + + #[error("Connection failed: {0}")] + ConnectionFailed(String), + + #[error("Database error: {0}")] + DatabaseError(String), + + #[error("Transaction error: {0}")] + TransactionError(String), + + #[error("Query error: {0}")] + QueryError(String), + + #[error("Serialization error: {0}")] + SerializationError(String), + + #[error("Deserialization error: {0}")] + DeserializationError(String), + + #[error("Invalid data: {0}")] + InvalidData(String), + + #[error("Timeout: {0}")] + Timeout(String), + + #[error("Pool error: {0}")] + PoolError(String), +} + +impl From for PostgresError { + fn from(err: tokio_postgres::Error) -> Self { + PostgresError::DatabaseError(err.to_string()) + } +} + +impl From for PostgresError { + fn from(err: deadpool_postgres::PoolError) -> Self { + PostgresError::PoolError(err.to_string()) + } +} + +impl From for PostgresError { + fn from(err: serde_json::Error) -> Self { + PostgresError::SerializationError(err.to_string()) + } +} + +impl From for PostgresError { + fn from(err: uuid::Error) -> Self { + PostgresError::InvalidData(err.to_string()) + } +} + +impl JsErrorClass for PostgresError { + fn get_class(&self) -> std::borrow::Cow<'static, str> { + std::borrow::Cow::Borrowed("PostgresError") + } + + fn get_message(&self) -> std::borrow::Cow<'static, str> { + std::borrow::Cow::Owned(self.to_string()) + } + + fn get_additional_properties(&self) -> Box, deno_error::PropertyValue)> + 'static> { + Box::new(std::iter::empty()) + } + + fn get_ref(&self) -> &(dyn std::error::Error + Send + Sync + 'static) { + self + } +} + +impl From for JsErrorBox { + fn from(err: PostgresError) -> Self { + JsErrorBox::generic(err.to_string()) + } +} + +/// Result type for PostgreSQL operations +pub type PostgresResult = Result; \ No newline at end of file diff --git a/postgres/lib.rs b/postgres/lib.rs new file mode 100644 index 0000000..65820dd --- /dev/null +++ b/postgres/lib.rs @@ -0,0 +1,231 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +mod backend; +mod config; +mod error; +mod message_handle; +mod notifier; +mod time; + +use std::collections::HashMap; +use std::pin::Pin; +use std::sync::Arc; +use std::time::Duration; + +use async_stream::try_stream; +use async_trait::async_trait; +use chrono::DateTime; +use chrono::Utc; +use deadpool_postgres::{Config, Pool, Runtime, Manager}; +use deno_error::JsErrorBox; +use denokv_proto::{ + AtomicWrite, CommitResult, Database, KvEntry, KvValue, QueueMessageHandle, + ReadRange, ReadRangeOutput, SnapshotReadOptions, Versionstamp, WatchKeyOutput, +}; +use futures::Stream; +use tokio::sync::{watch, RwLock}; +use tokio_postgres::NoTls; + +pub use config::PostgresConfig; +pub use error::{PostgresError, PostgresResult}; + +use backend::PostgresBackend; +use message_handle::PostgresMessageHandle; +use notifier::PostgresNotifier; + +/// PostgreSQL implementation of the DenoKV Database trait +#[derive(Clone)] +pub struct Postgres { + pool: Pool, + notifier: PostgresNotifier, + backend: Arc, +} + +impl Postgres { + /// Create a new PostgreSQL database instance + pub async fn new(config: PostgresConfig) -> PostgresResult { + // Parse the connection string + let pg_config = config.url.parse::() + .map_err(|e| PostgresError::InvalidConfig(format!("Invalid PostgreSQL URL: {}", e)))?; + + // Create deadpool manager + let manager = Manager::new(pg_config, NoTls); + + // Create the connection pool + let pool = Pool::builder(manager) + .max_size(config.max_connections) + .build() + .map_err(|e| PostgresError::ConnectionFailed(format!("Failed to create connection pool: {}", e)))?; + + // Test the connection + let conn = pool.get().await + .map_err(|e| PostgresError::ConnectionFailed(format!("Failed to get connection: {}", e)))?; + + // Initialize the database schema + let backend = Arc::new(PostgresBackend::new(pool.clone())); + backend.initialize_schema().await?; + + // Create notifier + let notifier = PostgresNotifier::new(); + + let pg = Postgres { + pool, + notifier, + backend, + }; + + // Spawn background tasks matching SQLite backend behaviour: + // 1. Periodic expired-key collection (every 60 s) + // 2. Periodic queue cleanup — requeue messages stuck in queue_running + // past their deadline (every 30 s) + { + let backend = pg.backend.clone(); + tokio::spawn(async move { + loop { + tokio::time::sleep(Duration::from_secs(60)).await; + match backend.collect_expired().await { + Ok(n) if n > 0 => { + eprintln!("[denokv/postgres] collected {n} expired key(s)"); + } + Err(e) => { + eprintln!("[denokv/postgres] collect_expired error: {e}"); + } + _ => {} // nothing to collect + } + } + }); + } + { + let backend = pg.backend.clone(); + tokio::spawn(async move { + loop { + tokio::time::sleep(Duration::from_secs(30)).await; + match backend.queue_cleanup().await { + Ok(n) if n > 0 => { + eprintln!("[denokv/postgres] requeued {n} dead queue message(s)"); + } + Err(e) => { + eprintln!("[denokv/postgres] queue_cleanup error: {e}"); + } + _ => {} + } + } + }); + } + + Ok(pg) + } + + /// Get a connection from the pool + async fn get_connection(&self) -> PostgresResult { + self.pool.get().await + .map_err(|e| PostgresError::ConnectionFailed(format!("Failed to get connection: {}", e))) + } +} + +#[async_trait] +impl Database for Postgres { + type QMH = PostgresMessageHandle; + + async fn snapshot_read( + &self, + requests: Vec, + options: SnapshotReadOptions, + ) -> Result, JsErrorBox> { + let conn = self.get_connection().await + .map_err(JsErrorBox::from_err)?; + + let mut outputs = Vec::new(); + for request in requests { + let entries = self.backend.read_range(&conn, &request).await + .map_err(JsErrorBox::from_err)?; + outputs.push(ReadRangeOutput { entries }); + } + + Ok(outputs) + } + + async fn atomic_write( + &self, + write: AtomicWrite, + ) -> Result, JsErrorBox> { + // Collect mutated keys before the write consumes them + let mutated_keys: Vec> = write.mutations.iter() + .map(|m| m.key.clone()) + .collect(); + + let mut conn = self.get_connection().await + .map_err(JsErrorBox::from_err)?; + + let result = self.backend.atomic_write(&mut conn, write).await + .map_err(JsErrorBox::from_err)?; + + // Notify watchers of changed keys after a successful commit + if result.is_some() { + for key in &mutated_keys { + self.notifier.notify_key_update(key); + } + } + + Ok(result) + } + + async fn dequeue_next_message(&self) -> Result, JsErrorBox> { + let mut conn = self.get_connection().await + .map_err(JsErrorBox::from_err)?; + + let message_handle = self.backend.dequeue_next_message(&mut conn).await + .map_err(JsErrorBox::from_err)?; + + Ok(message_handle) + } + + fn watch(&self, keys: Vec>) -> Pin, JsErrorBox>> + Send>> { + let backend = self.backend.clone(); + let notifier = self.notifier.clone(); + + let stream = try_stream! { + // Subscribe to key changes + let mut subscriptions = Vec::new(); + for key in &keys { + subscriptions.push(notifier.subscribe(key.clone())); + } + + loop { + // Get current values + let conn = backend.pool.get().await + .map_err(|e| JsErrorBox::generic(format!("Failed to get connection: {}", e)))?; + + let mut outputs = Vec::new(); + for key in &keys { + let request = ReadRange { + start: key.clone(), + end: key.iter().copied().chain(Some(0)).collect(), + limit: std::num::NonZeroU32::new(1).unwrap(), + reverse: false, + }; + + let entries = backend.read_range(&conn, &request).await + .map_err(JsErrorBox::from_err)?; + + let entry = entries.into_iter().next(); + outputs.push(WatchKeyOutput::Changed { entry }); + } + + yield outputs; + + // Wait for changes + for subscription in &mut subscriptions { + subscription.wait_for_change().await; + } + } + }; + + Box::pin(stream) + } + + fn close(&self) { + // PostgreSQL connections are managed by the pool + // No explicit close needed + } +} \ No newline at end of file diff --git a/postgres/message_handle.rs b/postgres/message_handle.rs new file mode 100644 index 0000000..9ee128b --- /dev/null +++ b/postgres/message_handle.rs @@ -0,0 +1,119 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use async_trait::async_trait; +use chrono::Utc; +use deadpool_postgres::Pool; +use deno_error::JsErrorBox; +use denokv_proto::QueueMessageHandle; +use uuid::Uuid; + +use crate::error::{PostgresError, PostgresResult}; + +/// PostgreSQL message handle for queue operations +pub struct PostgresMessageHandle { + pub id: Uuid, + pub payload: Option>, + pub pool: Pool, +} + +impl PostgresMessageHandle { + /// Finish processing a message. + /// + /// On success: remove from queue_running and delete the message. + /// On failure: apply backoff schedule and requeue, or write + /// keys_if_undelivered when retries are exhausted (matching SQLite). + pub async fn finish(&self, success: bool) -> PostgresResult<()> { + let mut conn = self.pool.get().await?; + let tx = conn.transaction().await?; + let id_str = self.id.to_string(); + + if success { + // Remove from running and delete the original message + tx.execute("DELETE FROM queue_running WHERE message_id = $1", &[&id_str]).await?; + tx.execute("DELETE FROM queue_messages WHERE id = $1", &[&id_str]).await?; + } else { + // Fetch the message metadata for requeue decisions + let row = tx.query_opt( + r#"SELECT payload, deadline, keys_if_undelivered, backoff_schedule, retry_count + FROM queue_messages WHERE id = $1"#, + &[&id_str], + ).await?; + + if let Some(row) = row { + let payload: Vec = row.get("payload"); + let keys_json: String = row.get("keys_if_undelivered"); + let backoff_json: Option = row.get("backoff_schedule"); + let retry_count: i32 = row.get("retry_count"); + + let backoff_schedule: Vec = backoff_json + .and_then(|j| serde_json::from_str(&j).ok()) + .unwrap_or_default(); + + // Remove from running table + tx.execute("DELETE FROM queue_running WHERE message_id = $1", &[&id_str]).await?; + + if !backoff_schedule.is_empty() { + // Requeue with next backoff delay + let delay_ms = backoff_schedule[0] as i64; + let new_deadline = crate::time::utc_now().timestamp_millis() + delay_ms; + let remaining_backoff = serde_json::to_string(&backoff_schedule[1..]) + .unwrap_or_else(|_| "[]".to_string()); + + tx.execute( + r#"UPDATE queue_messages + SET deadline = $1, backoff_schedule = $2, retry_count = $3 + WHERE id = $4"#, + &[&new_deadline, &remaining_backoff, &(retry_count + 1), &id_str], + ).await?; + } else { + // No more retries — handle keys_if_undelivered, then delete + let keys_if_undelivered: Vec> = serde_json::from_str(&keys_json) + .unwrap_or_default(); + + if !keys_if_undelivered.is_empty() { + // Write a tombstone value to each key so watchers are notified + for key in &keys_if_undelivered { + let empty_value: Vec = Vec::new(); + tx.execute( + r#"INSERT INTO kv_store (key, value, value_encoding, versionstamp, updated_at) + VALUES ($1, $2, 1, $3, NOW()) + ON CONFLICT (key) DO UPDATE SET + value = EXCLUDED.value, + value_encoding = EXCLUDED.value_encoding, + versionstamp = EXCLUDED.versionstamp, + updated_at = NOW()"#, + &[key, &empty_value, &payload.as_slice()], + ).await?; + } + } + + // Delete the exhausted message + tx.execute("DELETE FROM queue_messages WHERE id = $1", &[&id_str]).await?; + } + } else { + // Message was already removed — just clean up running entry + tx.execute("DELETE FROM queue_running WHERE message_id = $1", &[&id_str]).await?; + } + } + + tx.commit().await?; + Ok(()) + } + + /// Take the payload from the message + pub async fn take_payload(&mut self) -> PostgresResult> { + self.payload.take() + .ok_or_else(|| PostgresError::InvalidData("Payload already taken".to_string())) + } +} + +#[async_trait] +impl QueueMessageHandle for PostgresMessageHandle { + async fn finish(&self, success: bool) -> Result<(), JsErrorBox> { + self.finish(success).await.map_err(JsErrorBox::from_err) + } + + async fn take_payload(&mut self) -> Result, JsErrorBox> { + self.take_payload().await.map_err(JsErrorBox::from_err) + } +} diff --git a/postgres/notifier.rs b/postgres/notifier.rs new file mode 100644 index 0000000..c287e92 --- /dev/null +++ b/postgres/notifier.rs @@ -0,0 +1,82 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use std::collections::HashMap; +use std::sync::{Arc, RwLock}; +use tokio::sync::watch; + +/// PostgreSQL notifier for key change events +#[derive(Clone, Default)] +pub struct PostgresNotifier { + inner: Arc, +} + +impl PostgresNotifier { + pub fn new() -> Self { + Self::default() + } +} + +#[derive(Default)] +struct PostgresNotifierInner { + key_watchers: RwLock, watch::Sender<()>>>, +} + +impl PostgresNotifier { + /// Subscribe to changes for a specific key + pub fn subscribe(&self, key: Vec) -> PostgresKeySubscription { + let mut key_watchers = self.inner.key_watchers.write().unwrap(); + let receiver = match key_watchers.entry(key.clone()) { + std::collections::hash_map::Entry::Occupied(entry) => entry.get().subscribe(), + std::collections::hash_map::Entry::Vacant(entry) => { + let (sender, receiver) = watch::channel(()); + entry.insert(sender); + receiver + } + }; + PostgresKeySubscription { + notifier: Arc::downgrade(&self.inner), + key: Some(key), + receiver, + } + } + + /// Notify that a key has changed + pub fn notify_key_update(&self, key: &[u8]) { + let key_watchers = self.inner.key_watchers.read().unwrap(); + if let Some(sender) = key_watchers.get(key) { + sender.send(()).ok(); // Ignore if no receivers + } + } +} + +pub struct PostgresKeySubscription { + notifier: std::sync::Weak, + key: Option>, + receiver: watch::Receiver<()>, +} + +impl PostgresKeySubscription { + /// Wait for a change to the key + pub async fn wait_for_change(&mut self) { + let _ = self.receiver.changed().await; + } +} + +impl Drop for PostgresKeySubscription { + fn drop(&mut self) { + if let Some(notifier) = self.notifier.upgrade() { + let key = self.key.take().unwrap(); + let mut key_watchers = notifier.key_watchers.write().unwrap(); + match key_watchers.entry(key) { + std::collections::hash_map::Entry::Occupied(entry) => { + // If there is only one subscriber left (this struct), then remove + // the entry from the map. + if entry.get().receiver_count() == 1 { + entry.remove(); + } + } + std::collections::hash_map::Entry::Vacant(_) => unreachable!("the entry should still exist"), + } + } + } +} \ No newline at end of file diff --git a/postgres/src/migration.rs b/postgres/src/migration.rs new file mode 100644 index 0000000..0f6a15f --- /dev/null +++ b/postgres/src/migration.rs @@ -0,0 +1,256 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use std::collections::HashMap; +use std::path::Path; + +use chrono::{DateTime, Utc}; +use rusqlite::{Connection, Row}; +use serde_json::Value; + +use crate::error::{PostgresError, PostgresResult}; +use crate::PostgresConfig; + +/// Migration tool for moving data from SQLite to PostgreSQL +pub struct MigrationTool { + sqlite_path: String, + postgres_config: PostgresConfig, +} + +impl MigrationTool { + /// Create a new migration tool + pub fn new(sqlite_path: String, postgres_config: PostgresConfig) -> Self { + Self { + sqlite_path, + postgres_config, + } + } + + /// Migrate all data from SQLite to PostgreSQL + pub async fn migrate_all(&self) -> PostgresResult<()> { + println!("Starting migration from SQLite to PostgreSQL..."); + + // Open SQLite database + let sqlite_conn = Connection::open(&self.sqlite_path) + .map_err(|e| PostgresError::DatabaseError(format!("Failed to open SQLite: {}", e)))?; + + // Create PostgreSQL instance + let postgres = crate::Postgres::new(self.postgres_config.clone()).await?; + + // Migrate KV data + self.migrate_kv_data(&sqlite_conn, &postgres).await?; + + // Migrate queue data + self.migrate_queue_data(&sqlite_conn, &postgres).await?; + + println!("Migration completed successfully!"); + Ok(()) + } + + /// Migrate KV data from SQLite to PostgreSQL + async fn migrate_kv_data( + &self, + sqlite_conn: &Connection, + postgres: &crate::Postgres, + ) -> PostgresResult<()> { + println!("Migrating KV data..."); + + let mut stmt = sqlite_conn.prepare( + "SELECT key, value, value_encoding, versionstamp, expires_at FROM kv_store" + )?; + + let rows = stmt.query_map([], |row| { + Ok(KvRow { + key: row.get("key")?, + value: row.get("value")?, + value_encoding: row.get("value_encoding")?, + versionstamp: row.get("versionstamp")?, + expires_at: row.get("expires_at")?, + }) + })?; + + let mut batch = Vec::new(); + let mut count = 0; + + for row in rows { + let row = row?; + batch.push(row); + + // Process in batches of 1000 + if batch.len() >= 1000 { + self.process_kv_batch(&postgres, &batch).await?; + count += batch.len(); + println!("Migrated {} KV entries...", count); + batch.clear(); + } + } + + // Process remaining entries + if !batch.is_empty() { + self.process_kv_batch(&postgres, &batch).await?; + count += batch.len(); + } + + println!("Migrated {} KV entries total", count); + Ok(()) + } + + /// Migrate queue data from SQLite to PostgreSQL + async fn migrate_queue_data( + &self, + sqlite_conn: &Connection, + postgres: &crate::Postgres, + ) -> PostgresResult<()> { + println!("Migrating queue data..."); + + let mut stmt = sqlite_conn.prepare( + "SELECT id, payload, deadline, keys_if_undelivered, backoff_schedule FROM queue_messages" + )?; + + let rows = stmt.query_map([], |row| { + Ok(QueueRow { + id: row.get("id")?, + payload: row.get("payload")?, + deadline: row.get("deadline")?, + keys_if_undelivered: row.get("keys_if_undelivered")?, + backoff_schedule: row.get("backoff_schedule")?, + }) + })?; + + let mut count = 0; + for row in rows { + let row = row?; + self.process_queue_row(&postgres, &row).await?; + count += 1; + } + + println!("Migrated {} queue messages", count); + Ok(()) + } + + /// Process a batch of KV rows + async fn process_kv_batch( + &self, + postgres: &crate::Postgres, + batch: &[KvRow], + ) -> PostgresResult<()> { + // Get a connection from the pool + let conn = postgres.pool.get().await?; + + for row in batch { + let value_encoding = match row.value_encoding { + 1 => "V8", + 2 => "LE64", + 3 => "BYTES", + _ => return Err(PostgresError::InvalidData(format!("Unknown encoding: {}", row.value_encoding))), + }; + + conn.execute( + r#" + INSERT INTO kv_store (key, value, value_encoding, versionstamp, expires_at, created_at, updated_at) + VALUES ($1, $2, $3, $4, $5, NOW(), NOW()) + ON CONFLICT (key) DO UPDATE SET + value = EXCLUDED.value, + value_encoding = EXCLUDED.value_encoding, + versionstamp = EXCLUDED.versionstamp, + expires_at = EXCLUDED.expires_at, + updated_at = NOW() + "#, + &[ + &row.key, + &row.value, + &row.value_encoding, + &row.versionstamp, + &row.expires_at, + ], + ).await?; + } + + Ok(()) + } + + /// Process a single queue row + async fn process_queue_row( + &self, + postgres: &crate::Postgres, + row: &QueueRow, + ) -> PostgresResult<()> { + let conn = postgres.pool.get().await?; + + // Parse JSON fields + let keys_json: Value = serde_json::from_str(&row.keys_if_undelivered)?; + let backoff_json: Option = if let Some(backoff) = &row.backoff_schedule { + Some(serde_json::from_str(backoff)?) + } else { + None + }; + + conn.execute( + r#" + INSERT INTO queue_messages (id, payload, deadline, keys_if_undelivered, backoff_schedule, created_at) + VALUES ($1, $2, $3, $4, $5, NOW()) + ON CONFLICT (id) DO UPDATE SET + payload = EXCLUDED.payload, + deadline = EXCLUDED.deadline, + keys_if_undelivered = EXCLUDED.keys_if_undelivered, + backoff_schedule = EXCLUDED.backoff_schedule + "#, + &[ + &row.id, + &row.payload, + &row.deadline, + &keys_json, + &backoff_json, + ], + ).await?; + + Ok(()) + } +} + +#[derive(Debug)] +struct KvRow { + key: Vec, + value: Vec, + value_encoding: i32, + versionstamp: Vec, + expires_at: Option>, +} + +#[derive(Debug)] +struct QueueRow { + id: String, + payload: Vec, + deadline: DateTime, + keys_if_undelivered: String, + backoff_schedule: Option, +} + +/// CLI tool for migration +pub async fn run_migration_cli() -> PostgresResult<()> { + use clap::Parser; + + #[derive(Parser)] + struct Args { + /// Path to SQLite database + #[clap(long)] + sqlite_path: String, + + /// PostgreSQL connection URL + #[clap(long)] + postgres_url: String, + + /// Maximum number of connections + #[clap(long, default_value = "10")] + max_connections: usize, + } + + let args = Args::parse(); + + let postgres_config = PostgresConfig::new(args.postgres_url) + .with_max_connections(args.max_connections); + + let migration_tool = MigrationTool::new(args.sqlite_path, postgres_config); + migration_tool.migrate_all().await?; + + Ok(()) +} \ No newline at end of file diff --git a/postgres/tests/integration.rs b/postgres/tests/integration.rs new file mode 100644 index 0000000..f0775c6 --- /dev/null +++ b/postgres/tests/integration.rs @@ -0,0 +1,164 @@ +// Copyright 2023 rawkakani. All rights reserved. MIT license. + +use denokv_postgres::{Postgres, PostgresConfig}; +use denokv_proto::{ + AtomicWrite, Check, Database, KvValue, Mutation, MutationKind, ReadRange, SnapshotReadOptions, +}; +use std::num::NonZeroU32; + +#[tokio::test] +async fn test_postgres_basic_operations() { + // Skip test if no PostgreSQL is available + if std::env::var("POSTGRES_URL").is_err() { + println!("Skipping PostgreSQL test - POSTGRES_URL not set"); + return; + } + + let postgres_url = std::env::var("POSTGRES_URL").unwrap(); + let config = PostgresConfig::new(postgres_url); + let postgres = Postgres::new(config).await.expect("Failed to create PostgreSQL instance"); + + // Test basic set operation + let key = b"test_key".to_vec(); + let value = KvValue::Bytes(b"test_value".to_vec()); + + let atomic_write = AtomicWrite { + checks: vec![], + mutations: vec![Mutation { + key: key.clone(), + kind: MutationKind::Set(value), + expire_at: None, + }], + enqueues: vec![], + }; + + let result = postgres.atomic_write(atomic_write).await.expect("Atomic write failed"); + assert!(result.is_some()); + + // Test read operation + let read_range = ReadRange { + start: key.clone(), + end: key.iter().copied().chain(Some(0)).collect(), + limit: NonZeroU32::new(1).unwrap(), + reverse: false, + }; + + let options = SnapshotReadOptions { + consistency: denokv_proto::Consistency::Strong, + }; + + let results = postgres + .snapshot_read(vec![read_range], options.clone()) + .await + .expect("Snapshot read failed"); + + assert_eq!(results.len(), 1); + assert_eq!(results[0].entries.len(), 1); + assert_eq!(results[0].entries[0].key, key); + match &results[0].entries[0].value { + KvValue::Bytes(bytes) => assert_eq!(bytes, b"test_value"), + _ => panic!("Expected Bytes value"), + } + + // Test delete operation + let delete_write = AtomicWrite { + checks: vec![], + mutations: vec![Mutation { + key: key.clone(), + kind: MutationKind::Delete, + expire_at: None, + }], + enqueues: vec![], + }; + + let result = postgres.atomic_write(delete_write).await.expect("Delete failed"); + assert!(result.is_some()); + + // Verify deletion + let read_range = ReadRange { + start: key.clone(), + end: key.iter().copied().chain(Some(0)).collect(), + limit: NonZeroU32::new(1).unwrap(), + reverse: false, + }; + + let results = postgres + .snapshot_read(vec![read_range], options.clone()) + .await + .expect("Snapshot read failed"); + + assert_eq!(results.len(), 1); + assert_eq!(results[0].entries.len(), 0); +} + +#[tokio::test] +async fn test_postgres_sum_operations() { + // Skip test if no PostgreSQL is available + if std::env::var("POSTGRES_URL").is_err() { + println!("Skipping PostgreSQL test - POSTGRES_URL not set"); + return; + } + + let postgres_url = std::env::var("POSTGRES_URL").unwrap(); + let config = PostgresConfig::new(postgres_url); + let postgres = Postgres::new(config).await.expect("Failed to create PostgreSQL instance"); + + let key = b"counter".to_vec(); + let initial_value = KvValue::U64(10); + + // Set initial value + let set_write = AtomicWrite { + checks: vec![], + mutations: vec![Mutation { + key: key.clone(), + kind: MutationKind::Set(initial_value), + expire_at: None, + }], + enqueues: vec![], + }; + + postgres.atomic_write(set_write).await.expect("Set failed"); + + // Test sum operation + let sum_value = KvValue::U64(5); + let sum_write = AtomicWrite { + checks: vec![], + mutations: vec![Mutation { + key: key.clone(), + kind: MutationKind::Sum { + value: sum_value, + min_v8: vec![], + max_v8: vec![], + clamp: false, + }, + expire_at: None, + }], + enqueues: vec![], + }; + + postgres.atomic_write(sum_write).await.expect("Sum failed"); + + // Verify result + let read_range = ReadRange { + start: key.clone(), + end: key.iter().copied().chain(Some(0)).collect(), + limit: NonZeroU32::new(1).unwrap(), + reverse: false, + }; + + let options = SnapshotReadOptions { + consistency: denokv_proto::Consistency::Strong, + }; + + let results = postgres + .snapshot_read(vec![read_range], options.clone()) + .await + .expect("Snapshot read failed"); + + assert_eq!(results.len(), 1); + assert_eq!(results[0].entries.len(), 1); + match &results[0].entries[0].value { + KvValue::U64(value) => assert_eq!(*value, 15), + _ => panic!("Expected U64 value"), + } +} \ No newline at end of file diff --git a/postgres/time.rs b/postgres/time.rs new file mode 100644 index 0000000..1a56e80 --- /dev/null +++ b/postgres/time.rs @@ -0,0 +1,19 @@ +// Copyright 2023 the Deno authors. All rights reserved. MIT license. + +/// Identical to chrono::Utc::now() but without the system "clock" +/// feature flag. +/// +/// The "clock" feature flag pulls in the "iana-time-zone" crate +/// which links to macOS's "CoreFoundation" framework which increases +/// startup time for the CLI. +pub fn utc_now() -> chrono::DateTime { + let now = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("system time before Unix epoch"); + let naive = chrono::NaiveDateTime::from_timestamp_opt( + now.as_secs() as i64, + now.subsec_nanos(), + ) + .unwrap(); + chrono::DateTime::from_naive_utc_and_offset(naive, chrono::Utc) +} diff --git a/proto/interface.rs b/proto/interface.rs index 1f35f0f..1613cfe 100644 --- a/proto/interface.rs +++ b/proto/interface.rs @@ -18,9 +18,9 @@ use uuid::Uuid; use crate::codec::canonicalize_f64; pub type WatchStream = - Pin, JsErrorBox>>>>; + Pin, JsErrorBox>> + Send>>; -#[async_trait(?Send)] +#[async_trait] pub trait Database: Clone + Sized { type QMH: QueueMessageHandle + 'static; @@ -43,13 +43,13 @@ pub trait Database: Clone + Sized { fn close(&self); } -#[async_trait(?Send)] -pub trait QueueMessageHandle { +#[async_trait] +pub trait QueueMessageHandle: Send + Sync { async fn take_payload(&mut self) -> Result, JsErrorBox>; async fn finish(&self, success: bool) -> Result<(), JsErrorBox>; } -#[async_trait(?Send)] +#[async_trait] impl QueueMessageHandle for Box { async fn take_payload(&mut self) -> Result, JsErrorBox> { (**self).take_payload().await @@ -205,7 +205,7 @@ pub struct KvEntry { /// /// - **Bytes**: an arbitrary byte array. /// - **U64**: a 64-bit unsigned integer. -#[derive(Debug)] +#[derive(Clone, Debug)] pub enum KvValue { V8(Vec), Bytes(Vec), @@ -230,6 +230,7 @@ pub enum KvValue { /// The mutations are performed in the order that they are specified in the /// `mutations` field. The order of checks is not specified, and is also not /// important because this ordering is un-observable. +#[derive(Clone)] pub struct AtomicWrite { pub checks: Vec, pub mutations: Vec, @@ -239,6 +240,7 @@ pub struct AtomicWrite { /// A request to perform a check on a key in the database. The check is not /// performed on the value of the key, but rather on the versionstamp of the /// key. +#[derive(Clone)] pub struct Check { pub key: Vec, pub versionstamp: Option, @@ -249,6 +251,7 @@ pub struct Check { /// /// The type of mutation is specified by the `kind` field. The action performed /// by each mutation kind is specified in the docs for [MutationKind]. +#[derive(Clone)] pub struct Mutation { pub key: Vec, pub kind: MutationKind, @@ -268,6 +271,7 @@ pub struct Mutation { /// /// If all retry attempts failed, the message is written to the KV under all /// keys specified in `keys_if_undelivered`. +#[derive(Clone)] pub struct Enqueue { pub payload: Vec, pub deadline: DateTime, @@ -316,7 +320,7 @@ pub struct Enqueue { /// the database must match the type of the value specified in the mutation. If /// the key does not exist in the database, then the value specified in the /// mutation is used as the new value of the key. -#[derive(Debug)] +#[derive(Clone, Debug)] pub enum MutationKind { Set(KvValue), Delete, diff --git a/rawkakani@102.37.137.29 b/rawkakani@102.37.137.29 new file mode 100644 index 0000000..927a175 --- /dev/null +++ b/rawkakani@102.37.137.29 @@ -0,0 +1,69 @@ +#!/bin/bash +# Script to expand 512GB disk on Azure VM +# Run this on the database server: 102.37.137.29 + +set -e # Exit on error + +echo "=== Expanding 512GB Disk ===" +echo "" + +# Step 1: Fix GPT table +echo "Step 1: Fixing GPT partition table..." +echo "Note: This will prompt you to write changes. Type 'w' then 'y' when prompted." +echo "Press Enter to continue..." +read + +sudo gdisk /dev/nvme0n1 < /dev/null; then + sudo dnf install -y cloud-utils-growpart || sudo yum install -y cloud-utils-growpart + echo "growpart installed" +else + echo "growpart already installed" +fi +echo "" + +# Step 3: Extend partition 4 +echo "Step 3: Extending partition 4 to use full disk..." +sudo growpart /dev/nvme0n1 4 +echo "Partition extended" +echo "" + +# Step 4: Resize physical volume +echo "Step 4: Resizing LVM physical volume..." +sudo pvresize /dev/nvme0n1p4 +echo "Physical volume resized" +echo "" + +# Step 5: Extend logical volume +echo "Step 5: Extending logical volume to use all available space..." +sudo lvextend -l +100%FREE /dev/mapper/rocky-lvroot +echo "Logical volume extended" +echo "" + +# Step 6: Resize filesystem +echo "Step 6: Resizing filesystem..." +sudo xfs_growfs / +echo "Filesystem resized" +echo "" + +# Step 7: Verify +echo "Step 7: Verifying disk space..." +echo "" +df -h +echo "" +echo "=== Disk expansion complete! ===" +echo "" +echo "Current disk status:" +sudo pvs +sudo vgs +sudo lvs diff --git a/remote/lib.rs b/remote/lib.rs index 856c3a4..ca0533a 100644 --- a/remote/lib.rs +++ b/remote/lib.rs @@ -140,7 +140,7 @@ enum MetadataState { Error(Arc), } -pub trait RemotePermissions: Clone + 'static { +pub trait RemotePermissions: Clone + Send + Sync + 'static { fn check_net_url(&self, url: &Url) -> Result<(), JsErrorBox>; } @@ -585,8 +585,6 @@ pub enum SnapshotReadError { #[derive(Debug, Error, JsError)] pub enum AtomicWriteError { #[class(generic)] - #[error("Enqueue operations are not supported in KV Connect")] - EnqueueOperationsUnsupported, #[class(inherit)] #[error(transparent)] CallData(#[from] CallDataError), @@ -620,7 +618,7 @@ pub enum WatchError { TryFromSlice(std::array::TryFromSliceError), } -#[async_trait(?Send)] +#[async_trait] impl Database for Remote { type QMH = DummyQueueMessageHandle; @@ -706,11 +704,7 @@ impl Database for Remote { &self, write: AtomicWrite, ) -> Result, JsErrorBox> { - if !write.enqueues.is_empty() { - return Err(JsErrorBox::from_err( - AtomicWriteError::EnqueueOperationsUnsupported, - )); - } + // Enqueue operations are now supported let mut checks = Vec::new(); for check in write.checks { @@ -801,12 +795,21 @@ impl Database for Remote { } } - assert!(write.enqueues.is_empty()); + // Convert enqueues to protobuf format + let mut enqueues = Vec::new(); + for enqueue in write.enqueues { + enqueues.push(pb::Enqueue { + payload: enqueue.payload, + deadline_ms: enqueue.deadline.timestamp_millis(), + keys_if_undelivered: enqueue.keys_if_undelivered, + backoff_schedule: enqueue.backoff_schedule.unwrap_or_default(), + }); + } let req = pb::AtomicWrite { checks, mutations, - enqueues: Vec::new(), + enqueues, }; let (res, _): (pb::AtomicWriteOutput, _) = self @@ -840,7 +843,7 @@ impl Database for Remote { fn watch( &self, keys: Vec>, - ) -> Pin, JsErrorBox>>>> { + ) -> Pin, JsErrorBox>> + Send>> { let this = self.clone(); let stream = try_stream! { let mut attempt = 0; @@ -923,7 +926,7 @@ impl Database for Remote { pub struct DummyQueueMessageHandle {} -#[async_trait(?Send)] +#[async_trait] impl QueueMessageHandle for DummyQueueMessageHandle { async fn take_payload(&mut self) -> Result, JsErrorBox> { unimplemented!() diff --git a/setup-complete.sh b/setup-complete.sh new file mode 100755 index 0000000..b101b7f --- /dev/null +++ b/setup-complete.sh @@ -0,0 +1,437 @@ +#!/bin/bash + +# Complete DenoKV Setup Script for Rocky Linux +# This script does everything: PostgreSQL setup, environment setup, and starts DenoKV in background +# Author: Assistant + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +PURPLE='\033[0;35m' +NC='\033[0m' + +# Configuration +DENOKV_USER="denokv" +DENOKV_PASSWORD="denokv_password" +DENOKV_DATABASE="denokv" +POSTGRES_DATA_DIR="/var/lib/pgsql/data" +DENOKV_PORT="4512" +DENOKV_ADDR="0.0.0.0:${DENOKV_PORT}" +DENOKV_LOG_FILE="denokv.log" +DENOKV_PID_FILE="denokv.pid" + +print_status() { echo -e "${BLUE}[INFO]${NC} $1"; } +print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; } +print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; } +print_error() { echo -e "${RED}[ERROR]${NC} $1"; } +print_step() { echo -e "${PURPLE}[STEP]${NC} $1"; } + +# Determine if we need sudo based on current user +if [[ $EUID -eq 0 ]]; then + SUDO_CMD="" # No sudo needed when running as root +else + if ! command -v sudo >/dev/null 2>&1; then + print_error "sudo is required but not installed. Please install sudo first." + exit 1 + fi + SUDO_CMD="sudo" # Use sudo when running as regular user +fi + +echo "🚀 Complete DenoKV Setup for Rocky Linux" +echo "=========================================" +echo "" + +# Step 1: Clean up unnecessary scripts +print_step "Step 1: Cleaning up unnecessary scripts..." +rm -f fresh-postgres-setup.sh setup-rocky-linux.sh start-denokv-background.sh start-denokv-simple.sh test_*.ts 2>/dev/null || true +print_success "Unnecessary scripts removed!" + +# Step 2: Stop and remove existing PostgreSQL +print_step "Step 2: Setting up PostgreSQL..." + +# Stop PostgreSQL services +$SUDO_CMD systemctl stop postgresql 2>/dev/null || true +$SUDO_CMD systemctl disable postgresql 2>/dev/null || true +$SUDO_CMD pkill -f postgres 2>/dev/null || true +sleep 2 + +# Remove PostgreSQL packages and data +$SUDO_CMD dnf remove -y postgresql* 2>/dev/null || true +$SUDO_CMD rm -rf /var/lib/pgsql /var/lib/postgresql /var/lib/postgres 2>/dev/null || true +$SUDO_CMD rm -rf /etc/postgresql /etc/postgresql-common /usr/lib/postgresql 2>/dev/null || true +$SUDO_CMD userdel postgres 2>/dev/null || true +$SUDO_CMD groupdel postgres 2>/dev/null || true +$SUDO_CMD rm -rf /tmp/.s.PGSQL.* /var/run/postgresql 2>/dev/null || true + +print_success "PostgreSQL completely removed!" + +# Step 3: Install latest PostgreSQL +print_status "Installing latest PostgreSQL packages..." +$SUDO_CMD dnf update -y + +# Install PostgreSQL (latest available version) +print_status "Installing PostgreSQL (latest available)..." +$SUDO_CMD dnf install -y postgresql postgresql-server postgresql-contrib postgresql-devel + +# Verify PostgreSQL version +POSTGRES_VERSION=$(postgres --version 2>/dev/null | grep -o '[0-9]\+\.[0-9]\+' | head -1 || echo "Unknown") +print_success "PostgreSQL $POSTGRES_VERSION installed" + +# Step 4: Initialize PostgreSQL +print_status "Initializing PostgreSQL database..." +$SUDO_CMD mkdir -p "$POSTGRES_DATA_DIR" +$SUDO_CMD mkdir -p /var/run/postgresql +$SUDO_CMD chown -R postgres:postgres /var/lib/pgsql +$SUDO_CMD chown -R postgres:postgres /var/run/postgresql +$SUDO_CMD chmod 700 "$POSTGRES_DATA_DIR" +$SUDO_CMD chmod 755 /var/run/postgresql + +# Initialize PostgreSQL +print_status "Initializing PostgreSQL database..." +$SUDO_CMD postgresql-setup --initdb + +# Step 5: Configure PostgreSQL +print_status "Configuring PostgreSQL..." +$SUDO_CMD tee "$POSTGRES_DATA_DIR/pg_hba.conf" > /dev/null << 'EOF' +# PostgreSQL Client Authentication Configuration File +local all all trust +host all all 127.0.0.1/32 trust +host all all 0.0.0.0/0 trust +host all all ::1/128 trust +host all all ::/0 trust +local replication all trust +host replication all 127.0.0.1/32 trust +host replication all ::1/128 trust +EOF + +$SUDO_CMD tee "$POSTGRES_DATA_DIR/postgresql.conf" > /dev/null << 'EOF' +# PostgreSQL configuration for DenoKV +listen_addresses = 'localhost' +port = 5432 +max_connections = 100 +shared_buffers = 128MB +effective_cache_size = 512MB +log_destination = 'stderr' +logging_collector = on +log_directory = 'log' +log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' +log_rotation_age = 1d +log_rotation_size = 10MB +log_min_duration_statement = 1000 +lc_messages = 'en_US.UTF-8' +lc_monetary = 'en_US.UTF-8' +lc_numeric = 'en_US.UTF-8' +lc_time = 'en_US.UTF-8' +default_text_search_config = 'pg_catalog.english' +EOF + +# Step 6: Start PostgreSQL +print_status "Starting PostgreSQL service..." +$SUDO_CMD systemctl enable postgresql +$SUDO_CMD systemctl start postgresql + +# Wait for PostgreSQL to be ready +print_status "Waiting for PostgreSQL to be ready..." +for i in {1..30}; do + if $SUDO_CMD -u postgres psql -c "SELECT 1;" >/dev/null 2>&1; then + print_success "PostgreSQL is ready!" + break + fi + if [ $i -eq 30 ]; then + print_error "PostgreSQL failed to start after 30 seconds" + exit 1 + fi + sleep 1 +done + +# Step 7: Create DenoKV database and user +print_status "Creating DenoKV database and user..." +$SUDO_CMD -u postgres psql -c "ALTER USER postgres PASSWORD 'postgres_password';" 2>/dev/null || true +$SUDO_CMD -u postgres psql -c "CREATE USER denokv WITH PASSWORD 'denokv_password';" 2>/dev/null || true +$SUDO_CMD -u postgres psql -c "CREATE DATABASE denokv OWNER denokv;" 2>/dev/null || true +$SUDO_CMD -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE denokv TO denokv;" +$SUDO_CMD -u postgres psql -c "GRANT ALL PRIVILEGES ON SCHEMA public TO denokv;" 2>/dev/null || true + +print_success "PostgreSQL setup completed!" + +# Step 8: Set up environment variables +print_step "Step 3: Setting up environment variables..." + +# Create environment file +cat > .env << EOF +# DenoKV PostgreSQL Configuration +POSTGRES_HOST=localhost +POSTGRES_PORT=5432 +POSTGRES_DB=denokv +POSTGRES_USER=denokv +POSTGRES_PASSWORD=denokv_password + +# DenoKV Server Configuration +DENOKV_PORT=4512 +DENOKV_ACCESS_TOKEN=$(openssl rand -hex 16) + +# Development Configuration +RUST_LOG=info +DENO_ENV=production +EOF + +# Add environment variables to shell profile +if [ -f ~/.bashrc ]; then + echo "" >> ~/.bashrc + echo "# DenoKV Environment Variables" >> ~/.bashrc + echo "export POSTGRES_HOST=localhost" >> ~/.bashrc + echo "export POSTGRES_PORT=5432" >> ~/.bashrc + echo "export POSTGRES_DB=denokv" >> ~/.bashrc + echo "export POSTGRES_USER=denokv" >> ~/.bashrc + echo "export POSTGRES_PASSWORD=denokv_password" >> ~/.bashrc + echo "export DENOKV_PORT=4512" >> ~/.bashrc + echo "export DENOKV_ACCESS_TOKEN=\$DENOKV_ACCESS_TOKEN" >> ~/.bashrc + echo "export RUST_LOG=info" >> ~/.bashrc + echo "export DENO_ENV=production" >> ~/.bashrc +fi + +# Source environment variables for current session +export POSTGRES_HOST=localhost +export POSTGRES_PORT=5432 +export POSTGRES_DB=denokv +export POSTGRES_USER=denokv +export POSTGRES_PASSWORD=denokv_password +export DENOKV_PORT=4512 +export DENOKV_ACCESS_TOKEN=$DENOKV_ACCESS_TOKEN +export RUST_LOG=info +export DENO_ENV=production + +print_success "Environment variables configured!" +echo "" +echo "🔐 Generated Access Token: $DENOKV_ACCESS_TOKEN" +echo "📝 This token has been saved to .env file and systemd service" + +# Step 9: Build DenoKV and setup systemd service +print_step "Step 4: Building DenoKV and setting up systemd service..." + +# Stop any existing DenoKV processes +print_status "Stopping any existing DenoKV processes..." +$SUDO_CMD systemctl stop denokv.service 2>/dev/null || true +if [ -f "$DENOKV_PID_FILE" ]; then + PID=$(cat "$DENOKV_PID_FILE") + kill "$PID" 2>/dev/null || true + rm -f "$DENOKV_PID_FILE" +fi + +# Build DenoKV +print_status "Building DenoKV..." +if [ ! -f "target/release/denokv" ]; then + cargo build --release + if [ $? -ne 0 ]; then + print_error "Failed to build DenoKV" + exit 1 + fi +fi + +print_success "DenoKV binary ready" + +# Create denokv user if it doesn't exist +if ! id "denokv" &>/dev/null; then + print_status "Creating denokv user..." + $SUDO_CMD useradd -r -s /bin/false -d /var/lib/denokv denokv + $SUDO_CMD mkdir -p /var/lib/denokv + $SUDO_CMD chown denokv:denokv /var/lib/denokv + $SUDO_CMD chmod 755 /var/lib/denokv +fi + +# Install DenoKV binary to system location +print_status "Installing DenoKV binary..." +$SUDO_CMD cp target/release/denokv /usr/local/bin/denokv +$SUDO_CMD chmod +x /usr/local/bin/denokv +$SUDO_CMD chown root:root /usr/local/bin/denokv + +# Create systemd service file +print_status "Creating systemd service..." +$SUDO_CMD tee /etc/systemd/system/denokv.service > /dev/null << EOF +[Unit] +Description=DenoKV Server +After=network.target postgresql.service +Requires=postgresql.service + +[Service] +Type=simple +User=denokv +Group=denokv +WorkingDirectory=/var/lib/denokv +ExecStart=/usr/local/bin/denokv serve --access-token ${DENOKV_ACCESS_TOKEN} --addr ${DENOKV_ADDR} +Restart=always +RestartSec=5 +StandardOutput=journal +StandardError=journal +SyslogIdentifier=denokv + +# Environment variables +Environment=RUST_LOG=info +Environment=DENO_ENV=production +Environment=DENO_KV_DATABASE_TYPE=postgres +Environment=DENO_KV_POSTGRES_URL=postgresql://denokv:denokv_password@localhost:5432/denokv +Environment=DENO_KV_ACCESS_TOKEN=${DENOKV_ACCESS_TOKEN} +Environment=POSTGRES_HOST=localhost +Environment=POSTGRES_PORT=5432 +Environment=POSTGRES_DB=denokv +Environment=POSTGRES_USER=denokv +Environment=POSTGRES_PASSWORD=denokv_password + +# Security settings +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/var/lib/denokv + +[Install] +WantedBy=multi-user.target +EOF + +# Ensure working directory exists and has proper permissions +print_status "Setting up DenoKV working directory..." +$SUDO_CMD mkdir -p /var/lib/denokv +$SUDO_CMD chown denokv:denokv /var/lib/denokv +$SUDO_CMD chmod 755 /var/lib/denokv + +# Reload systemd and enable service +print_status "Enabling DenoKV systemd service..." +$SUDO_CMD systemctl daemon-reload +$SUDO_CMD systemctl enable denokv.service + +# Start the service +print_status "Starting DenoKV service..." +$SUDO_CMD systemctl start denokv.service + +# Wait for service to start +print_status "Waiting for DenoKV to start..." +sleep 3 + +# Check if service is running +if $SUDO_CMD systemctl is-active --quiet denokv.service; then + print_success "DenoKV systemd service started successfully!" + + # Get the PID + DENOKV_PID=$($SUDO_CMD systemctl show -p MainPID --value denokv.service) + echo "$DENOKV_PID" > "$DENOKV_PID_FILE" + + # Test the connection + print_status "Testing DenoKV connection..." + sleep 2 + + if curl -s http://localhost:$DENOKV_PORT/ > /dev/null; then + print_success "DenoKV is responding on port $DENOKV_PORT" + else + print_warning "DenoKV may not be fully ready yet, but service is running" + fi + +else + print_error "Failed to start DenoKV systemd service" + print_status "Checking service status..." + $SUDO_CMD systemctl status denokv.service --no-pager + exit 1 +fi + +# Final summary +echo "" +print_success "🎉 Complete DenoKV setup finished!" +echo "" +echo "📋 Setup Summary:" +echo "==================" +echo "✅ PostgreSQL: Fresh installation with denokv database" +echo "✅ Environment: Variables configured and exported" +echo "✅ DenoKV: Built and running as systemd service" +echo "✅ Systemd: Service created and enabled for auto-start" +echo "✅ Cleanup: Unnecessary scripts removed" +echo "" +echo "🔧 Service Information:" +echo "=======================" +echo "Service: denokv.service" +echo "Status: $(systemctl is-active denokv.service)" +echo "PID: $DENOKV_PID" +echo "Port: $DENOKV_PORT" +echo "Address: $DENOKV_ADDR" +echo "User: denokv" +echo "Binary: /usr/local/bin/denokv" +echo "" +echo "🌍 Environment Variables:" +echo "==========================" +echo "POSTGRES_HOST=localhost" +echo "POSTGRES_PORT=5432" +echo "POSTGRES_DB=denokv" +echo "POSTGRES_USER=denokv" +echo "POSTGRES_PASSWORD=denokv_password" +echo "DENOKV_PORT=4512" +echo "DENOKV_ACCESS_TOKEN=$DENOKV_ACCESS_TOKEN" +echo "" +echo "🔧 Systemd Management Commands:" +echo "===============================" +echo "Start: sudo systemctl start denokv.service" +echo "Stop: sudo systemctl stop denokv.service" +echo "" +echo "🔐 Access Token: $DENOKV_ACCESS_TOKEN" +echo "📝 Use this token to connect to DenoKV from your applications" +echo "" +echo "🧪 Test Script:" +echo "===============" +echo "Create a test script with:" +echo "cat > test_denokv.ts << 'EOF'" +echo "const KV_URL = \"http://localhost:4512\";" +echo "const ACCESS_TOKEN = \"$DENOKV_ACCESS_TOKEN\";" +echo "" +echo "async function testDenoKV() {" +echo " try {" +echo " console.log('🔗 Testing DenoKV connection...');" +echo " " +echo " // Open KV connection using native Deno KV API" +echo " const kv = await Deno.openKv(KV_URL, { accessToken: ACCESS_TOKEN });" +echo " console.log('✅ KV connection opened successfully');" +echo " " +echo " // Test KV operations" +echo " const testKey = ['test', 'key'];" +echo " const testValue = 'Hello DenoKV!';" +echo " " +echo " // Set a value" +echo " await kv.set(testKey, testValue);" +echo " console.log('✅ Set operation successful - Key:', testKey, 'Value:', testValue);" +echo " " +echo " // Get the value" +echo " const result = await kv.get(testKey);" +echo " console.log('✅ Get operation successful - Retrieved:', result.value);" +echo " " +echo " // Clean up" +echo " await kv.delete(testKey);" +echo " console.log('✅ Delete operation successful - Removed key:', testKey);" +echo " " +echo " // Close connection" +echo " kv.close();" +echo " console.log('🎉 All DenoKV tests passed!');" +echo " } catch (error) {" +echo " console.error('❌ Test failed:', error);" +echo " }" +echo "}" +echo "" +echo "testDenoKV();" +echo "EOF" +echo "" +echo "Run test with: deno run --allow-net --unstable-kv test_denokv.ts" +echo "Restart: sudo systemctl restart denokv.service" +echo "Status: sudo systemctl status denokv.service" +echo "Logs: sudo journalctl -u denokv.service -f" +echo "Enable: sudo systemctl enable denokv.service" +echo "Disable: sudo systemctl disable denokv.service" +echo "" +echo "🌐 Test Connection:" +echo "===================" +echo "curl http://localhost:$DENOKV_PORT/" +echo "curl http://102.37.137.29:$DENOKV_PORT/" +echo "" +echo "🚀 DenoKV is ready for production use!" +echo " - Auto-starts on boot" +echo " - Auto-restarts on crash" +echo " - Runs as secure system user" +echo " - Integrated with systemd logging" \ No newline at end of file diff --git a/sqlite/lib.rs b/sqlite/lib.rs index f94d2c5..7ff6005 100644 --- a/sqlite/lib.rs +++ b/sqlite/lib.rs @@ -622,7 +622,7 @@ impl Sqlite { } } -#[async_trait::async_trait(?Send)] +#[async_trait::async_trait] impl Database for Sqlite { type QMH = SqliteMessageHandle; @@ -659,7 +659,7 @@ impl Database for Sqlite { fn watch( &self, keys: Vec>, - ) -> Pin, JsErrorBox>>>> { + ) -> Pin, JsErrorBox>> + Send>> { Sqlite::watch(self, keys) } @@ -696,7 +696,7 @@ impl SqliteMessageHandle { } } -#[async_trait::async_trait(?Send)] +#[async_trait::async_trait] impl QueueMessageHandle for SqliteMessageHandle { async fn finish(&self, success: bool) -> Result<(), JsErrorBox> { SqliteMessageHandle::finish(self, success) diff --git a/test-postgres-connection.sh b/test-postgres-connection.sh new file mode 100755 index 0000000..3a7f123 --- /dev/null +++ b/test-postgres-connection.sh @@ -0,0 +1,148 @@ +#!/bin/bash + +# PostgreSQL Connection Test Script for DenoKV + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' + +print_status() { echo -e "${BLUE}[INFO]${NC} $1"; } +print_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; } +print_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; } +print_error() { echo -e "${RED}[ERROR]${NC} $1"; } + +echo "🔍 PostgreSQL Connection Test for DenoKV" +echo "==========================================" +echo "" + +# Default environment variables +DEFAULT_POSTGRES_URL="postgresql://denokv:denokv_password@localhost:5432/denokv" +DEFAULT_DENO_KV_POSTGRES_URL="postgresql://denokv:denokv_password@localhost:5432/denokv" + +# Check if .env file exists and source it +if [ -f ".env" ]; then + print_status "Loading environment variables from .env file..." + source .env +fi + +# Use environment variables or defaults +POSTGRES_URL=${POSTGRES_URL:-$DEFAULT_POSTGRES_URL} +DENO_KV_POSTGRES_URL=${DENO_KV_POSTGRES_URL:-$DEFAULT_DENO_KV_POSTGRES_URL} + +echo "📋 Environment Variables:" +echo " POSTGRES_URL: $POSTGRES_URL" +echo " DENO_KV_POSTGRES_URL: $DENO_KV_POSTGRES_URL" +echo "" + +# Test 1: Check if PostgreSQL service is running +print_status "Test 1: Checking PostgreSQL service status..." +if systemctl is-active --quiet postgresql; then + print_success "PostgreSQL service is running" +else + print_error "PostgreSQL service is not running" + print_status "Start it with: sudo systemctl start postgresql" + exit 1 +fi + +# Test 2: Check if PostgreSQL is accepting connections +print_status "Test 2: Checking PostgreSQL connection..." +if sudo -u postgres pg_isready; then + print_success "PostgreSQL is accepting connections" +else + print_error "PostgreSQL is not accepting connections" + exit 1 +fi + +# Test 3: Test connection with psql +print_status "Test 3: Testing database connection with psql..." + +# Extract connection details from URL +# Format: postgresql://user:password@host:port/database +if [[ $DENO_KV_POSTGRES_URL =~ postgresql://([^:]+):([^@]+)@([^:]+):([^/]+)/(.+) ]]; then + DB_USER="${BASH_REMATCH[1]}" + DB_PASSWORD="${BASH_REMATCH[2]}" + DB_HOST="${BASH_REMATCH[3]}" + DB_PORT="${BASH_REMATCH[4]}" + DB_NAME="${BASH_REMATCH[5]}" + + echo " User: $DB_USER" + echo " Host: $DB_HOST" + echo " Port: $DB_PORT" + echo " Database: $DB_NAME" + echo "" + + # Test connection + if PGPASSWORD="$DB_PASSWORD" psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c "SELECT 1 as test_connection;" >/dev/null 2>&1; then + print_success "Database connection successful!" + else + print_error "Database connection failed" + print_status "Trying to diagnose the issue..." + + # Check if user exists + if sudo -u postgres psql -c "SELECT 1 FROM pg_user WHERE usename='$DB_USER';" | grep -q "1 row"; then + print_status "User '$DB_USER' exists" + else + print_error "User '$DB_USER' does not exist" + print_status "Create user with: sudo -u postgres psql -c \"CREATE USER $DB_USER WITH PASSWORD '$DB_PASSWORD';\"" + fi + + # Check if database exists + if sudo -u postgres psql -c "SELECT 1 FROM pg_database WHERE datname='$DB_NAME';" | grep -q "1 row"; then + print_status "Database '$DB_NAME' exists" + else + print_error "Database '$DB_NAME' does not exist" + print_status "Create database with: sudo -u postgres psql -c \"CREATE DATABASE $DB_NAME;\"" + fi + + exit 1 + fi +else + print_error "Could not parse PostgreSQL URL: $DENO_KV_POSTGRES_URL" + exit 1 +fi + +# Test 4: Test DenoKV specific operations +print_status "Test 4: Testing DenoKV specific database operations..." + +# Test creating a simple table (if it doesn't exist) +PGPASSWORD="$DB_PASSWORD" psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d "$DB_NAME" -c " +CREATE TABLE IF NOT EXISTS test_table ( + id SERIAL PRIMARY KEY, + test_data TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +);" >/dev/null 2>&1 + +if [ $? -eq 0 ]; then + print_success "DenoKV database operations test successful" +else + print_warning "DenoKV database operations test failed (may need permissions)" +fi + +# Test 5: Check port 4512 (DenoKV server port) +print_status "Test 5: Checking DenoKV server port (4512)..." +if netstat -tlnp 2>/dev/null | grep -q ":4512 "; then + print_success "Port 4512 is open (DenoKV server may be running)" +else + print_warning "Port 4512 is closed (DenoKV server not running)" + print_status "Start DenoKV server with: ./manage-services.sh start" +fi + +echo "" +print_success "🎉 PostgreSQL connection test completed!" +echo "" +print_status "Summary:" +echo " ✅ PostgreSQL service: Running" +echo " ✅ PostgreSQL connection: Working" +echo " ✅ Database access: Working" +echo " ✅ DenoKV operations: Working" +echo "" +print_status "Your PostgreSQL URL is ready for DenoKV:" +echo " $DENO_KV_POSTGRES_URL" +echo "" +print_status "To start DenoKV server:" +echo " ./manage-services.sh start" \ No newline at end of file diff --git a/test-postgres.sh b/test-postgres.sh new file mode 100755 index 0000000..f9dfa11 --- /dev/null +++ b/test-postgres.sh @@ -0,0 +1,32 @@ +#!/bin/bash + +# Test script for PostgreSQL backend + +set -e + +echo "Starting PostgreSQL test environment..." + +# Start PostgreSQL +docker-compose -f docker-compose.test.yml up -d postgres + +# Wait for PostgreSQL to be ready +echo "Waiting for PostgreSQL to be ready..." +until docker-compose -f docker-compose.test.yml exec postgres pg_isready -U postgres; do + echo "PostgreSQL is not ready yet..." + sleep 2 +done + +echo "PostgreSQL is ready!" + +# Set environment variable for tests +export POSTGRES_URL="postgresql://postgres:password@localhost:5432/denokv_test" + +# Run the tests +echo "Running PostgreSQL tests..." +cargo test --package denokv_postgres test_postgres + +# Clean up +echo "Cleaning up..." +docker-compose -f docker-compose.test.yml down + +echo "Tests completed!" \ No newline at end of file diff --git a/test_all_operations.ts b/test_all_operations.ts new file mode 100644 index 0000000..3d8408d --- /dev/null +++ b/test_all_operations.ts @@ -0,0 +1,333 @@ +// Comprehensive Deno KV operations test +// Usage: DENO_KV_ACCESS_TOKEN= deno run --allow-net --allow-env --unstable-kv test_all_operations.ts [url] + +const KV_URL = Deno.args[0] || "http://localhost:4512"; +const ACCESS_TOKEN = + Deno.env.get("DENO_KV_ACCESS_TOKEN") || "test-access-token"; +Deno.env.set("DENO_KV_ACCESS_TOKEN", ACCESS_TOKEN); + +let passed = 0; +let failed = 0; + +function assert(condition: boolean, msg: string) { + if (!condition) throw new Error(`Assertion failed: ${msg}`); +} + +async function test(name: string, fn: (kv: Deno.Kv) => Promise, kv: Deno.Kv) { + try { + await fn(kv); + passed++; + console.log(` PASS: ${name}`); + } catch (e) { + failed++; + console.log(` FAIL: ${name} - ${(e as Error).message}`); + } +} + +async function run() { + console.log(`Connecting to ${KV_URL} ...`); + const kv = await Deno.openKv(KV_URL); + console.log("Connected.\n"); + + // --- SET / GET --- + console.log("[set / get]"); + await test("set and get string", async (kv) => { + await kv.set(["test", "string"], "hello"); + const r = await kv.get(["test", "string"]); + assert(r.value === "hello", `expected "hello", got ${r.value}`); + }, kv); + + await test("set and get number", async (kv) => { + await kv.set(["test", "number"], 42); + const r = await kv.get(["test", "number"]); + assert(r.value === 42, `expected 42, got ${r.value}`); + }, kv); + + await test("set and get boolean", async (kv) => { + await kv.set(["test", "bool"], true); + const r = await kv.get(["test", "bool"]); + assert(r.value === true, `expected true, got ${r.value}`); + }, kv); + + await test("set and get object", async (kv) => { + const obj = { name: "deno", version: 2, tags: ["kv", "test"] }; + await kv.set(["test", "object"], obj); + const r = await kv.get(["test", "object"]); + assert(r.value?.name === "deno", `object mismatch`); + assert(r.value?.tags.length === 2, `array in object mismatch`); + }, kv); + + await test("set and get Uint8Array", async (kv) => { + const bytes = new Uint8Array([1, 2, 3, 4, 5]); + await kv.set(["test", "bytes"], bytes); + const r = await kv.get(["test", "bytes"]); + assert(r.value instanceof Uint8Array, "not Uint8Array"); + assert(r.value!.length === 5, `length mismatch`); + }, kv); + + await test("set and get bigint", async (kv) => { + await kv.set(["test", "bigint"], 9007199254740993n); + const r = await kv.get(["test", "bigint"]); + assert(r.value === 9007199254740993n, `bigint mismatch`); + }, kv); + + await test("set and get null", async (kv) => { + await kv.set(["test", "null"], null); + const r = await kv.get(["test", "null"]); + assert(r.value === null, `expected null, got ${r.value}`); + }, kv); + + // --- GET non-existent key --- + console.log("\n[get non-existent]"); + await test("get non-existent key returns null with versionstamp null", async (kv) => { + const r = await kv.get(["does", "not", "exist", crypto.randomUUID()]); + assert(r.value === null, `expected null`); + assert(r.versionstamp === null, `expected null versionstamp`); + }, kv); + + // --- DELETE --- + console.log("\n[delete]"); + await test("delete removes key", async (kv) => { + await kv.set(["test", "delete_me"], "bye"); + await kv.delete(["test", "delete_me"]); + const r = await kv.get(["test", "delete_me"]); + assert(r.value === null, `expected null after delete`); + }, kv); + + // --- GET MANY --- + console.log("\n[getMany]"); + await test("getMany returns multiple values", async (kv) => { + await kv.set(["multi", "a"], 1); + await kv.set(["multi", "b"], 2); + await kv.set(["multi", "c"], 3); + const results = await kv.getMany([["multi", "a"], ["multi", "b"], ["multi", "c"]]); + assert(results.length === 3, `expected 3 results`); + assert(results[0].value === 1, `first value mismatch`); + assert(results[1].value === 2, `second value mismatch`); + assert(results[2].value === 3, `third value mismatch`); + }, kv); + + // --- LIST --- + console.log("\n[list]"); + await test("list with prefix", async (kv) => { + const prefix = crypto.randomUUID(); + await kv.set(["list", prefix, "a"], 1); + await kv.set(["list", prefix, "b"], 2); + await kv.set(["list", prefix, "c"], 3); + const entries = []; + for await (const entry of kv.list({ prefix: ["list", prefix] })) { + entries.push(entry); + } + assert(entries.length === 3, `expected 3 entries, got ${entries.length}`); + }, kv); + + await test("list with limit", async (kv) => { + const prefix = crypto.randomUUID(); + for (let i = 0; i < 5; i++) { + await kv.set(["limit", prefix, `item${i}`], i); + } + const entries = []; + for await (const entry of kv.list({ prefix: ["limit", prefix] }, { limit: 2 })) { + entries.push(entry); + } + assert(entries.length === 2, `expected 2 entries, got ${entries.length}`); + }, kv); + + await test("list reverse", async (kv) => { + const prefix = crypto.randomUUID(); + await kv.set(["rev", prefix, "a"], "first"); + await kv.set(["rev", prefix, "b"], "second"); + await kv.set(["rev", prefix, "c"], "third"); + const entries = []; + for await (const entry of kv.list({ prefix: ["rev", prefix] }, { reverse: true })) { + entries.push(entry); + } + assert(entries.length === 3, `expected 3`); + assert(entries[0].value === "third", `expected reverse order`); + }, kv); + + // --- ATOMIC OPERATIONS --- + console.log("\n[atomic]"); + await test("atomic set multiple keys", async (kv) => { + const result = await kv.atomic() + .set(["atomic", "x"], 10) + .set(["atomic", "y"], 20) + .commit(); + assert(result.ok, "atomic commit failed"); + const rx = await kv.get(["atomic", "x"]); + const ry = await kv.get(["atomic", "y"]); + assert(rx.value === 10, "x mismatch"); + assert(ry.value === 20, "y mismatch"); + }, kv); + + await test("atomic check (optimistic concurrency) - success", async (kv) => { + await kv.set(["atomic", "check"], "v1"); + const current = await kv.get(["atomic", "check"]); + const result = await kv.atomic() + .check(current) + .set(["atomic", "check"], "v2") + .commit(); + assert(result.ok, "atomic check commit should succeed"); + const r = await kv.get(["atomic", "check"]); + assert(r.value === "v2", `expected v2`); + }, kv); + + await test("atomic check (optimistic concurrency) - conflict", async (kv) => { + await kv.set(["atomic", "conflict"], "v1"); + const stale = await kv.get(["atomic", "conflict"]); + // Another write changes the versionstamp + await kv.set(["atomic", "conflict"], "v2"); + const result = await kv.atomic() + .check(stale) + .set(["atomic", "conflict"], "v3") + .commit(); + assert(!result.ok, "atomic check should fail on conflict"); + const r = await kv.get(["atomic", "conflict"]); + assert(r.value === "v2", `expected v2, value should not have changed to v3`); + }, kv); + + await test("atomic delete", async (kv) => { + await kv.set(["atomic", "del"], "remove_me"); + const result = await kv.atomic() + .delete(["atomic", "del"]) + .commit(); + assert(result.ok, "atomic delete commit failed"); + const r = await kv.get(["atomic", "del"]); + assert(r.value === null, "expected null after atomic delete"); + }, kv); + + // --- SUM (atomic mutation) --- + console.log("\n[atomic mutations]"); + await test("atomic sum mutation", async (kv) => { + await kv.set(["counter", "sum"], new Deno.KvU64(10n)); + const result = await kv.atomic() + .mutate({ type: "sum", key: ["counter", "sum"], value: new Deno.KvU64(5n) }) + .commit(); + assert(result.ok, "sum mutation failed"); + const r = await kv.get(["counter", "sum"]); + assert(r.value!.value === 15n, `expected 15n, got ${r.value!.value}`); + }, kv); + + await test("atomic min mutation", async (kv) => { + await kv.set(["counter", "min"], new Deno.KvU64(10n)); + await kv.atomic() + .mutate({ type: "min", key: ["counter", "min"], value: new Deno.KvU64(5n) }) + .commit(); + const r = await kv.get(["counter", "min"]); + assert(r.value!.value === 5n, `expected 5n`); + }, kv); + + await test("atomic max mutation", async (kv) => { + await kv.set(["counter", "max"], new Deno.KvU64(10n)); + await kv.atomic() + .mutate({ type: "max", key: ["counter", "max"], value: new Deno.KvU64(20n) }) + .commit(); + const r = await kv.get(["counter", "max"]); + assert(r.value!.value === 20n, `expected 20n`); + }, kv); + + // --- EXPIRATION (expireIn) --- + console.log("\n[expireIn]"); + await test("set with expireIn", async (kv) => { + await kv.set(["test", "expiring"], "temp", { expireIn: 60000 }); + const r = await kv.get(["test", "expiring"]); + assert(r.value === "temp", "value should exist before expiry"); + }, kv); + + // --- KEY TYPES --- + console.log("\n[key types]"); + await test("key with string parts", async (kv) => { + await kv.set(["str", "key", "parts"], "ok"); + const r = await kv.get(["str", "key", "parts"]); + assert(r.value === "ok", "string key parts failed"); + }, kv); + + await test("key with number parts", async (kv) => { + await kv.set(["num", 1, 2, 3], "ok"); + const r = await kv.get(["num", 1, 2, 3]); + assert(r.value === "ok", "number key parts failed"); + }, kv); + + await test("key with boolean parts", async (kv) => { + await kv.set(["bool", true, false], "ok"); + const r = await kv.get(["bool", true, false]); + assert(r.value === "ok", "boolean key parts failed"); + }, kv); + + await test("key with bigint parts", async (kv) => { + await kv.set(["bigint", 999999999999999999n], "ok"); + const r = await kv.get(["bigint", 999999999999999999n]); + assert(r.value === "ok", "bigint key parts failed"); + }, kv); + + await test("key with Uint8Array parts", async (kv) => { + const keyPart = new Uint8Array([0xDE, 0xAD]); + await kv.set(["bytes", keyPart], "ok"); + const r = await kv.get(["bytes", keyPart]); + assert(r.value === "ok", "Uint8Array key parts failed"); + }, kv); + + // --- VERSIONSTAMP --- + console.log("\n[versionstamp]"); + await test("versionstamp changes on update", async (kv) => { + await kv.set(["vs", "track"], "v1"); + const r1 = await kv.get(["vs", "track"]); + await kv.set(["vs", "track"], "v2"); + const r2 = await kv.get(["vs", "track"]); + assert(r1.versionstamp !== r2.versionstamp, "versionstamp should change"); + }, kv); + + // --- WATCH --- + console.log("\n[watch]"); + await test("watch detects changes", async (kv) => { + const key = ["watch", crypto.randomUUID()]; + await kv.set(key, "initial"); + const stream = kv.watch<[string]>([key]); + const reader = stream.getReader(); + + // Read initial value + const { value: initial } = await reader.read(); + assert(initial![0].value === "initial", "watch initial value mismatch"); + + // Trigger a change + await kv.set(key, "updated"); + const { value: updated } = await reader.read(); + assert(updated![0].value === "updated", "watch updated value mismatch"); + + reader.releaseLock(); + stream.cancel(); + }, kv); + + // --- ENQUEUE / LISTEN (basic) --- + console.log("\n[enqueue]"); + await test("enqueue message", async (kv) => { + // Just test that enqueue doesn't throw - full listen requires a handler + await kv.enqueue({ type: "test", data: "hello" }); + }, kv); + + // --- CLEANUP --- + console.log("\n[cleanup]"); + const prefixes = [ + "test", "multi", "list", "limit", "rev", "atomic", + "counter", "str", "num", "bool", "bigint", "bytes", "vs", "watch", + ]; + for (const prefix of prefixes) { + for await (const entry of kv.list({ prefix: [prefix] })) { + await kv.delete(entry.key); + } + } + console.log(" Cleaned up test keys.\n"); + + kv.close(); + + // --- SUMMARY --- + console.log("=".repeat(40)); + console.log(`Results: ${passed} passed, ${failed} failed, ${passed + failed} total`); + if (failed > 0) { + Deno.exit(1); + } else { + console.log("All tests passed!"); + } +} + +run(); diff --git a/test_denokv.ts b/test_denokv.ts new file mode 100644 index 0000000..4ce9b32 --- /dev/null +++ b/test_denokv.ts @@ -0,0 +1,50 @@ +const KV_URL = "http://102.37.137.29:4512"; +const ACCESS_TOKEN = "d4f2332c86df1ec68911c73b51c9dbad"; + +async function testDenoKV() { + try { + console.log('🔗 Testing DenoKV connection...'); + + // Set the access token as environment variable + Deno.env.set("DENO_KV_ACCESS_TOKEN", ACCESS_TOKEN); + + // Open KV connection using native Deno KV API + const kv = await Deno.openKv(KV_URL); + console.log('✅ KV connection opened successfully'); + + // Test KV operations + const testKey = ['test', 'key']; + const testValue = 'Hello DenoKV!'; + + // Set a value + await kv.set(testKey, testValue); + console.log('✅ Set operation successful - Key:', testKey, 'Value:', testValue); + + // Get the value + const result = await kv.get(testKey); + console.log('✅ Get operation successful - Retrieved:', result.value); + + // Clean up + await kv.delete(testKey); + console.log('✅ Delete operation successful - Removed key:', testKey); + + // Close connection + kv.close(); + console.log('🎉 All DenoKV tests passed!'); + } catch (error) { + console.error('❌ Test failed:', error); + } +} + +testDenoKV(); + + + 1 │# Check PostgreSQL logs for crash details + 2 │sudo tail -100 /var/log/postgresql/postgresql-*.log | grep -i "crash\|fatal\|panic" + 3 │ + 4 │# Check for OOM kills + 5 │sudo dmesg | grep -i "out of memory\|killed process" + 6 │ + 7 │# Check system resources + 8 │free -h + 9 │df -h \ No newline at end of file diff --git a/test_postgres_features.ts b/test_postgres_features.ts new file mode 100644 index 0000000..ed7f267 --- /dev/null +++ b/test_postgres_features.ts @@ -0,0 +1,236 @@ +// Tests for postgres-specific features: key expiration, concurrency, large values +// Enqueue is NOT supported via KV Connect protocol, so not tested here. +// +// Usage: DENO_KV_ACCESS_TOKEN= deno run --allow-net --allow-env --unstable-kv test_postgres_features.ts [url] +// +// The key expiration tests validate the postgres read-time filtering added in +// commit 21c1e41. On the SQLite backend, expired keys are only cleaned up by a +// background task (~60s interval), so those tests will fail against SQLite. + +const KV_URL = Deno.args[0] || "http://localhost:4512"; +const ACCESS_TOKEN = + Deno.env.get("DENO_KV_ACCESS_TOKEN") || "test-access-token"; +Deno.env.set("DENO_KV_ACCESS_TOKEN", ACCESS_TOKEN); + +let passed = 0; +let failed = 0; + +function assert(condition: boolean, msg: string) { + if (!condition) throw new Error(`Assertion failed: ${msg}`); +} + +async function test(name: string, fn: () => Promise) { + try { + await fn(); + passed++; + console.log(` PASS: ${name}`); + } catch (e) { + failed++; + console.log(` FAIL: ${name} - ${(e as Error).message}`); + } +} + +async function run() { + console.log(`Connecting to ${KV_URL} ...`); + const kv = await Deno.openKv(KV_URL); + console.log("Connected.\n"); + + // --- KEY EXPIRATION (postgres read-time filtering) --- + console.log("[key expiration]"); + + await test("key with expireIn is readable before expiry", async () => { + const key = ["expire_test", crypto.randomUUID()]; + await kv.set(key, "temporary", { expireIn: 30000 }); // 30s + const r = await kv.get(key); + assert(r.value === "temporary", `expected "temporary", got ${r.value}`); + await kv.delete(key); + }); + + await test("expired key filtered on get (postgres only)", async () => { + const key = ["expire_test", crypto.randomUUID()]; + await kv.set(key, "will_expire", { expireIn: 1000 }); + const before = await kv.get(key); + assert(before.value === "will_expire", "should exist before expiry"); + + console.log(" (waiting 2s for key to expire...)"); + await new Promise((r) => setTimeout(r, 2000)); + + const after = await kv.get(key); + if (after.value === null) { + console.log(" -> read-time filtering is active (postgres backend)"); + } else { + console.log(" -> key still visible - backend relies on background cleanup (sqlite behavior)"); + } + // This is informational — pass either way since both are valid behaviors + // depending on the backend. The postgres fix makes this null immediately. + assert(true, ""); + // cleanup in case it's still there + await kv.delete(key); + }); + + await test("expired key filtered in list (postgres only)", async () => { + const prefix = crypto.randomUUID(); + await kv.set(["expire_list", prefix, "persistent"], "stays"); + await kv.set(["expire_list", prefix, "ephemeral"], "goes", { expireIn: 1000 }); + + console.log(" (waiting 2s for key to expire...)"); + await new Promise((r) => setTimeout(r, 2000)); + + const entries = []; + for await (const entry of kv.list({ prefix: ["expire_list", prefix] })) { + entries.push(entry); + } + if (entries.length === 1) { + console.log(" -> expired key excluded from list (postgres read-time filtering)"); + } else { + console.log(` -> ${entries.length} entries returned - expired key still in list (sqlite behavior)`); + } + assert(true, ""); + + // cleanup + for await (const entry of kv.list({ prefix: ["expire_list", prefix] })) { + await kv.delete(entry.key); + } + }); + + await test("atomic check on expired key (postgres only)", async () => { + const key = ["expire_check", crypto.randomUUID()]; + await kv.set(key, "old_value", { expireIn: 1000 }); + + console.log(" (waiting 2s for key to expire...)"); + await new Promise((r) => setTimeout(r, 2000)); + + // On postgres: expired key treated as non-existent, check(null) succeeds + // On sqlite: key still exists, check(null) fails + const result = await kv.atomic() + .check({ key, versionstamp: null }) + .set(key, "new_value") + .commit(); + if (result.ok) { + console.log(" -> atomic check treated expired key as non-existent (postgres)"); + const r = await kv.get(key); + assert(r.value === "new_value", `expected "new_value", got ${r.value}`); + } else { + console.log(" -> atomic check saw expired key as still existing (sqlite behavior)"); + } + assert(true, ""); + await kv.delete(key); + }); + + // --- CONCURRENT OPERATIONS (connection pool stress) --- + console.log("\n[concurrent operations]"); + + await test("50 concurrent set/get operations", async () => { + const prefix = crypto.randomUUID(); + const ops = Array.from({ length: 50 }, (_, i) => { + const key = ["concurrent", prefix, `key${i}`]; + return kv.set(key, `value${i}`).then(() => kv.get(key)).then((r) => { + assert(r.value === `value${i}`, `concurrent get mismatch at ${i}`); + }); + }); + await Promise.all(ops); + + // cleanup + for await (const entry of kv.list({ prefix: ["concurrent", prefix] })) { + await kv.delete(entry.key); + } + }); + + await test("10 concurrent atomic transactions", async () => { + const prefix = crypto.randomUUID(); + const ops = Array.from({ length: 10 }, async (_, i) => { + const key = ["atomic_concurrent", prefix, `key${i}`]; + const result = await kv.atomic() + .set(key, i * 100) + .commit(); + assert(result.ok, `atomic ${i} failed`); + }); + await Promise.all(ops); + + for (let i = 0; i < 10; i++) { + const r = await kv.get(["atomic_concurrent", prefix, `key${i}`]); + assert(r.value === i * 100, `value mismatch for key${i}`); + } + + // cleanup + for await (const entry of kv.list({ prefix: ["atomic_concurrent", prefix] })) { + await kv.delete(entry.key); + } + }); + + await test("atomic conflict under concurrent writes", async () => { + const key = ["conflict_race", crypto.randomUUID()]; + await kv.set(key, 0); + const initial = await kv.get(key); + + // Two atomic writes using the same stale versionstamp — one should fail + const [r1, r2] = await Promise.all([ + kv.atomic().check(initial).set(key, 1).commit(), + kv.atomic().check(initial).set(key, 2).commit(), + ]); + + const succeeded = [r1.ok, r2.ok].filter(Boolean).length; + assert(succeeded === 1, `expected exactly 1 success, got ${succeeded}`); + await kv.delete(key); + }); + + // --- LARGE VALUES --- + console.log("\n[large values]"); + + await test("store and retrieve near-max value (60KB)", async () => { + const key = ["large", crypto.randomUUID()]; + const largeString = "x".repeat(60 * 1024); // 60KB, under 65536 limit + await kv.set(key, largeString); + const r = await kv.get(key); + assert(r.value?.length === 60 * 1024, `expected 60KB, got ${r.value?.length}`); + await kv.delete(key); + }); + + await test("value over 65536 bytes is rejected", async () => { + const key = ["large_reject", crypto.randomUUID()]; + const tooLarge = "x".repeat(65537); + try { + await kv.set(key, tooLarge); + assert(false, "should have thrown for oversized value"); + } catch (_e) { + // Expected + } + }); + + // --- MIXED KEY TYPES IN RANGE QUERIES --- + console.log("\n[range queries with mixed key types]"); + + await test("list with start/end range", async () => { + const prefix = crypto.randomUUID(); + for (let i = 0; i < 10; i++) { + await kv.set(["range", prefix, `item_${String(i).padStart(2, "0")}`], i); + } + const entries = []; + for await (const entry of kv.list({ + start: ["range", prefix, "item_03"], + end: ["range", prefix, "item_07"], + })) { + entries.push(entry); + } + assert(entries.length === 4, `expected 4 entries in range, got ${entries.length}`); + assert(entries[0].value === 3, `first should be 3`); + assert(entries[3].value === 6, `last should be 6`); + + // cleanup + for await (const entry of kv.list({ prefix: ["range", prefix] })) { + await kv.delete(entry.key); + } + }); + + // --- SUMMARY --- + kv.close(); + console.log("\n" + "=".repeat(40)); + console.log(`Results: ${passed} passed, ${failed} failed, ${passed + failed} total`); + if (failed > 0) { + Deno.exit(1); + } else { + console.log("All tests passed!"); + } +} + +run(); diff --git a/upgrade-denokv.sh b/upgrade-denokv.sh new file mode 100755 index 0000000..ac66880 --- /dev/null +++ b/upgrade-denokv.sh @@ -0,0 +1,219 @@ +#!/bin/bash + +# Upgrade script for DenoKV on Rocky Linux +# This script pulls the latest changes and rebuilds the project + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Function to print colored output +print_status() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +print_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}[WARNING]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +echo "🔄 DenoKV Upgrade Script for Rocky Linux" +echo "==========================================" +echo "" + +# Check if we're in the right directory +if [ ! -f "Cargo.toml" ] || [ ! -f "setup-rocky-linux.sh" ]; then + print_error "This script must be run from the DenoKV project root directory" + print_error "Make sure you're in the directory that contains Cargo.toml and setup-rocky-linux.sh" + exit 1 +fi + +# Check if git is available +if ! command -v git &> /dev/null; then + print_error "Git is not installed. Please run the setup script first:" + print_error " ./setup-rocky-linux.sh" + exit 1 +fi + +# Check if cargo is available +if ! command -v cargo &> /dev/null; then + print_error "Cargo is not installed. Please run the setup script first:" + print_error " ./setup-rocky-linux.sh" + exit 1 +fi + +# Check current status +print_status "Checking current git status..." +git status --porcelain + +# Check if there are uncommitted changes +if [ -n "$(git status --porcelain)" ]; then + print_warning "You have uncommitted changes:" + git status --short + echo "" + read -p "Do you want to stash these changes before upgrading? (y/N): " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + print_status "Stashing uncommitted changes..." + git stash push -m "Auto-stash before upgrade $(date)" + print_success "Changes stashed" + else + print_warning "Proceeding with uncommitted changes..." + fi +fi + +# Fetch latest changes +print_status "Fetching latest changes from remote..." +git fetch origin + +# Check current branch +CURRENT_BRANCH=$(git branch --show-current) +print_status "Current branch: $CURRENT_BRANCH" + +# Check if there are updates available +BEHIND=$(git rev-list --count HEAD..origin/$CURRENT_BRANCH 2>/dev/null || echo "0") +if [ "$BEHIND" -eq 0 ]; then + print_success "You're already up to date!" + print_status "No new commits to pull" +else + print_status "Found $BEHIND new commit(s) to pull" +fi + +# Pull latest changes +print_status "Pulling latest changes..." +if git pull origin $CURRENT_BRANCH; then + print_success "Successfully pulled latest changes" +else + print_error "Failed to pull changes. Please resolve conflicts manually." + exit 1 +fi + +# Clean build artifacts +print_status "Cleaning previous build artifacts..." +cargo clean + +# Source Rust environment +print_status "Sourcing Rust environment..." +source ~/.cargo/env + +# Update dependencies +print_status "Updating dependencies..." +cargo update + +# Build the project +print_status "Building DenoKV with latest changes..." +if cargo build --release; then + print_success "Build completed successfully!" +else + print_error "Build failed. Please check the error messages above." + exit 1 +fi + +# Check if any scripts need to be updated +print_status "Checking for script updates..." + +# Update script permissions +chmod +x setup-rocky-linux.sh 2>/dev/null || true +chmod +x start-denokv-server.sh 2>/dev/null || true +chmod +x test-postgres-integration.sh 2>/dev/null || true +chmod +x generate-access-token.sh 2>/dev/null || true + +print_success "Script permissions updated" + +# Show upgrade summary +echo "" +print_success "🎉 Upgrade completed successfully!" +echo "" +print_status "Summary:" +echo " ✅ Latest changes pulled from remote" +echo " ✅ Dependencies updated" +echo " ✅ Project rebuilt successfully" +echo " ✅ Script permissions updated" +echo "" + +# Install the new binary +print_status "Installing new DenoKV binary..." +if [ -f "target/release/denokv" ]; then + # Stop service first so we can overwrite the binary + if systemctl list-unit-files | grep -q "denokv.service"; then + if sudo systemctl is-active --quiet denokv.service; then + print_status "Stopping DenoKV service to update binary..." + sudo systemctl stop denokv.service + fi + fi + + # Copy the binary + sudo cp target/release/denokv /usr/local/bin/denokv + sudo chmod +x /usr/local/bin/denokv + sudo chown root:root /usr/local/bin/denokv + print_success "Binary installed to /usr/local/bin/denokv" +else + print_error "Binary not found at target/release/denokv" + exit 1 +fi + +# Check if systemd service exists and start it +if systemctl list-unit-files | grep -q "denokv.service"; then + print_status "Starting DenoKV systemd service with new binary..." + if sudo systemctl start denokv.service; then + sleep 2 + if sudo systemctl is-active --quiet denokv.service; then + print_success "DenoKV service restarted successfully!" + else + print_error "Service restarted but is not active. Check status:" + echo " sudo systemctl status denokv.service" + fi + else + print_error "Failed to restart service" + exit 1 + fi +else + print_warning "Systemd service not found. Server may be running manually." + if pgrep -f "denokv.*serve" > /dev/null; then + print_status "DenoKV process found. You may want to restart it manually:" + echo " pkill -f 'denokv.*serve' # Stop current server" + echo " ./start-denokv-server.sh # Start with new version" + fi +fi + +echo "" +print_status "Available commands:" +echo " sudo systemctl status denokv.service - Check service status" +echo " sudo journalctl -u denokv.service -f - View service logs" +echo " ./upgrade-denokv.sh - Run this upgrade script again" +echo "" + + + 1 │# Stop the service first + 2 │sudo systemctl stop denokv.service + 3 │ + 4 │# Then copy the binary + 5 │sudo cp target/release/denokv /usr/local/bin/denokv + 6 │sudo chmod +x /usr/local/bin/denokv + 7 │sudo chown root:root /usr/local/bin/denokv + 8 │ + 9 │# Start the service + 10 │sudo systemctl start denokv.service + 11 │ + 12 │# Verify + 13 │sudo systemctl status denokv.service + + + 3 │cargo build --release + 4 │sudo systemctl stop denokv.service + 5 │sudo cp target/release/denokv /usr/local/bin/denokv + 6 │sudo chmod +x /usr/local/bin/denokv + 7 │sudo systemctl start denokv.service + 8 │sudo systemctl status denokv.service \ No newline at end of file