Open
Conversation
added 30 commits
September 27, 2025 17:49
- Add setup-rocky-linux.sh: Comprehensive setup script for Rocky Linux VMs - Add docker-compose.test.yml: PostgreSQL test container configuration - Add test-postgres.sh: PostgreSQL integration test script - Add postgres/ directory: PostgreSQL backend implementation - Add ROCKY_LINUX_SETUP.md: Documentation for Rocky Linux setup - Update Cargo.toml files: Add PostgreSQL dependencies - Update various source files: PostgreSQL integration support The setup script installs all necessary dependencies including: - Development tools and PostgreSQL dev libraries - Docker and Docker Compose - Rust toolchain - Clones repository and builds project - Creates test environment for PostgreSQL KV integration
…ript - Update setup script to include access token requirements - Add start-denokv-server.sh script for production deployment - Update test script to include test access token - Add comprehensive documentation about authentication - Include security notes about token generation and protection - Add client authentication examples for Deno applications The access token is required for remote access and must be: - Minimum 12 characters long - Securely generated and protected - Used in Authorization header as Bearer token
- Update start-denokv-server.sh to auto-generate secure tokens if not provided - Add generate-access-token.sh utility script for manual token generation - Support multiple token generation methods (OpenSSL, /dev/urandom) - Add option to save tokens to .env file - Update documentation with automatic token generation options - Improve user experience by removing manual token requirement Token generation features: - Uses OpenSSL rand -hex 16 (32 characters) when available - Falls back to /dev/urandom with base64 encoding - Displays token securely with partial masking - Provides clear usage instructions and security notes
- Automatically configure firewalld to open port 4512 during setup - Add firewall configuration instructions to documentation - Include port verification commands - Ensure remote access is properly configured The setup script now handles: - Opening port 4512/tcp in firewall - Reloading firewall rules - Providing verification commands - Warning if firewalld is not available
- Create upgrade-denokv.sh script for easy updates - Handles git pull, dependency updates, and rebuilds - Includes safety checks for uncommitted changes - Provides stash option for uncommitted work - Shows upgrade summary and next steps - Updates script permissions automatically - Checks for running server and provides restart guidance Features: - Fetches and pulls latest changes from remote - Cleans build artifacts before rebuilding - Updates Cargo dependencies - Rebuilds project with latest changes - Provides helpful status messages and next steps - Handles edge cases like uncommitted changes gracefully
- Add PostgreSQL Docker startup to setup script - Automatically set environment variables and create .env file - Generate and save access token during setup - Run integration tests during setup - Start DenoKV server automatically in background - Add comprehensive service management script (manage-services.sh) - Provide complete status reporting and management commands New features: - Full end-to-end setup: install → configure → start → test - PostgreSQL starts in Docker automatically - Environment variables configured and saved to .env - Access token generated and saved - DenoKV server starts automatically - Service management script for start/stop/restart/status/logs - Complete status reporting with PIDs and connection info The setup script now provides a complete working DenoKV server ready for remote connections!
- Replace Docker test PostgreSQL with production PostgreSQL service - Install and configure PostgreSQL server as system service - Create dedicated denokv database and user with proper credentials - Update service management to keep PostgreSQL running persistently - Add separate commands for PostgreSQL service control - Update environment variables to use production database - Fix service management script to not stop PostgreSQL on DenoKV stop Key changes: - PostgreSQL runs as persistent system service (not Docker test container) - Database: denokv, User: denokv, Password: denokv_password - DenoKV server can be started/stopped independently of PostgreSQL - PostgreSQL service remains running for data persistence - Added start-postgres/stop-postgres commands for PostgreSQL control
- Update test script to use production PostgreSQL service instead of Docker - Remove Docker container startup from test script - Ensure test script uses same database as production setup - Update documentation to reflect production PostgreSQL service - Remove Docker cleanup commands from test script - Fix environment variable documentation Now both setup and test scripts use the same PostgreSQL service: - Database: denokv - User: denokv - Password: denokv_password - Service: PostgreSQL system service (not Docker)
- Remove ?Send restriction from async_trait implementations - Add Send + Sync bounds to RemotePermissions trait - Fix watch method to return Send stream - Resolve compilation errors related to thread safety The async trait methods now properly implement Send requirements for thread-safe execution across async runtimes.
- Add PostgreSQL authentication configuration to setup script - Update pg_hba.conf to use md5 authentication instead of ident - Add database connection testing after user creation - Create standalone fix-postgres-auth.sh script for existing installations - Handle multiple PostgreSQL data directory locations - Add explicit denokv user entry in pg_hba.conf - Include authentication fix in management commands This resolves the 'Ident authentication failed' error by: - Changing authentication method from 'ident' to 'md5' - Adding explicit user entries for password authentication - Testing connections after configuration changes - Providing standalone fix script for troubleshooting
- Create fix-postgres-auth.sh as a standalone script - Fixes 'Ident authentication failed' errors - Updates pg_hba.conf from ident/peer to md5 authentication - Handles multiple PostgreSQL data directory locations - Includes connection testing and troubleshooting - Can be run independently of the main setup script Usage: ./fix-postgres-auth.sh
- Create manage-services.sh as a standalone script - Provides start/stop/restart/status/logs commands for DenoKV services - Manages PostgreSQL service and DenoKV server independently - Includes service status checking and port monitoring - Can be used without running the full setup script Commands: - start: Start DenoKV server (PostgreSQL must be running) - stop: Stop DenoKV server only (PostgreSQL stays running) - restart: Restart DenoKV server only - status: Show service status - logs: Show DenoKV server logs - start-postgres: Start PostgreSQL service - stop-postgres: Stop PostgreSQL service
- Create test-postgres-connection.sh for testing database connectivity - Tests PostgreSQL service status and connection - Validates database user and database existence - Tests DenoKV-specific database operations - Provides detailed diagnostics for connection issues - Shows current environment variables and connection details Usage: ./test-postgres-connection.sh This script helps diagnose PostgreSQL connection issues and verifies the database is ready for DenoKV operations.
- Update setup script to check if PostgreSQL is already initialized - Prevent initialization errors when data directory exists - Add setup-existing-postgres.sh for existing PostgreSQL setups - Improve error handling and user feedback - Add sleep delay in fix script for service startup Changes: - Check if /var/lib/pgsql/data exists and is not empty before init - Create dedicated script for existing PostgreSQL installations - Better error messages and status reporting - Handle both fresh and existing PostgreSQL setups gracefully
- Update setup-existing-postgres.sh to handle password authentication - Add fallback for empty password when peer auth fails - Create quick-setup.sh for simple database setup - Improve error handling for authentication scenarios - Provide clear instructions for password setup The scripts now handle both peer and password authentication and provide better error messages when authentication fails.
- Update scripts to source Rust environment from correct user directory - Add fallback to find cargo if environment file not found - Handle both current user and rawkakani user paths - Prevent 'No such file or directory' errors for /root/.cargo/env This fixes the issue where scripts tried to source Rust from root directory when running as non-root user.
- Add setup-complete.sh: All-in-one setup script - Remove redundant scripts: fresh-postgres-setup.sh, setup-rocky-linux.sh, test_kv_connection.ts - Add native Deno KV test scripts: test_deno_kv.ts, test_native_deno_kv.ts - Setup includes: PostgreSQL fresh install, environment variables, systemd service - DenoKV runs as systemd service with auto-start and auto-restart - Complete cleanup of unnecessary files - Production-ready setup with proper security and logging
- Install PostgreSQL 16 (latest stable version) instead of generic postgresql - Remove Docker installation (not needed for local PostgreSQL) - Use version-specific PostgreSQL commands and services - Update systemd service dependencies to postgresql-16.service - Verify PostgreSQL version during installation - Clean up PostgreSQL 16 packages during removal - Simplified setup without unnecessary Docker overhead
- Change working directory from /home/denokv to /var/lib/denokv - Create denokv user with /var/lib/denokv as home directory - Add ReadWritePaths=/var/lib/denokv to systemd service - Ensure working directory exists with proper permissions before starting service - Fixes 'status=200/CHDIR' error in systemd service
- Change from postgresql16* to postgresql* packages - Use standard postgresql-setup instead of version-specific setup - Use postgresql.service instead of postgresql-16.service - Fixes 'No match for argument' error in Rocky Linux repositories - Maintains compatibility with Rocky Linux package naming
- Add --access-token argument to ExecStart command - Add DENOKV_ACCESS_TOKEN environment variable to systemd service - Fixes 'required arguments were not provided: --access-token' error - DenoKV service now has proper access token configuration
- Add DENO_KV_DATABASE_TYPE=postgres to systemd service - Add DENO_KV_POSTGRES_URL with proper PostgreSQL connection string - Fixes 'SQLite path is required' error by configuring PostgreSQL - DenoKV now properly connects to PostgreSQL database
- Generate random 32-character hex access token using openssl rand -hex 16 - Replace hardcoded token with dynamic generation - Display generated token to user for application connection - Token is saved to .env file and systemd service automatically - Improves security by using unique token per installation
- Provides ready-to-use test script with generated access token - Tests basic connectivity, set/get/delete operations - Includes proper error handling and cleanup - Users can copy-paste the script and run with: deno run --allow-net test_denokv.ts - Makes it easy to verify DenoKV installation is working correctly
- Replace HTTP fetch calls with Deno.openKv() native API - Use proper KV key format with arrays ['test', 'key'] - Add atomic operations testing (set multiple keys atomically) - Add list operations with prefix filtering - Include proper connection cleanup with kv.close() - Update run command to include --unstable-kv flag - More comprehensive testing of DenoKV features
- Remove atomic and list operations for simplicity - Add detailed logging for set operation (key and value) - Add detailed logging for get operation (retrieved value) - Add detailed logging for delete operation (removed key) - Keep test focused on core KV functionality
…emd service - Variable is set before systemd service creation - Heredoc should now properly expand the access token - Fixes 'a value is required for --access-token' error - Remove old test files that are no longer needed
- Change $DENOKV_ACCESS_TOKEN to ${DENOKV_ACCESS_TOKEN} in ExecStart
- Change $DENOKV_ADDR to ${DENOKV_ADDR} in ExecStart
- Change $DENOKV_ACCESS_TOKEN to ${DENOKV_ACCESS_TOKEN} in Environment
- Ensures proper variable expansion in heredoc for systemd service
- Fixes 'a value is required for --access-token' error
- Remove enqueue restriction in KV Connect remote client - Implement enqueue conversion to protobuf format - Add retry logic with exponential backoff for transient connection failures - Add connection validation before use - Improve error handling to distinguish transient vs permanent errors - Configure deadpool with better connection recovery settings - Add Clone trait to KvValue and MutationKind for retry logic - Add documentation for connection issues and disk expansion - Add utility scripts for disk management and PostgreSQL troubleshooting
…onnection-recovery Add enqueue support to KV Connect and improve connection recovery
…ecovery - Add enqueue operations support in remote/KV Connect protocol - Implement connection retry logic with exponential backoff for transient failures - Add connection validation and health checks - Improve error handling to distinguish transient vs permanent errors - Add Clone traits to proto structs for retry logic - Fix deadpool wait_timeout issue (removed as it requires runtime) - Update upgrade script to stop service before binary update and restart after
- Remove connection retry logic and transient error handling - Remove is_transient() method from PostgresError - Keep enqueue operations support in KV Connect protocol - Simplify connection handling back to original implementation
The postgres backend stored `expires_at` but never filtered expired keys on reads or checks, and never cleaned them up. This caused stale locks (e.g. cron leader keys) to persist indefinitely, blocking all workers. Changes: - Filter expired keys in `read_range` queries (WHERE expires_at IS NULL OR expires_at > now) - Treat expired keys as non-existent in `atomic_write` checks - Add `collect_expired()` method + 60s background task to delete expired rows - Add `queue_cleanup()` method + 30s background task to requeue dead messages - Fix `message_handle::finish()` to support backoff schedule and keys_if_undelivered (matching SQLite backend behavior) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: postgres backend key expiration + queue parity with SQLite
- Bump postgres crate version to 0.13.0 to match upstream - Add time.rs module to postgres crate using utc_now() instead of Utc::now() (chrono workspace dep lacks "clock" feature) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Atomic concurrency: READ COMMITTED isolation allowed two concurrent atomic writes to both pass the same versionstamp check. Fixed by using SERIALIZABLE isolation and catching serialization failures (40001) as check conflicts (returns ok: false instead of 500). 2. Watch hanging: notify_key_update() was defined but never called after mutations, so watchers blocked forever. Added notification of watchers in atomic_write after successful commits. Also adds comprehensive test scripts for all KV operations and postgres-specific features (expiration filtering, concurrency, etc). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace SERIALIZABLE isolation + serialization error catching with a monotonic version counter in a `data_version` table, matching the SQLite backend design. UPDATE ... RETURNING on the single version row takes an exclusive row lock, serializing all writers under plain READ COMMITTED isolation. Concurrent writers queue up instead of aborting — no 500 errors, no retries needed. The second writer blocks until the first commits, then sees the updated versionstamp and fails the check cleanly. This is the standard pessimistic locking pattern used in fintech for transaction serialization (SELECT FOR UPDATE / UPDATE RETURNING). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.