|
1 | 1 | [](https://github.com/Inferara/inf-wasm-tools/actions/workflows/build.yml) |
2 | 2 |  |
3 | 3 |
|
4 | | -# JsonMutexDb |
| 4 | +# JsonMutexDB 💾 |
5 | 5 |
|
6 | 6 | Ridiculously simple, fast and thread safe JSON file database |
| 7 | + |
| 8 | +Ever found yourself needing a *really* simple way to persist some state in a Rust application? Maybe a configuration file, some user preferences, or the results of that *one* calculation you don't want to run again? And did you also need multiple threads to poke at that state without setting your data on fire? 🔥 |
| 9 | + |
| 10 | +`JsonMutexDB` was born out of a desire for a straightforward, thread-safe mechanism to manage data stored in a single JSON file. It doesn't try to be a full-fledged database, but it's pretty handy for those "I just need to save this `struct` somewhere" moments. |
| 11 | + |
| 12 | +## What's Inside? ✨ |
| 13 | + |
| 14 | +* **Thread-Safe Access:** Uses `std::sync::Mutex` (or potentially `parking_lot::Mutex` depending on historical versions) under the hood, allowing multiple threads to safely read (`get`) and write (`update`) data. |
| 15 | +* **JSON Persistence:** Reads from and saves data to a JSON file you specify. Handles empty or non-existent files gracefully on startup. |
| 16 | +* **Atomic Saves:** Writes are performed atomically by default (using `tempfile` and rename) to prevent data corruption if your application crashes mid-save. Safety first! |
| 17 | +* **Serialization Options:** |
| 18 | + * Save JSON in a compact format (default) or human-readable "pretty" format. |
| 19 | + * Optionally use `simd-json` for potentially faster serialization when saving in compact mode. Speed boost! 🚀 |
| 20 | +* **Optional Asynchronous Updates:** For scenarios where you don't want your main threads blocked by updates, you can enable `async_updates`. Updates are sent to a dedicated background thread for processing. |
| 21 | + * **State Synchronization:** When async mode is enabled, `get()` and `save_sync()` intelligently query the background thread to ensure they operate on the *absolute latest* state. (This involves some channel communication overhead). |
| 22 | +* **Asynchronous Saving:** Offload the potentially slow file I/O of saving to a background thread with `save_async()`. |
| 23 | + |
| 24 | +## Quick Start 🚀 |
| 25 | + |
| 26 | +```bash |
| 27 | +cargo add json-mutex-db |
| 28 | +``` |
| 29 | + |
| 30 | +And get going in your code: |
| 31 | + |
| 32 | +```rust |
| 33 | +use jsonmutexdb::{JsonMutexDB, DbError}; |
| 34 | +use serde::{Serialize, Deserialize}; |
| 35 | +use serde_json::json; |
| 36 | + |
| 37 | +#[derive(Serialize, Deserialize, PartialEq, Debug, Clone)] // Needed for storage/retrieval |
| 38 | +struct Config { |
| 39 | + api_key: Option<String>, |
| 40 | + retries: u32, |
| 41 | +} |
| 42 | + |
| 43 | +fn main() -> Result<(), DbError> { |
| 44 | + let db_path = "my_app_state.json"; |
| 45 | + // Create DB (sync mode, compact, standard serialization) |
| 46 | + let db = JsonMutexDB::new(db_path, false, false, false)?; |
| 47 | + |
| 48 | + // Get initial data (starts empty if file doesn't exist) |
| 49 | + let initial_val = db.get()?; |
| 50 | + println!("Initial value: {}", initial_val); // Outputs: {} |
| 51 | + |
| 52 | + // Update the data - replace the whole value |
| 53 | + let initial_config = Config { api_key: None, retries: 3 }; |
| 54 | + db.update(|data| { |
| 55 | + *data = serde_json::to_value(&initial_config).unwrap(); |
| 56 | + })?; |
| 57 | + |
| 58 | + // Update part of the data (if it's an object) |
| 59 | + db.update(|data| { |
| 60 | + if let Some(obj) = data.as_object_mut() { |
| 61 | + obj.insert("retries".to_string(), json!(5)); |
| 62 | + obj.insert("new_feature_enabled".to_string(), json!(true)); |
| 63 | + } |
| 64 | + })?; |
| 65 | + |
| 66 | + // Get the current state |
| 67 | + let current_val = db.get()?; |
| 68 | + println!("Current value: {}", current_val); |
| 69 | + // Example Output: {"api_key":null,"new_feature_enabled":true,"retries":5} |
| 70 | + |
| 71 | + // Try to deserialize it back |
| 72 | + let current_config: Config = serde_json::from_value(current_val.clone()) |
| 73 | + .expect("Failed to deserialize"); |
| 74 | + println!("Deserialized: {:?}", current_config); |
| 75 | + assert_eq!(current_config.retries, 5); |
| 76 | + |
| 77 | + |
| 78 | + // Save it synchronously (atomic by default) |
| 79 | + println!("Saving data..."); |
| 80 | + db.save_sync()?; |
| 81 | + println!("Data saved to {}", db_path); |
| 82 | + |
| 83 | + // Cleanup the file for the example |
| 84 | + std::fs::remove_file(db_path).ok(); |
| 85 | + |
| 86 | + Ok(()) |
| 87 | +} |
| 88 | + |
| 89 | +// Remember to define your crate (replace with actual implementation) |
| 90 | +mod jsonmutexdb { |
| 91 | + // Paste the full JsonMutexDB implementation here... |
| 92 | + pub use crate::{JsonMutexDB, DbError}; // Assuming it's in src/lib.rs |
| 93 | +} |
| 94 | +``` |
| 95 | + |
| 96 | +## Configuration Options (new) ⚙️ |
| 97 | + |
| 98 | +When creating a JsonMutexDB, you have a few choices: |
| 99 | + |
| 100 | +```rust |
| 101 | +pub fn new( |
| 102 | + path: &str, // Path to the JSON file |
| 103 | + pretty: bool, // `true` for pretty-printed JSON, `false` for compact |
| 104 | + async_updates: bool, // `true` to enable background thread for updates |
| 105 | + fast_serialization: bool, // `true` to use simd-json for compact serialization (if `pretty` is false) |
| 106 | +) -> Result<Self, DbError> |
| 107 | +``` |
| 108 | +* `path`: The path to the JSON file. If it doesn't exist, it will be created. |
| 109 | +* `pretty`: If true, the JSON will be saved in a human-readable format. If false, it will be compact. This affects both `save_sync()` and `save_async()`. |
| 110 | +* `async_updates`: If true, updates are sent to a background thread. This allows the main thread to continue without waiting for the update to complete. If false, updates are synchronous and block until completed. |
| 111 | +* `fast_serialization`: If true and pretty is false, uses the `simd-json` crate for faster serialization. This is only effective when saving in compact mode. |
| 112 | + |
| 113 | +## Examples 🧐 |
| 114 | + |
| 115 | +### Async Updates |
| 116 | + |
| 117 | +```rust |
| 118 | +use jsonmutexdb::JsonMutexDB; |
| 119 | +use serde_json::json; |
| 120 | +use std::thread; |
| 121 | +use std::sync::Arc; |
| 122 | +use std::time::Duration; |
| 123 | + |
| 124 | +# fn main() -> Result<(), Box<dyn std::error::Error>> { // Use Box<dyn Error> for example brevity |
| 125 | +let db_path = "async_example.json"; |
| 126 | +// Enable async updates, use fast compact saving |
| 127 | +let db = Arc::new(JsonMutexDB::new(db_path, false, true, true)?); |
| 128 | + |
| 129 | +let db_clone = Arc::clone(&db); |
| 130 | +thread::spawn(move || { |
| 131 | + println!("Background thread updating..."); |
| 132 | + db_clone.update(|data| { |
| 133 | + let obj = data.as_object_mut().unwrap(); |
| 134 | + obj.insert("worker_id".to_string(), json!(123)); |
| 135 | + obj.insert("status".to_string(), json!("running")); |
| 136 | + }).expect("Failed to send update"); |
| 137 | + println!("Background thread update sent."); |
| 138 | +}); |
| 139 | + |
| 140 | +// Give the background thread a moment to process |
| 141 | +thread::sleep(Duration::from_millis(50)); |
| 142 | + |
| 143 | +// Get the latest state (will block briefly to query background thread) |
| 144 | +let current_state = db.get()?; |
| 145 | +println!("State after async update: {}", current_state); |
| 146 | +assert_eq!(current_state["status"], "running"); |
| 147 | + |
| 148 | +db.save_sync()?; // Save the state fetched from background |
| 149 | +println!("Async state saved."); |
| 150 | + |
| 151 | +// Required: Drop the Arc to signal background thread shutdown before cleanup |
| 152 | +drop(db); |
| 153 | +thread::sleep(Duration::from_millis(50)); // Allow time for shutdown/final save |
| 154 | + |
| 155 | +std::fs::remove_file(db_path).ok(); |
| 156 | +# Ok(()) |
| 157 | +# } |
| 158 | +# mod jsonmutexdb { pub use crate::{JsonMutexDB, DbError}; } // Shim for example |
| 159 | +# use jsonmutexdb::{JsonMutexDB, DbError}; // Shim for example |
| 160 | +``` |
| 161 | + |
| 162 | +### Async Saving |
| 163 | + |
| 164 | +```rust |
| 165 | +use jsonmutexdb::JsonMutexDB; |
| 166 | +use serde_json::json; |
| 167 | +use std::thread; |
| 168 | +use std::time::Duration; |
| 169 | + |
| 170 | +# fn main() -> Result<(), Box<dyn std::error::Error>> { |
| 171 | +let db_path = "async_save_example.json"; |
| 172 | +// Sync updates, pretty printing |
| 173 | +let db = JsonMutexDB::new(db_path, true, false, false)?; |
| 174 | + |
| 175 | +db.update(|d| *d = json!({"message": "Hello from async save!"}))?; |
| 176 | + |
| 177 | +println!("Triggering async save..."); |
| 178 | +db.save_async()?; // Returns immediately |
| 179 | + |
| 180 | +println!("Main thread doing other work..."); |
| 181 | +// In a real app, the main thread continues here. |
| 182 | +// We sleep just to allow the save to likely complete for the example. |
| 183 | +thread::sleep(Duration::from_millis(100)); |
| 184 | + |
| 185 | +println!("Checking file..."); |
| 186 | +let content = std::fs::read_to_string(db_path)?; |
| 187 | +println!("File content:\n{}", content); |
| 188 | +assert!(content.contains(" \"message\":")); // Check for pretty printing |
| 189 | + |
| 190 | +std::fs::remove_file(db_path).ok(); |
| 191 | +# Ok(()) |
| 192 | +# } |
| 193 | +# mod jsonmutexdb { pub use crate::{JsonMutexDB, DbError}; } // Shim for example |
| 194 | +# use jsonmutexdb::{JsonMutexDB, DbError}; // Shim for example |
| 195 | +``` |
| 196 | + |
| 197 | +## Performance Notes ⚡️ |
| 198 | + |
| 199 | +* Serialization: `simd-json` (fast_serialization: true) can significantly speed up saving compact JSON. Pretty printing will always use `serde_json`. |
| 200 | +* I/O: Saves use `std::io::BufWriter` to minimize system calls. Atomic saves add the overhead of writing to a temporary file and renaming. `save_async` offloads serialization and I/O but involves thread spawning and fetching state (if async updates are on). |
| 201 | +* Async Updates: Enabling async_updates reduces blocking on update calls but adds overhead for channel communication on get and save_sync to maintain consistency. Choose based on whether update latency or get/save latency is more critical. |
| 202 | + |
| 203 | +## Error Handling ⚠️ |
| 204 | + |
| 205 | +Most operations return `Result<_, DbError>`. This enum covers: |
| 206 | +* `DbError::Io(std::io::Error)`: Filesystem errors, invalid JSON loading, serialization errors. |
| 207 | +* `DbError::Sync(String)`: Errors related to the async background thread communication (channel errors, poisoned mutexes in sync mode). |
| 208 | +Match on the result or use ? to propagate errors. |
| 209 | + |
| 210 | +## Limitations & Considerations 🤔 |
| 211 | + |
| 212 | +Single File: This manages one JSON file. It's not designed for complex relational data or large datasets where a real database would be more appropriate. |
| 213 | +Memory Usage: The entire JSON structure is loaded into memory. Very large JSON files might consume significant RAM. |
| 214 | +Async Mode Latency: While async_updates: true makes update() non-blocking, get() and save_sync() do block while communicating with the background thread to retrieve the latest state. |
| 215 | +unsafe: Uses unsafe internally for simd-json's from_str for performance. While believed to be safe in this context, be aware if auditing for unsafe. |
| 216 | + |
| 217 | +## Contributing 🤝 |
| 218 | + |
| 219 | +Welcome! |
| 220 | + |
| 221 | +## Happy JSON juggling! |
| 222 | + |
| 223 | +🎉 |
0 commit comments