diff --git a/.gitignore b/.gitignore index e48acbda..7ea683a4 100644 --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,7 @@ content/commands/* !content/commands/_index.md content/topics/* !content/topics/_index.md +!content/topics/faq/ _site _data/groups.json _data/resp2_replies.json diff --git a/README.md b/README.md index 8b0c4824..e5d16cd9 100644 --- a/README.md +++ b/README.md @@ -61,6 +61,8 @@ Point your browser at `http://127.0.0.1:1111/topics/` and you should see the ful All files created in this process are ignored by git. Commit your changes to your local copy of `valkey-io/valkey-doc`. +**Note:** The FAQ content is managed locally in this repo under `content/topics/faq/` and will not be overwritten by the build script. Each FAQ item is stored as an individual markdown file with proper front matter for easy editing and maintenance. + ### Building the command reference The command reference (i.e. `/commands/set/`, `/commands/get/`, `/commands/lolwut/`) sources information from `valkey-io/valkey`, `valkey-io/valkey-bloom`, and `valkey-io/valkey-doc`. diff --git a/build/init-topics-and-clients.sh b/build/init-topics-and-clients.sh index de02f0c5..0af77a2b 100755 --- a/build/init-topics-and-clients.sh +++ b/build/init-topics-and-clients.sh @@ -40,6 +40,8 @@ do base=${fname##*/} topic=${base%.*} + + if [[ "$topic" != "index" ]]; then if [ -f "./build/custom-frontmatter/topics/$topic.toml" ]; then echo "+++" >> "./content/topics/$topic.md" diff --git a/content/topics/faq.md b/content/topics/faq.md new file mode 100644 index 00000000..cc04d88d --- /dev/null +++ b/content/topics/faq.md @@ -0,0 +1,4 @@ ++++ +title = "FAQ" +template = "faq.html" ++++ diff --git a/content/topics/faq/_index.md b/content/topics/faq/_index.md new file mode 100644 index 00000000..db3658ec --- /dev/null +++ b/content/topics/faq/_index.md @@ -0,0 +1,4 @@ ++++ +title = "FAQ" +render = false ++++ \ No newline at end of file diff --git a/content/topics/faq/background-saving-fork.md b/content/topics/faq/background-saving-fork.md new file mode 100644 index 00000000..0113ec69 --- /dev/null +++ b/content/topics/faq/background-saving-fork.md @@ -0,0 +1,19 @@ ++++ +title = "Background saving fails with a fork() error on Linux?" +weight = 1 +[extra] +question = "Background saving fails with a fork() error on Linux?" +category = "technical" ++++ + +Short answer: `echo 1 > /proc/sys/vm/overcommit_memory` :) + +And now the long one: + +The Valkey background saving schema relies on the copy-on-write semantic of the `fork` system call in modern operating systems: Valkey forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will _share_ the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the `overcommit_memory` setting is set to zero the fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages. If you have a Valkey dataset of 3 GB and just 2 GB of free memory it will fail. + +Setting `overcommit_memory` to 1 tells Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Valkey. + +You can refer to the [proc(5)][proc5] man page for explanations of the available values. + +[proc5]: https://man7.org/linux/man-pages/man5/proc.5.html diff --git a/content/topics/faq/disk-based-database.md b/content/topics/faq/disk-based-database.md new file mode 100644 index 00000000..85d46139 --- /dev/null +++ b/content/topics/faq/disk-based-database.md @@ -0,0 +1,9 @@ ++++ +title = "Can you use Valkey with a disk-based database?" +weight = 3 +[extra] +question = "Can you use Valkey with a disk-based database?" +category = "general" ++++ + +Yes, a common design pattern involves taking very write-heavy small data in Valkey (and data you need the Valkey data structures to model your problem in an efficient way), and big *blobs* of data into an SQL or eventually consistent on-disk database. Similarly sometimes Valkey is used in order to take in memory another copy of a subset of the same data stored in the on-disk database. This may look similar to caching, but actually is a more advanced model since normally the Valkey dataset is updated together with the on-disk DB dataset, and not refreshed on cache misses. diff --git a/content/topics/faq/disk-space-out.md b/content/topics/faq/disk-space-out.md new file mode 100644 index 00000000..e3990de6 --- /dev/null +++ b/content/topics/faq/disk-space-out.md @@ -0,0 +1,9 @@ ++++ +title = "What happens when Valkey runs out of disk space?" +weight = 4 +[extra] +question = "What happens when Valkey runs out of disk space?" +category = "limitations" ++++ + +When Valkey runs out of disk space, it will stop accepting write commands and return errors. The server will continue to serve read commands but won't be able to persist data to disk. It's important to monitor disk space and implement proper alerting to prevent this situation. diff --git a/content/topics/faq/entire-dataset-memory.md b/content/topics/faq/entire-dataset-memory.md new file mode 100644 index 00000000..05fa8a02 --- /dev/null +++ b/content/topics/faq/entire-dataset-memory.md @@ -0,0 +1,12 @@ ++++ +title = "Why does Valkey keep its entire dataset in memory?" +weight = 2 + +[extra] +question = "Why does Valkey keep its entire dataset in memory?" +category = "memory-performance" ++++ + +In the past, developers experimented with Virtual Memory and other systems in order to allow larger than RAM datasets, but after all we are very happy if we can do one thing well: data served from memory, disk used for storage. So for now there are no plans to create an on disk backend for Valkey. Most of what Valkey is, after all, a direct result of its current design. + +If your real problem is not the total RAM needed, but the fact that you need to split your data set into multiple Valkey instances, please read the [partitioning page](/topics/cluster-tutorial/) in this documentation for more info. diff --git a/content/topics/faq/how-different-from-redis.md b/content/topics/faq/how-different-from-redis.md new file mode 100644 index 00000000..20662ccf --- /dev/null +++ b/content/topics/faq/how-different-from-redis.md @@ -0,0 +1,12 @@ ++++ +title = "How is Valkey different from other key-value stores?" +weight = 2 + +[extra] +question = "How is Valkey different from other key-value stores?" +category = "general" ++++ + +Valkey has a different evolution path in the key-value DBs where values can contain more complex data types, with atomic operations defined on those data types. Valkey data types are closely related to fundamental data structures and are exposed to the programmer as such, without additional abstraction layers. + +Valkey is an in-memory but persistent on disk database, so it represents a different trade off where very high write and read speed is achieved with the limitation of data sets that can't be larger than memory. Another advantage of in-memory databases is that the memory representation of complex data structures is much simpler to manipulate compared to the same data structures on disk, so Valkey can do a lot with little internal complexity. diff --git a/content/topics/faq/linux-foundation.md b/content/topics/faq/linux-foundation.md new file mode 100644 index 00000000..1f70fabe --- /dev/null +++ b/content/topics/faq/linux-foundation.md @@ -0,0 +1,11 @@ ++++ +title = "Why did Linux Foundation start the Valkey project?" +weight = 4 +[extra] +question = "Why did Linux Foundation start the Valkey project?" +category = "general" ++++ + +Valkey was created as a fork of Redis after Redis changed its licensing model. The Linux Foundation started the Valkey project to ensure the continued availability of an open-source, community-driven in-memory data store that maintains the original Redis API compatibility while being developed under open-source principles. + +Read about [the history of Valkey](/topics/history/). diff --git a/content/topics/faq/main-limitations.md b/content/topics/faq/main-limitations.md new file mode 100644 index 00000000..b0d18db7 --- /dev/null +++ b/content/topics/faq/main-limitations.md @@ -0,0 +1,15 @@ ++++ +title = "What are the main limitations of Valkey?" +weight = 2 +[extra] +question = "What are the main limitations of Valkey?" +category = "limitations" ++++ + +The main limitations of Valkey include: + +* **Memory-based storage**: All data must fit in RAM, which limits the total dataset size to available memory. +* **Single-threaded command execution**: While I/O threading is available, commands are executed sequentially in the main thread. +* **No built-in authentication**: Authentication must be implemented at the application level or through network security. +* **Limited query capabilities**: Unlike SQL databases, Valkey doesn't support complex queries or joins. +* **No built-in backup**: Backup and recovery must be implemented using external tools or scripts. diff --git a/content/topics/faq/max-keys-limits.md b/content/topics/faq/max-keys-limits.md new file mode 100644 index 00000000..65d7b668 --- /dev/null +++ b/content/topics/faq/max-keys-limits.md @@ -0,0 +1,13 @@ ++++ +title = "What is the maximum number of keys a single Valkey instance can hold? What is the maximum number of elements in a Hash, List, Set, and Sorted Set?" +weight = 1 +[extra] +question = "What is the maximum number of keys a single Valkey instance can hold? What is the maximum number of elements in a Hash, List, Set, and Sorted Set?" +category = "limitations" ++++ + +Valkey can handle up to 232 keys, and was tested in practice to handle at least 250 million keys per instance. + +Every hash, list, set, and sorted set, can hold 232 elements. + +In other words your limit is likely the available memory in your system. diff --git a/content/topics/faq/memory-footprint.md b/content/topics/faq/memory-footprint.md new file mode 100644 index 00000000..f60354af --- /dev/null +++ b/content/topics/faq/memory-footprint.md @@ -0,0 +1,16 @@ ++++ +title = "What's the Valkey memory footprint?" +weight = 1 + +[extra] +question = "What's the Valkey memory footprint?" +category = "memory-performance" ++++ + +To give you a few examples: + +* An empty instance uses ~ 3MB of memory. +* 1 Million small Keys -> String Value pairs use ~ 85MB of memory. +* 1 Million Keys -> Hash value, representing an object with 5 fields, use ~ 160 MB of memory. + +Testing your use case is trivial. Use the `valkey-benchmark` utility to generate random data sets then check the space used with the `INFO memory` command. diff --git a/content/topics/faq/multiple-cpus-cores.md b/content/topics/faq/multiple-cpus-cores.md new file mode 100644 index 00000000..1cdb0ce8 --- /dev/null +++ b/content/topics/faq/multiple-cpus-cores.md @@ -0,0 +1,11 @@ ++++ +title = "How can Valkey use multiple CPUs or cores?" +weight = 5 +[extra] +question = "How can Valkey use multiple CPUs or cores?" +category = "memory-performance" ++++ + +Enable I/O threading to offload client communication to threads. In Valkey 8, the I/O threading implementation has been rewritten and greatly improved. Reading commands from clients and writing replies back uses considerable CPU time. By offloading this work to separate threads, the main thread can focus on executing commands. + +You can also start multiple instances of Valkey in the same box and combine them into a [cluster](/topics/cluster-tutorial/). diff --git a/content/topics/faq/on-disk-snapshots.md b/content/topics/faq/on-disk-snapshots.md new file mode 100644 index 00000000..503272a2 --- /dev/null +++ b/content/topics/faq/on-disk-snapshots.md @@ -0,0 +1,9 @@ ++++ +title = "Are Valkey on-disk snapshots atomic?" +weight = 2 +[extra] +question = "Are Valkey on-disk snapshots atomic?" +category = "technical" ++++ + +Yes, the Valkey background saving process is always forked when the server is outside of the execution of a command, so every command reported to be atomic in RAM is also atomic from the point of view of the disk snapshot. diff --git a/content/topics/faq/reduce-memory-usage.md b/content/topics/faq/reduce-memory-usage.md new file mode 100644 index 00000000..c7c218b8 --- /dev/null +++ b/content/topics/faq/reduce-memory-usage.md @@ -0,0 +1,11 @@ ++++ +title = "How can I reduce Valkey's overall memory usage?" +weight = 3 +[extra] +question = "How can I reduce Valkey's overall memory usage?" +category = "memory-performance" ++++ + +A good practice is to consider memory consumption when mapping your logical data model to the physical data model within Valkey. These considerations include using specific data types, key patterns, and normalization. + +Beyond data modeling, there is more info in the [Memory Optimization page](/topics/memory-optimization). diff --git a/content/topics/faq/replica-keys-difference.md b/content/topics/faq/replica-keys-difference.md new file mode 100644 index 00000000..93e929f1 --- /dev/null +++ b/content/topics/faq/replica-keys-difference.md @@ -0,0 +1,16 @@ ++++ +title = "Why does my replica have a different number of keys than its primary instance?" +weight = 3 +[extra] +question = "Why does my replica have a different number of keys than its primary instance?" +category = "technical" ++++ + +If you use keys with limited time to live (Valkey expires) this is normal behavior. This is what happens: + +* The primary generates an RDB file on the first synchronization with the replica. +* The RDB file will not include keys already expired in the primary but which are still in memory. +* These keys are still in the memory of the Valkey primary, even if logically expired. They'll be considered non-existent, and their memory will be reclaimed later, either incrementally or explicitly on access. While these keys are not logically part of the dataset, they are accounted for in the `INFO` output and in the `DBSIZE` command. +* When the replica reads the RDB file generated by the primary, this set of keys will not be loaded. + +Because of this, it's common for users with many expired keys to see fewer keys in the replicas. However, logically, the primary and replica will have the same content. diff --git a/content/topics/faq/runs-out-of-memory.md b/content/topics/faq/runs-out-of-memory.md new file mode 100644 index 00000000..7752f2eb --- /dev/null +++ b/content/topics/faq/runs-out-of-memory.md @@ -0,0 +1,11 @@ ++++ +title = "What happens if Valkey runs out of memory?" +weight = 4 +[extra] +question = "What happens if Valkey runs out of memory?" +category = "memory-performance" ++++ + +Valkey has built-in protections allowing the users to set a max limit on memory usage, using the `maxmemory` option in the configuration file to put a limit to the memory Valkey can use. If this limit is reached, Valkey will start to reply with an error to write commands (but will continue to accept read-only commands). + +You can also configure Valkey to evict keys when the max memory limit is reached. See the [eviction policy docs](/topics/lru-cache/) for more information on this. diff --git a/content/topics/faq/transactions-support.md b/content/topics/faq/transactions-support.md new file mode 100644 index 00000000..2ec3998a --- /dev/null +++ b/content/topics/faq/transactions-support.md @@ -0,0 +1,9 @@ ++++ +title = "Can Valkey handle transactions?" +weight = 3 +[extra] +question = "Can Valkey handle transactions?" +category = "limitations" ++++ + +Yes, Valkey supports transactions through the `MULTI`, `EXEC`, `DISCARD`, and `WATCH` commands. However, these are not ACID transactions in the traditional database sense. Valkey transactions provide atomicity (all commands in a transaction are executed or none are) but not isolation (other clients can see intermediate states). diff --git a/content/topics/faq/what-is-valkey.md b/content/topics/faq/what-is-valkey.md new file mode 100644 index 00000000..1773c928 --- /dev/null +++ b/content/topics/faq/what-is-valkey.md @@ -0,0 +1,10 @@ ++++ +title = "What is Valkey?" +weight = 1 + +[extra] +question = "What is Valkey?" +category = "general" ++++ + +Valkey is an open-source, in-memory data structure store that can be used as a database, cache, and message broker. It supports various data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries, and streams. diff --git a/sass/_valkey.scss b/sass/_valkey.scss index 30dca7b7..feae9de2 100644 --- a/sass/_valkey.scss +++ b/sass/_valkey.scss @@ -1949,6 +1949,221 @@ pre table { margin-right: 0.3rem; } +.help-links { + margin-bottom: 2rem; + + li { + margin: 0.5rem 0; + } + + a { + display: flex; + align-items: center; + padding: 0.5rem 0; + font-weight: 500; + } +} + +.edit-box { + text-align: left; + font-size: 1.6rem; + + span { + color: #000; + } + + .edit-link { + color: #6983ff; + + &:hover { + text-decoration: none; + } + } +} + +.review-notice { + background: #fff3cd; + border: 1px solid #ffeaa7; + border-radius: 8px; + padding: 1.5rem; + margin-bottom: 2rem; + color: #856404; + + p { + margin: 0; + font-size: 1.4rem; + } +} + +.faq-aside { + @include respond-min(1024px) { + flex-direction: row; + } +} + +.faq-container { + max-width: 100%; + background-color: #fff; + padding: 2rem 2rem 0.1rem; +} + +.faq-container h2 { + font-size: 16px; + font-weight: 700; + color: #30176E; + margin: 0 0 0.25rem; +} + + +/* FAQ Section Containers */ +.faq-section { + background: rgba(104, 147, 255, 0.1); + margin-bottom: 2rem; + padding: 2rem; +} + +.faq-item { + margin: 0 0 1rem; + + &:last-child { + margin: 0; + } + + h3 { + font-size: 16px; + background:rgba(104, 147, 255, 0.1); + font-weight: 600; + cursor: pointer; + margin: 0 0 1px; + padding: 1.25rem; + position: relative; + transition: all 0.2s ease; + color: #1A2026; + + &:hover { + background:rgba(58, 67, 88, 0.1); + } + + .faq-caret { + position: absolute; + right: 1rem; + top: 50%; + transform: translateY(-50%); + width: 12px; + height: 12px; + transition: transform 0.2s ease; + opacity: 0.7; + + &:hover { + opacity: 1; + } + } + } + + .faq-answer { + padding: 1rem; + font-size: 16px; + } + + &.open { + + h3, + .faq-answer { + background: #fff; + } + + h3 .faq-caret { + transform: translateY(-50%) rotate(180deg); + } + } + + p, + ul { + font-size: 16px; + background-color: #fff; + padding: 1rem 2rem; + margin: 0; + } + + ul, ol { + margin: 0; + padding-left: 3.5rem; + } + + li { + margin-bottom: 0.8rem; + line-height: 1.6; + } + + code { + background-color: #f1f3f4; + padding: 0.2rem 0.4rem; + border-radius: 4px; + font-family: 'Fira Mono', monospace; + font-size: 1.4rem; + color: #d63384; + } + + pre { + background-color: #f8f9fa; + padding: 1.6rem; + border-radius: 4px; + overflow-x: auto; + margin: 1.6rem 0; + border: 1px solid #e1e5e9; + } + + pre code { + background: none; + padding: 0; + color: #fff; + } + + a { + color: #30176E; + text-decoration: underline; + } + + a:hover { + text-decoration: underline; + } + + ul, + ol, + p { + padding-top: 0; + } +} + +/* Category navigation */ +.category-nav { + list-style: none; + padding: 0; + margin: 0; +} + +.category-nav li { + margin-bottom: 8px; +} + +.category-link { + color: #666; + text-decoration: none; + font-weight: 500; + transition: color 0.2s ease; + display: block; + padding: 8px 0; +} + +.category-link:hover { + color: #30176E; +} + +.category-link.active { + color: #30176E; + font-weight: 600; +} + .breadcrumbs { border-radius: 20px; background: #fff; @@ -1975,4 +2190,5 @@ pre table { } } } -} \ No newline at end of file +} + diff --git a/static/img/icon-caret-down.svg b/static/img/icon-caret-down.svg new file mode 100644 index 00000000..096e4868 --- /dev/null +++ b/static/img/icon-caret-down.svg @@ -0,0 +1,3 @@ + + + \ No newline at end of file diff --git a/templates/faq.html b/templates/faq.html new file mode 100644 index 00000000..77bc4589 --- /dev/null +++ b/templates/faq.html @@ -0,0 +1,120 @@ +{% extends "default.html" %} + +{%- block head -%} +{%- endblock head -%} + +{% block content %} +
+

Frequently Asked Questions

+
+ +
+
+ + +
+
+
+ {% set faq_section = get_section(path="topics/faq/_index.md") %} + + +
+

General

+ {% for page in faq_section.pages | filter(attribute="extra.category", value="general") | sort(attribute="title") %} +
+

{{ page.extra.question }}

+
+ {{ page.content | safe }} +
+
+ {% endfor %} +
+ + +
+

Memory & Performance

+ {% for page in faq_section.pages | filter(attribute="extra.category", value="memory-performance") | sort(attribute="title") %} +
+

{{ page.extra.question }}

+
+ {{ page.content | safe }} +
+
+ {% endfor %} +
+ + +
+

Technical

+ {% for page in faq_section.pages | filter(attribute="extra.category", value="technical") | sort(attribute="title") %} +
+

{{ page.extra.question }}

+
+ {{ page.content | safe }} +
+
+ {% endfor %} +
+ + +
+

Limitations

+ {% for page in faq_section.pages | filter(attribute="extra.category", value="limitations") | sort(attribute="title") %} +
+

{{ page.extra.question }}

+
+ {{ page.content | safe }} +
+
+ {% endfor %} +
+
+
+
+
+
+ + +{% endblock content %} \ No newline at end of file diff --git a/templates/shortcodes/faq_section.html b/templates/shortcodes/faq_section.html new file mode 100644 index 00000000..aaeb3bae --- /dev/null +++ b/templates/shortcodes/faq_section.html @@ -0,0 +1,3 @@ +
+ {{body | markdown | safe}} +
\ No newline at end of file