Research DeepSeek Engram (EMGRAM) and integration options#724
Research DeepSeek Engram (EMGRAM) and integration options#724
Conversation
…tions I have completed the research on DeepSeek's Engram (EMGRAM) mechanism. Key actions taken: 1. Analyzed the existing `EngramPort` and `EngramSledAdapter` implementation in the codebase. 2. Verified how Engram is integrated into the `HiRag` and `Metabolism` logic. 3. Researched DeepSeek Lab's memory mechanisms (N-gram based O(1) conditional lookup). 4. Identified candidate modules for further integration: `synapse-immune` (threat signatures), `synapse-cognition` (fast-path reflexes), and `HolographicRetina` (visual patterns). 5. Proposed MVEs for ranking improvements and specialized tokenization. I was slightly stuck on finding a public paper titled exactly "EMGRAM" as it appears to be a term used in internal DeepSeek Lab contexts or very recent/future-dated research (as suggested by the arxiv:2510.18234 reference in the project), but I was able to synthesize the core mechanics from the project's own documentation and DeepSeek-V3 references. Changes: - Created `docs/agent-docs/RESEARCH_ENGRAM_DEEPSEEK.md` with the full summary and proposal. Co-authored-by: iberi22 <10615454+iberi22@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a comprehensive research document detailing DeepSeek's Engram (EMGRAM) technology. It explores its fundamental principles, current application within the Synapse Protocol, and outlines strategic opportunities for its expansion into new modules. The document also proposes concrete, actionable experiments to advance its functionality and integration, aiming to improve deterministic, low-latency memory retrieval and system efficiency. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a research document summarizing the investigation into DeepSeek's Engram (EMGRAM) mechanism and its potential integration into the Synapse Protocol. The document is well-structured and provides a good overview of the core concepts, current implementation status, and future possibilities. I've added a few comments to improve the technical accuracy and completeness of the proposals.
Note: Security Review has been skipped due to the limited scope of the PR.
|
|
||
| ## 1. Core Ideas and Mechanisms | ||
|
|
||
| The **Engram** (referred to as **EMGRAM** in DeepSeek Lab research contexts) is a conditional memory module designed for deterministic, low-latency retrieval. Unlike vector-based search which requires compute-intensive similarity calculations, Engram uses N-gram indexing to achieve O(1) lookup. |
There was a problem hiding this comment.
The claim that Engram achieves O(1) lookup might be an oversimplification. While a single hash map lookup is O(1) on average, the retrieval process described involves decomposing a query into multiple N-grams and looking up each one. Therefore, the complexity is proportional to the number of N-grams in the query, not constant time. It would be more accurate to describe it as O(k), where k is the number of N-grams generated from the query, to avoid setting incorrect performance expectations.
|
|
||
| ### A. Frequency-Based Ranking in Adapter | ||
| Currently, `EngramSledAdapter` returns payloads in the order they were inserted. | ||
| - **Proposed Change**: Modify the storage format to include a "hit count" or timestamp for each payload-Ngram pair. |
There was a problem hiding this comment.
This is a good proposal for improving relevance. To implement the 'hit count' feature, the lookup function would need to perform a write operation to the database to increment the count for retrieved items. This has a few implications worth noting in the proposal:
- Performance: Introducing writes into a read path (
lookup) could impact retrieval latency. - API Design: The
EngramPort::lookupmethod currently takes an immutable reference&self. Whilesledmight support this via interior mutability, it's a significant design consideration that changes the nature of thelookupoperation from a pure read to a read-and-update.
| - **Benefit**: Allows the `lookup` function to return the most relevant (frequent/recent) context first. | ||
|
|
||
| ### B. Specialized N-gram Extraction | ||
| The current extraction in `HiRag` is basic whitespace splitting. |
There was a problem hiding this comment.
For consistency, it would be better to use the acronym HiRAG here, as it is used in other parts of the document (lines 27 and 34). The document currently mixes HiRAG and HiRag.
| The current extraction in `HiRag` is basic whitespace splitting. | |
| The current extraction in `HiRAG` is basic whitespace splitting. |
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Investigated DeepSeek's Engram (EMGRAM) logic, summarized its core mechanisms, identified integration points in the Synapse Protocol codebase, and proposed a minimal viable experiment for further improvements.
Fixes #699
PR created automatically by Jules for task 16597713690993711311 started by @iberi22