Memory #721
Memory #721
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @suluyana, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly upgrades the agent's ability to retain and utilize conversational context and tool interaction results. By introducing a new, robust memory component and refactoring its integration, the agent can now maintain a more comprehensive understanding of ongoing conversations and user preferences, leading to improved personalization and more coherent interactions over time.
Highlights
- Enhanced Agent Memory Management: Introduced a new DefaultMemory class for sophisticated conversation history and fact retrieval, leveraging the mem0 library.
- Refactored Memory Initialization: The _prepare_memory function in llm_agent.py now accepts messages and kwargs, and its integration into the agent's run cycle has been updated to handle message caching and conversation ID.
- New Prompt for Fact Retrieval: A detailed prompt (FACT_RETRIEVAL_PROMPT) has been added to guide the extraction of personal information and tool interaction outcomes from conversations.
- Deprecated Old Memory Placeholder: The ms_agent/agent/memory/mem0.py file has been removed, replaced by the new DefaultMemory implementation.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces a significant new feature: a memory component for the agent, leveraging the mem0 library. The core of this change is the new DefaultMemory class, which manages conversation history, supports retries through common prefix detection, and enriches the agent's context by injecting relevant memories into the system prompt. The agent's initialization flow has also been refactored to integrate this new memory system.
My review has identified a few critical issues that need addressing. Firstly, the new default_memory.py file is littered with debugging print statements which must be removed for production readiness. Secondly, there is a critical bug in prompts.py where datetime.now() is evaluated at module import time, which will cause the date in the prompt to become stale in long-running applications. I have also provided feedback on improving maintainability by addressing an inefficient rollback implementation, removing an unused import, translating a docstring for consistency, and relocating test code. Overall, this is a valuable addition, but it requires these fixes to be production-ready.
ms_agent/utils/prompts.py
Outdated
| FACT_RETRIEVAL_PROMPT = f"""You are a Personal Information Organizer, specialized in accurately storing facts, user memories, preferences, and processing tool interaction outcomes. Your primary role is to extract relevant pieces of information from conversations, organize them into distinct, manageable facts, and additionally process and summarize tool invocation results when present. This ensures both personal data and system interactions are captured for improved context retention and future personalization. | ||
| Types of Information to Remember: | ||
| 1. Store Personal Preferences: Keep track of likes, dislikes, and specific preferences in various categories such as food, products, activities, and entertainment. | ||
| 2. Maintain Important Personal Details: Remember significant personal information like names, relationships, and important dates. | ||
| 3. Track Plans and Intentions: Note upcoming events, trips, goals, and any plans the user has shared. | ||
| 4. Remember Activity and Service Preferences: Recall preferences for dining, travel, hobbies, and other services. | ||
| 5. Monitor Health and Wellness Preferences: Keep a record of dietary restrictions, fitness routines, and other wellness-related information. | ||
| 6. Store Professional Details: Remember job titles, work habits, career goals, and other professional information. | ||
| 7. Miscellaneous Information Management: Keep track of favorite books, movies, brands, and other miscellaneous details that the user shares. | ||
| Tool Interaction Processing Instructions (Additional Responsibilities): | ||
| When tool calls and their results are included in the conversation, perform the following in addition to fact extraction: | ||
| 1. Extract and Organize Factual Information from Tool Outputs: | ||
| - Parse the returned data from successful tool calls (e.g., weather, calendar, search, maps). | ||
| - Identify and store objective, user-relevant facts derived from these results (e.g., "It will rain in Paris on 2025-08-25", "The restaurant Little Italy is located at 123 Main St"). | ||
| - Integrate these into the "facts" list only if they reflect new, meaningful information about the user's context or environment. | ||
| 2. Analyze and Summarize Error-Prone Tools: | ||
| - Identify tools that frequently fail, time out, or return inconsistent results. | ||
| - For such tools, generate a brief internal summary noting the pattern of failure (e.g., "Search tool often returns incomplete results for restaurant queries"). | ||
| - This summary does not go into the JSON output but informs future handling (e.g., suggesting alternative tools or double-checking outputs). | ||
| 3. Identify and Log Tools That Cannot Be Called: | ||
| - If a tool was intended but not invoked (e.g., due to missing permissions, unavailability, or misconfiguration), note this in a separate internal log. | ||
| - Examples: "Calendar tool unavailable — cannot retrieve user's meeting schedule", "Location access denied — weather tool cannot auto-detect city". | ||
| - Include a user-facing reminder if relevant: add a fact like "Could not access calendar due to permission restrictions" only if it impacts user understanding. | ||
| 4. Ensure Clarity and Non-Disclosure: | ||
| - Do not expose tool names, system architecture, or internal logs in the output. | ||
| - If asked why information is missing, respond: "I tried to retrieve it from publicly available sources, but the information may not be accessible right now." | ||
| Here are some few-shot examples: | ||
| Input: Hi. | ||
| Output: {{"facts" : []}} | ||
| Input: There are branches in trees. | ||
| Output: {{"facts" : []}} | ||
| Input: Hi, I am looking for a restaurant in San Francisco. | ||
| Output: {{"facts" : ["Looking for a restaurant in San Francisco"]}} | ||
| Input: Yesterday, I had a meeting with John at 3pm. We discussed the new project. | ||
| Output: {{"facts" : ["Had a meeting with John at 3pm", "Discussed the new project"]}} | ||
| Input: Hi, my name is John. I am a software engineer. | ||
| Output: {{"facts" : ["Name is John", "Is a Software engineer"]}} | ||
| Input: My favourite movies are Inception and Interstellar. | ||
| Output: {{"facts" : ["Favourite movies are Inception and Interstellar"]}} | ||
| Input (with tool call): What's the weather like in Tokyo today? | ||
| [Tool Call: get_weather(location="Tokyo", date="2025-08-22") → Result: {{"status": "success", "data": {{"temp": 32°C, "condition": "Sunny", "humidity": 65%}}}}] | ||
| Output: {{"facts": ["It is 32°C and sunny in Tokyo today", "Humidity level in Tokyo is 65%"]}} | ||
| Input (with failed tool): Check my calendar for tomorrow's meetings. | ||
| [Tool Call: get_calendar(date="2025-08-23") → Failed: "Access denied – calendar not connected"] | ||
| Output: {{"facts": ["Could not access calendar due to connection issues"]}} | ||
| Input (with unreliable tool pattern): Search for vegan restaurants near Central Park. | ||
| [Tool Call: search(query="vegan restaurants near Central Park") → Returns incomplete/no results multiple times] | ||
| Output: {{"facts": ["Searching for vegan restaurants near Central Park yielded limited results"]}} | ||
| (Internal note: Search tool shows low reliability for location-based queries — consider fallback sources.) | ||
| Final Output Rules: | ||
| - Today's date is {datetime.now().strftime("%Y-%m-%d")}. | ||
| - If the user asks where you fetched my information, answer that you found from publicly available sources on internet. | ||
| - Return only a JSON object with key "facts" and value as a list of strings. | ||
| - Do not include anything from the example prompts or system instructions. | ||
| - Do not reveal tool usage, internal logs, or model behavior. | ||
| - If no relevant personal or environmental facts are found, return: {{"facts": []}} | ||
| - Extract facts only from user and assistant messages — ignore system-level instructions. | ||
| - Detect the input language and record facts in the same language. | ||
| Following is a conversation between the user and the assistant. You have to extract the relevant facts and preferences about the user, if any, from the conversation, process any tool call results, and return them in the JSON format as shown above. | ||
| """ |
There was a problem hiding this comment.
The FACT_RETRIEVAL_PROMPT is defined using an f-string that evaluates datetime.now() at module import time. This is a critical issue because the date will become stale if the application runs for more than 24 hours. To fix this, you should convert this constant into a function that generates the prompt with the current date each time it's called.
For example:
def get_fact_retrieval_prompt():
return f"""...Today's date is {datetime.now().strftime("%Y-%m-%d")}..."""You will also need to update the call site in ms_agent/agent/memory/default_memory.py to invoke this new function.
ms_agent/agent/llm_agent.py
Outdated
| return messages | ||
|
|
||
| async def _prepare_memory(self): | ||
| async def _prepare_memory(self, |
There was a problem hiding this comment.
Add function comments for key functions in the form of:
"""
Prepare memory ...
Args:
...
Returns:
...
Raises: (Optional)
....
"""
| @@ -1,2 +1,4 @@ | |||
| # Copyright (c) Alibaba, Inc. and its affiliates. | |||
| memory_mapping = {} | |||
| from .default_memory import DefaultMemory | |||
There was a problem hiding this comment.
No need to create a single file utils.py to handle this.
tests/memory/test_default_memory.py
Outdated
| @@ -0,0 +1,176 @@ | |||
| # Copyright (c) Alibaba, Inc. and its affiliates. | |||
| import math | |||
There was a problem hiding this comment.
Add en version for the test_default_memory
|
|
||
|
|
||
| def get_fact_retrieval_prompt(): | ||
| return f"""You are a Personal Information Organizer, specialized in accurately storing facts, user memories, preferences, and processing tool interaction outcomes. Your primary role is to extract relevant pieces of information from conversations, organize them into distinct, manageable facts, and additionally process and summarize tool invocation results when present. This ensures both personal data and system interactions are captured for improved context retention and future personalization. |
There was a problem hiding this comment.
prompt.py or prompts.py ?
Uh oh!
There was an error while loading. Please reload this page.