Skip to content

Fix: defined default var for langchain cache path to enable custom path (issue 987) #982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Fix: defined default var for langchain cache path to enable custom path (issue 987) #982

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5520569
defined default var for langchain cache path to enable custom path
jetmicham May 9, 2024
c153257
fixed log info notification upon lc cach creation
jetmicham May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/chainlit/cache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def init_lc_cache():
if config.project.lc_cache_path is not None:
set_llm_cache(SQLiteCache(database_path=config.project.lc_cache_path))

if not os.path.exists(config.project.lc_cache_path):
if os.path.exists(config.project.lc_cache_path):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to understand why you've removed this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is because previous line st_llm_cache generates sqlite db file at path

set_llm_cache(SQLiteCache(database_path=config.project.lc_cache_path))

consequently in next line we want to inform cache was generated by testing if path exists and not the opposite. I hope I was clear - was trying not to use too many "if", "not" in the explaination

logger.info(
f"LangChain cache created at: {config.project.lc_cache_path}"
)
Expand Down
15 changes: 8 additions & 7 deletions backend/chainlit/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
# Enable third parties caching (e.g LangChain cache)
cache = false

# Set langchain cache path (default is .langchain.db inside config dir)
# lc_cache_path = './.my-cache-path.db'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a default config option, it seems better to uncomment it, in line with other default config options.


# Authorized origins
allow_origins = ["*"]

Expand Down Expand Up @@ -268,9 +271,9 @@ class CodeSettings:

author_rename: Optional[Callable[[str], str]] = None
on_settings_update: Optional[Callable[[Dict[str, Any]], Any]] = None
set_chat_profiles: Optional[Callable[[Optional["User"]], List["ChatProfile"]]] = (
None
)
set_chat_profiles: Optional[
Callable[[Optional["User"]], List["ChatProfile"]]
] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this relevant to the issue at hand?



@dataclass()
Expand All @@ -280,7 +283,8 @@ class ProjectSettings(DataClassJsonMixin):
# List of environment variables to be provided by each user to use the app. If empty, no environment variables will be asked to the user.
user_env: Optional[List[str]] = None
# Path to the local langchain cache database
lc_cache_path: Optional[str] = None
# default to "{config_dir}/.langchain.db"
lc_cache_path: Optional[str] = os.path.join(config_dir, ".langchain.db")
# Path to the local chat db
# Duration (in seconds) during which the session is saved when the connection is lost
session_timeout: int = 3600
Expand Down Expand Up @@ -420,10 +424,7 @@ def load_settings():
"Your config file is outdated. Please delete it and restart the app to regenerate it."
)

lc_cache_path = os.path.join(config_dir, ".langchain.db")

project_settings = ProjectSettings(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you're missing something to change the project_config["lc_cache_path"] value, like:

Suggested change
project_settings = ProjectSettings(
project_config["lc_cache_path"] = project_config.get("lc_cache_path", lc_cache_path)
project_settings = ProjectSettings(

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way i understand it and tested it locally, possible custom path will be picked up by toml_dict module here:

project_config = toml_dict.get("project", {})

or will default to default value as proposed by my PR here

# default to "{config_dir}/.langchain.db"
lc_cache_path: Optional[str] = os.path.join(config_dir, ".langchain.db")

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @necjamahim is right on this one. Could you please have another look @tpatel?

lc_cache_path=lc_cache_path,
**project_config,
)

Expand Down