diff --git a/rag-with-dockling/README.md b/rag-with-dockling/README.md index 4c37f99b7..f7b8da82b 100644 --- a/rag-with-dockling/README.md +++ b/rag-with-dockling/README.md @@ -19,6 +19,31 @@ This project leverages LlamaIndex and IBM's Docling for RAG over excel sheets. Y --- +## 🐞 Issue You may face +### 1. Pickle Error with `@st.cache_resource` +The app might fail with: `An error occurred: cannot pickle 'classmethod' object` + +> βœ… **Solution**: Don’t cache non-picklable objects like the Ollama client or embeddings. Instead, store them in Streamlit's session state: +> ```python +> if "llm_client" not in st.session_state: +> st.session_state.llm_client = Ollama(model="llama3.2") +> ``` + +### 2. App Extremely Slow (incase Used another Large Model) +This often happens due to high memory usage. `ollama` was observed using 10+ GB RAM on an 8 GB Mac, leading to heavy swapping. Large embedding models like `bge-large-en-v1.5` also consume significant memory. + +> βœ… **Solution**: +> * **Use smaller Ollama models**: +> * `qwen2:1.5b` +> * `llama3.2:1b` +> * `mistral:7b-instruct-q4_K_M` +> * **Use smaller embeddings**: +> ```python +> HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5") +> ``` + +--- + ## πŸ“¬ Stay Updated with Our Newsletter! **Get a FREE Data Science eBook** πŸ“– with 150+ essential lessons in Data Science when you subscribe to our newsletter! Stay in the loop with the latest tutorials, insights, and exclusive resources. [Subscribe now!](https://join.dailydoseofds.com)