Skip to content

Commit 8b8fc05

Browse files
sfc-gh-cnantasenamatsfc-gh-cnantasenamat
authored
Merge pull request #1 from nerdai/nerdai/llama-index-agents-cookbook
Add llama-agents (llama-index) cookbook
2 parents 8be7e07 + f6d0205 commit 8b8fc05

29 files changed

Lines changed: 6086 additions & 0 deletions

‎recipes/llamaindex/.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.env.*
2+
.env
3+
index.html.*
4+
index.html
5+
task_results
6+
.ipynb_checkpoints/
7+
secrets.yaml
8+
Dockerfile.local
9+
docker-compose.local.yml
10+
pyproject.local.toml
11+
__pycache__
12+
data
13+
notebooks

‎recipes/llamaindex/Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
FROM --platform=linux/amd64 python:3.10-slim as builder
2+
3+
WORKDIR /app
4+
5+
ENV POETRY_VERSION=1.7.1
6+
7+
# Install libraries for necessary python package builds
8+
RUN apt-get update && apt-get --no-install-recommends install build-essential python3-dev libpq-dev -y && \
9+
pip install --no-cache-dir --upgrade pip && \
10+
pip install --no-cache-dir --upgrade poetry==${POETRY_VERSION}
11+
12+
# Install ssh
13+
RUN apt-get -yq update && apt-get -yqq install ssh
14+
15+
# Configure Poetry
16+
ENV POETRY_CACHE_DIR=/tmp/poetry_cache
17+
ENV POETRY_NO_INTERACTION=1
18+
ENV POETRY_VIRTUALENVS_IN_PROJECT=true
19+
ENV POETRY_VIRTUALENVS_CREATE=true
20+
21+
# Install dependencies
22+
COPY ./poetry.lock ./pyproject.toml ./
23+
24+
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
25+
26+
RUN poetry install --no-cache --no-root
27+
28+
FROM --platform=linux/amd64 python:3.10-slim as runtime
29+
30+
# Install wget for healthcheck
31+
RUN apt-get update && apt-get install -y wget
32+
33+
RUN apt-get update -y && \
34+
apt-get install --no-install-recommends libpq5 -y && \
35+
rm -rf /var/lib/apt/lists/* # Install libpq for psycopg2
36+
37+
RUN groupadd -r appuser && useradd --no-create-home -g appuser -r appuser
38+
USER appuser
39+
40+
WORKDIR /app
41+
42+
ENV VIRTUAL_ENV=/app/.venv
43+
COPY --from=builder ${VIRTUAL_ENV} ${VIRTUAL_ENV}
44+
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"
45+
46+
# Copy source code
47+
COPY ./logging.ini ./logging.ini
48+
COPY ./llamaindex_cookbook ./llamaindex_cookbook

‎recipes/llamaindex/README.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,72 @@
1+
# LlamaAgents Demo With Snowflake/Cybersyn Data Agents
12

3+
<img width="960" alt="image" src="https://github.com/user-attachments/assets/2d82ac2b-d37f-4b86-9867-69947402c924">
4+
5+
For this demo app, we have a multi-agent system comprised with the following
6+
components:
7+
8+
- A data **Agent** that performs queries over [Cybersyn's Financial & Economic Essentials](https://app.snowflake.com/marketplace/listing/GZTSZAS2KF7/cybersyn-financial-economic-essentials?originTab=provider&providerName=Cybersyn&profileGlobalName=GZTSZAS2KCS) Dataset
9+
- A data **Agent** that performs queries over [Cyberysyn' Government Essentials](https://app.snowflake.com/marketplace/listing/GZTSZAS2KGK/cybersyn-government-essentials?originTab=provider&providerName=Cybersyn&profileGlobalName=GZTSZAS2KCS) Dataset
10+
- A general **Agent** that can answer all general queries
11+
- A **Human (In the Loop) Service** that provides inputs to the two data agents
12+
- A **ControlPlane** that features a router which routes tasks to the most appropriate orchestrator
13+
- A **RabbitMQ MessageQueue** to broker communication between agents, human-in-the-loop, and control-plane
14+
15+
For the frontend, we built a Streamlit App to interact with this multi-agent
16+
system.
17+
18+
## Pre-Requisites
19+
20+
### Poetry
21+
22+
For this app, we use [Poetry](https://python-poetry.org/) as the package's
23+
dependency manager. The `poetry` cli tool is what we'll need to install the package's
24+
virtual environment in order to run our streamlit app.
25+
26+
### Docker
27+
28+
To run this demo, we make use of `Docker`, specifically `docker-compose`. For this
29+
demo, all of the necessary services (with the exception of the message queue)
30+
are packaged in one common Docker image and can be instantianted through their
31+
respective commands (i.e., see `docker-compose.yml`.)
32+
33+
### Credentials
34+
35+
For this app, we use OpenAI as the LLM provider and so an `OPENAI_API_KEY` will
36+
need to be supplied. Moreover, the Cybersyn data is pulled from Snowflake and so
37+
various Snowflake params are also required. See the section "# FILL-IN" in the
38+
`template.env.docker` file. Once, you've filled in the necessary environment
39+
variables, rename the file to `.env.docker`.
40+
41+
Similarly, you need to provide credentials in the `template.env.local`. Once
42+
filled in, rename the file to `.env.local`.
43+
44+
## Running The App
45+
46+
### The backend (multi-agent system)
47+
48+
To start the multi-agent system, use the following command while in the root of
49+
the project directory:
50+
51+
```sh
52+
docker-compose up --build
53+
```
54+
55+
### Streamlit App
56+
57+
Once the services are all running, you can then run the streamlit app. First,
58+
ensure that you have the package's virtual environment active and the environment
59+
variables set. Again, while in the root directory of this project, run the commands
60+
found below:
61+
62+
```sh
63+
poetry shell
64+
poetry install
65+
set -a && source .env.local
66+
```
67+
68+
Next, run the streamlit app:
69+
70+
```sh
71+
streamlit run llamaindex_cookbook/apps/streamlit.py
72+
```

‎recipes/llamaindex/docker-compose.yml‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
version: "3"
2+
services:
3+
rabbitmq:
4+
image: rabbitmq:3.13-management
5+
hostname: "rabbitmq"
6+
env_file:
7+
- .env.docker
8+
ports:
9+
- "5672:5672"
10+
- "15672:15672"
11+
volumes:
12+
- rabbitmq:/var/lib/rabbitmq/
13+
healthcheck:
14+
test: rabbitmq-diagnostics -q ping
15+
interval: 30s
16+
timeout: 10s
17+
retries: 5
18+
control_plane:
19+
image: llamaindex_cookbook:latest
20+
command: sh -c "python -m llamaindex_cookbook.core_services.control_plane"
21+
env_file:
22+
- .env.docker
23+
ports:
24+
- "8001:8001"
25+
volumes:
26+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
27+
- ./logging.ini:/app/logging.ini
28+
depends_on:
29+
rabbitmq:
30+
condition: service_healthy
31+
platform: linux/amd64
32+
build:
33+
context: .
34+
dockerfile: ./Dockerfile
35+
healthcheck:
36+
test: wget --no-verbose --tries=1 http://0.0.0.0:8001/ || exit 1
37+
interval: 30s
38+
retries: 5
39+
start_period: 20s
40+
timeout: 10s
41+
funny_agent:
42+
image: llamaindex_cookbook:latest
43+
command: sh -c "python -m llamaindex_cookbook.agent_services.funny_agent"
44+
env_file:
45+
- .env.docker
46+
ports:
47+
- "8002:8002"
48+
volumes:
49+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
50+
- ./data:/app/data
51+
- ./logging.ini:/app/logging.ini
52+
depends_on:
53+
rabbitmq:
54+
condition: service_healthy
55+
control_plane:
56+
condition: service_healthy
57+
platform: linux/amd64
58+
build:
59+
context: .
60+
dockerfile: ./Dockerfile
61+
healthcheck:
62+
test: wget --no-verbose --tries=1 http://0.0.0.0:8002/is_worker_running || exit 1
63+
interval: 30s
64+
retries: 5
65+
start_period: 20s
66+
timeout: 10s
67+
goods_getter_agent:
68+
image: llamaindex_cookbook:latest
69+
command: sh -c "python -m llamaindex_cookbook.agent_services.financial_and_economic_essentials.goods_getter_agent"
70+
env_file:
71+
- .env.docker
72+
ports:
73+
- "8003:8003"
74+
volumes:
75+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
76+
- ./data:/app/data
77+
- ./logging.ini:/app/logging.ini
78+
depends_on:
79+
rabbitmq:
80+
condition: service_healthy
81+
control_plane:
82+
condition: service_healthy
83+
platform: linux/amd64
84+
build:
85+
context: .
86+
dockerfile: ./Dockerfile
87+
healthcheck:
88+
test: wget --no-verbose --tries=1 http://0.0.0.0:8003/is_worker_running || exit 1
89+
interval: 30s
90+
retries: 5
91+
start_period: 20s
92+
timeout: 10s
93+
time_series_getter_agent:
94+
image: llamaindex_cookbook:latest
95+
command: sh -c "python -m llamaindex_cookbook.agent_services.financial_and_economic_essentials.time_series_getter_agent"
96+
env_file:
97+
- .env.docker
98+
ports:
99+
- "8004:8004"
100+
volumes:
101+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
102+
- ./data:/app/data
103+
- ./logging.ini:/app/logging.ini
104+
depends_on:
105+
rabbitmq:
106+
condition: service_healthy
107+
control_plane:
108+
condition: service_healthy
109+
platform: linux/amd64
110+
build:
111+
context: .
112+
dockerfile: ./Dockerfile
113+
healthcheck:
114+
test: wget --no-verbose --tries=1 http://0.0.0.0:8004/is_worker_running || exit 1
115+
interval: 30s
116+
retries: 5
117+
start_period: 20s
118+
timeout: 10s
119+
stats_getter_agent:
120+
image: llamaindex_cookbook:latest
121+
command: sh -c "python -m llamaindex_cookbook.agent_services.government_essentials.stats_getter_agent"
122+
env_file:
123+
- .env.docker
124+
ports:
125+
- "8005:8005"
126+
volumes:
127+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
128+
- ./data:/app/data
129+
- ./logging.ini:/app/logging.ini
130+
depends_on:
131+
rabbitmq:
132+
condition: service_healthy
133+
control_plane:
134+
condition: service_healthy
135+
platform: linux/amd64
136+
build:
137+
context: .
138+
dockerfile: ./Dockerfile
139+
healthcheck:
140+
test: wget --no-verbose --tries=1 http://0.0.0.0:8005/is_worker_running || exit 1
141+
interval: 30s
142+
retries: 5
143+
start_period: 20s
144+
timeout: 10s
145+
stats_fulfiller_agent:
146+
image: llamaindex_cookbook:latest
147+
command: sh -c "python -m llamaindex_cookbook.agent_services.government_essentials.stats_fulfiller_agent"
148+
env_file:
149+
- .env.docker
150+
ports:
151+
- "8006:8006"
152+
volumes:
153+
- ./llamaindex_cookbook:/app/llamaindex_cookbook # load local code change to container without the need of rebuild
154+
- ./data:/app/data
155+
- ./logging.ini:/app/logging.ini
156+
depends_on:
157+
rabbitmq:
158+
condition: service_healthy
159+
control_plane:
160+
condition: service_healthy
161+
platform: linux/amd64
162+
build:
163+
context: .
164+
dockerfile: ./Dockerfile
165+
healthcheck:
166+
test: wget --no-verbose --tries=1 http://0.0.0.0:8006/is_worker_running || exit 1
167+
interval: 30s
168+
retries: 5
169+
start_period: 20s
170+
timeout: 10s
171+
volumes:
172+
rabbitmq:

‎recipes/llamaindex/llamaindex_cookbook/__init__.py‎

Whitespace-only changes.

‎recipes/llamaindex/llamaindex_cookbook/additional_services/__init__.py‎

Whitespace-only changes.

‎recipes/llamaindex/llamaindex_cookbook/additional_services/human_in_the_loop.py‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
import asyncio
2+
import logging
3+
import queue
4+
from typing import Any, TypedDict
5+
6+
from llama_agents import HumanService, ServiceComponent
7+
from llama_agents.message_queues.rabbitmq import RabbitMQMessageQueue
8+
9+
from llamaindex_cookbook.utils import load_from_env
10+
11+
logger = logging.getLogger("llamaindex_cookbook")
12+
logging.basicConfig(level=logging.INFO)
13+
14+
message_queue_host = load_from_env("RABBITMQ_HOST")
15+
message_queue_port = load_from_env("RABBITMQ_NODE_PORT")
16+
message_queue_username = load_from_env("RABBITMQ_DEFAULT_USER")
17+
message_queue_password = load_from_env("RABBITMQ_DEFAULT_PASS")
18+
control_plane_host = load_from_env("CONTROL_PLANE_HOST")
19+
control_plane_port = load_from_env("CONTROL_PLANE_PORT")
20+
localhost = load_from_env("LOCALHOST")
21+
22+
23+
class HumanRequest(TypedDict):
24+
prompt: str
25+
task_id: str
26+
27+
28+
# # human in the loop function
29+
human_input_request_queue: queue.Queue[HumanRequest] = queue.Queue()
30+
human_input_result_queue: queue.Queue[str] = queue.Queue()
31+
32+
33+
async def human_input_fn(prompt: str, task_id: str, **kwargs: Any) -> str:
34+
logger.info("human input fn invoked.")
35+
human_input_request_queue.put({"prompt": prompt, "task_id": task_id})
36+
logger.info("placed new prompt in queue.")
37+
38+
# poll until human answer is stored
39+
async def _poll_for_human_input_result() -> str:
40+
human_input = None
41+
while human_input is None:
42+
try:
43+
human_input = human_input_result_queue.get_nowait()
44+
except queue.Empty:
45+
human_input = None
46+
await asyncio.sleep(0.1)
47+
logger.info("human input recieved")
48+
return human_input
49+
50+
try:
51+
human_input = await asyncio.wait_for(
52+
_poll_for_human_input_result(),
53+
timeout=6000,
54+
)
55+
logger.info(f"Recieved human input: {human_input}")
56+
except (
57+
asyncio.exceptions.TimeoutError,
58+
asyncio.TimeoutError,
59+
TimeoutError,
60+
):
61+
logger.info(f"Timeout reached for tool_call with prompt {prompt}")
62+
human_input = "Something went wrong."
63+
64+
return human_input
65+
66+
67+
# create our multi-agent framework components
68+
message_queue = RabbitMQMessageQueue(
69+
url=f"amqp://{message_queue_username}:{message_queue_password}@{message_queue_host}:{message_queue_port}/"
70+
)
71+
human_service = HumanService(
72+
message_queue=message_queue,
73+
description="For human input.",
74+
fn_input=human_input_fn,
75+
human_input_prompt="{input_str}",
76+
)
77+
human_component = ServiceComponent.from_service_definition(human_service)

0 commit comments

Comments
 (0)