Skip to content

Commit ff2706e

Browse files
MQ37jirispilka
andauthored
feat: update langchain apify docs (#1441)
- fix and update [LangChain Apify integration docs](https://docs.apify.com/platform/integrations/langchain) to use our new [langchain-apify](https://github.com/apify/langchain-apify) integration package. - add docs on how to use Apify tools with agents using [LangGraph](https://www.langchain.com/langgraph) --------- Co-authored-by: Jiri Spilka <[email protected]> Co-authored-by: Jiří Spilka <[email protected]>
1 parent 0d4a857 commit ff2706e

File tree

3 files changed

+186
-13
lines changed

3 files changed

+186
-13
lines changed

sources/platform/integrations/ai/haystack.md

+1
Original file line numberDiff line numberDiff line change
@@ -185,4 +185,5 @@ To run it, you can use the following command: `python apify_integration.py`
185185
- [Apify-haystack integration documentation](https://haystack.deepset.ai/integrations/apify)
186186
- [Apify-haystack integration source code](https://github.com/apify/apify-haystack)
187187
- [Example: RAG - Extract and use website content for question answering](https://haystack.deepset.ai/cookbook/apify_haystack_rag)
188+
- [Example: RAG: Web Search and Analysis with Apify and Haystack](https://haystack.deepset.ai/cookbook/apify_haystack_rag_web_browser)
188189
- [Example: Analyze Your Instagram Comments’ Vibe](https://haystack.deepset.ai/cookbook/apify_haystack_instagram_comments_analysis)

sources/platform/integrations/ai/langchain.md

+25-13
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ If you prefer to use JavaScript, you can follow the [JavaScript LangChain docum
2020

2121
Before we start with the integration, we need to install all dependencies:
2222

23-
`pip install apify-client langchain langchain_community langchain_openai openai tiktoken`
23+
`pip install langchain langchain-openai langchain-apify`
2424

2525
After successful installation of all dependencies, we can start writing code.
2626

@@ -30,9 +30,10 @@ First, import all required packages:
3030
import os
3131

3232
from langchain.indexes import VectorstoreIndexCreator
33-
from langchain_community.utilities import ApifyWrapper
34-
from langchain_core.document_loaders.base import Document
35-
from langchain_openai import OpenAI
33+
from langchain_apify import ApifyWrapper
34+
from langchain_core.documents import Document
35+
from langchain_core.vectorstores import InMemoryVectorStore
36+
from langchain_openai import ChatOpenAI
3637
from langchain_openai.embeddings import OpenAIEmbeddings
3738
```
3839

@@ -49,6 +50,7 @@ Note that if you already have some results in an Apify dataset, you can load the
4950

5051
```python
5152
apify = ApifyWrapper()
53+
llm = ChatOpenAI(model="gpt-4o-mini")
5254

5355
loader = apify.call_actor(
5456
actor_id="apify/website-content-crawler",
@@ -68,14 +70,17 @@ The Actor call may take some time as it crawls the LangChain documentation websi
6870
Initialize the vector index from the crawled documents:
6971

7072
```python
71-
index = VectorstoreIndexCreator(embedding=OpenAIEmbeddings()).from_loaders([loader])
73+
index = VectorstoreIndexCreator(
74+
vectorstore_cls=InMemoryVectorStore,
75+
embedding=OpenAIEmbeddings()
76+
).from_loaders([loader])
7277
```
7378

7479
And finally, query the vector index:
7580

7681
```python
7782
query = "What is LangChain?"
78-
result = index.query_with_sources(query, llm=OpenAI())
83+
result = index.query_with_sources(query, llm=llm)
7984

8085
print("answer:", result["answer"])
8186
print("source:", result["sources"])
@@ -87,15 +92,17 @@ If you want to test the whole example, you can simply create a new file, `langch
8792
import os
8893

8994
from langchain.indexes import VectorstoreIndexCreator
90-
from langchain_community.utilities import ApifyWrapper
91-
from langchain_core.document_loaders.base import Document
92-
from langchain_openai import OpenAI
95+
from langchain_apify import ApifyWrapper
96+
from langchain_core.documents import Document
97+
from langchain_core.vectorstores import InMemoryVectorStore
98+
from langchain_openai import ChatOpenAI
9399
from langchain_openai.embeddings import OpenAIEmbeddings
94100

95101
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"
96102
os.environ["APIFY_API_TOKEN"] = "Your Apify API token"
97103

98104
apify = ApifyWrapper()
105+
llm = ChatOpenAI(model="gpt-4o-mini")
99106

100107
print("Call website content crawler ...")
101108
loader = apify.call_actor(
@@ -104,9 +111,12 @@ loader = apify.call_actor(
104111
dataset_mapping_function=lambda item: Document(page_content=item["text"] or "", metadata={"source": item["url"]}),
105112
)
106113
print("Compute embeddings...")
107-
index = VectorstoreIndexCreator(embedding=OpenAIEmbeddings()).from_loaders([loader])
114+
index = VectorstoreIndexCreator(
115+
vectorstore_cls=InMemoryVectorStore,
116+
embedding=OpenAIEmbeddings()
117+
).from_loaders([loader])
108118
query = "What is LangChain?"
109-
result = index.query_with_sources(query, llm=OpenAI())
119+
result = index.query_with_sources(query, llm=llm)
110120

111121
print("answer:", result["answer"])
112122
print("source:", result["sources"])
@@ -117,9 +127,11 @@ To run it, you can use the following command: `python langchain_integration.py`
117127
After running the code, you should see the following output:
118128

119129
```text
120-
answer: LangChain is a framework for developing applications powered by language models. It provides standard, extendable interfaces, external integrations, and end-to-end implementations for off-the-shelf use. It also integrates with other LLMs, systems, and products to create a vibrant and thriving ecosystem.
130+
answer: LangChain is a framework designed for developing applications powered by large language models (LLMs). It simplifies the
131+
entire application lifecycle, from development to productionization and deployment. LangChain provides open-source components a
132+
nd integrates with various third-party tools, making it easier to build and optimize applications using language models.
121133
122-
source: https://python.langchain.com
134+
source: https://python.langchain.com/docs/get_started/introduction
123135
```
124136

125137
LangChain is a standard interface through which you can interact with a variety of large language models (LLMs).
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
---
2+
title: 🦜🔘➡️ LangGraph integration
3+
sidebar_label: LangGraph
4+
description: Learn how to build AI Agents with Apify and LangGraph 🦜🔘➡️.
5+
sidebar_position: 1
6+
slug: /integrations/langgraph
7+
---
8+
9+
**Learn how to build AI Agents with Apify and LangGraph.**
10+
11+
---
12+
13+
## What is LangGraph
14+
15+
[LangGraph](https://www.langchain.com/langgraph) is a framework designed for constructing stateful, multi-agent applications with Large Language Models (LLMs), allowing developers to build complex AI agent workflows that can leverage tools, APIs, and databases.
16+
17+
:::note Explore LangGraph
18+
19+
For more in-depth details on LangGraph, check out its [official documentation](https://langchain-ai.github.io/langgraph/).
20+
21+
:::
22+
23+
## How to use Apify with LangGraph
24+
25+
This guide will demonstrate how to use Apify Actors with LangGraph by building a ReAct agent that will use the [RAG Web Browser](https://apify.com/apify/rag-web-browser) Actor to search Google for TikTok profiles and [TikTok Data Extractor](https://apify.com/clockworks/free-tiktok-scraper) Actor to extract data from the TikTok profiles to analyze the profiles.
26+
27+
### Prerequisites
28+
29+
- **Apify API token**: To use Apify Actors in LangGraph, you need an Apify API token. If you don't have one, you can learn how to obtain it in the [Apify documentation](https://docs.apify.com/platform/integrations/api).
30+
31+
- **OpenAI API key**: In order to work with agents in LangGraph, you need an OpenAI API key. If you don't have one, you can get it from the [OpenAI platform](https://platform.openai.com/account/api-keys).
32+
33+
- **Python packages**: You need to install the following Python packages:
34+
35+
```bash
36+
pip install langgraph langchain-apify langchain-openai
37+
```
38+
39+
### Building the TikTok profile search and analysis agent
40+
41+
First, import all required packages:
42+
43+
```python
44+
import os
45+
46+
from langchain_apify import ApifyActorsTool
47+
from langchain_core.messages import HumanMessage
48+
from langchain_openai import ChatOpenAI
49+
from langgraph.prebuilt import create_react_agent
50+
```
51+
52+
Next, set the environment variables for the Apify API token and OpenAI API key:
53+
54+
```python
55+
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"
56+
os.environ["APIFY_API_TOKEN"] = "Your Apify API token"
57+
```
58+
59+
Instantiate LLM and Apify Actors tools:
60+
61+
```python
62+
llm = ChatOpenAI(model="gpt-4o-mini")
63+
64+
browser = ApifyActorsTool("apify/rag-web-browser")
65+
tiktok = ApifyActorsTool("clockworks/free-tiktok-scraper")
66+
```
67+
68+
Create the ReAct agent with the LLM and Apify Actors tools:
69+
70+
```python
71+
tools = [browser, tiktok]
72+
agent_executor = create_react_agent(llm, tools)
73+
```
74+
75+
Finally, run the agent and stream the messages:
76+
77+
```python
78+
for state in agent_executor.stream(
79+
stream_mode="values",
80+
input={
81+
"messages": [
82+
HumanMessage(content="Search the web for OpenAI TikTok profile and analyze their profile.")
83+
]
84+
}):
85+
state["messages"][-1].pretty_print()
86+
```
87+
88+
:::note Search and analysis may take some time
89+
90+
The agent tool call may take some time as it searches the web for OpenAI TikTok profiles and analyzes them.
91+
92+
:::
93+
94+
You will see the agent's messages in the console, which will show each step of the agent's workflow.
95+
96+
```text
97+
================================ Human Message =================================
98+
99+
Search the web for OpenAI TikTok profile and analyze their profile.
100+
================================== AI Message ==================================
101+
Tool Calls:
102+
apify_actor_apify_rag-web-browser (call_y2rbmQ6gYJYC2lHzWJAoKDaq)
103+
Call ID: call_y2rbmQ6gYJYC2lHzWJAoKDaq
104+
Args:
105+
run_input: {"query":"OpenAI TikTok profile","maxResults":1}
106+
107+
...
108+
109+
================================== AI Message ==================================
110+
111+
The OpenAI TikTok profile is titled "OpenAI (@openai) Official." Here are some key details about the profile:
112+
113+
- **Followers**: 592.3K
114+
- **Likes**: 3.3M
115+
- **Description**: The profile features "low key research previews" and includes videos that showcase their various projects and research developments.
116+
117+
### Profile Overview:
118+
- **Profile URL**: [OpenAI TikTok Profile](https://www.tiktok.com/@openai?lang=en)
119+
- **Content Focus**: The posts primarily involve previews of OpenAI's research and various AI-related innovations.
120+
121+
...
122+
123+
```
124+
125+
126+
If you want to test the whole example, you can simply create a new file, `langgraph_integration.py`, and copy the whole code into it.
127+
128+
```python
129+
import os
130+
131+
from langchain_apify import ApifyActorsTool
132+
from langchain_core.messages import HumanMessage
133+
from langchain_openai import ChatOpenAI
134+
from langgraph.prebuilt import create_react_agent
135+
136+
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"
137+
os.environ["APIFY_API_TOKEN"] = "Your Apify API token"
138+
139+
llm = ChatOpenAI(model="gpt-4o-mini")
140+
141+
browser = ApifyActorsTool("apify/rag-web-browser")
142+
tiktok = ApifyActorsTool("clockworks/free-tiktok-scraper")
143+
144+
tools = [browser, tiktok]
145+
agent_executor = create_react_agent(llm, tools)
146+
147+
for state in agent_executor.stream(
148+
stream_mode="values",
149+
input={
150+
"messages": [
151+
HumanMessage(content="Search the web for OpenAI TikTok profile and analyze their profile.")
152+
]
153+
}):
154+
state["messages"][-1].pretty_print()
155+
```
156+
157+
## Resources
158+
159+
- [Apify Actors](https://docs.apify.com/platform/actors)
160+
- [LangGraph - How to Create a ReAct Agent](https://langchain-ai.github.io/langgraph/how-tos/create-react-agent/)

0 commit comments

Comments
 (0)