|
1 | 1 | # Groq Python API library |
2 | 2 |
|
3 | | -[](https://pypi.org/project/groq/) |
| 3 | +[>)](https://pypi.org/project/groq/) |
4 | 4 |
|
5 | 5 | The Groq Python library provides convenient access to the Groq REST API from any Python 3.8+ |
6 | 6 | application. The library includes type definitions for all request params and response fields, |
@@ -80,6 +80,46 @@ asyncio.run(main()) |
80 | 80 |
|
81 | 81 | Functionality between the synchronous and asynchronous clients is otherwise identical. |
82 | 82 |
|
| 83 | +### With aiohttp |
| 84 | + |
| 85 | +By default, the async client uses `httpx` for HTTP requests. However, for improved concurrency performance you may also use `aiohttp` as the HTTP backend. |
| 86 | + |
| 87 | +You can enable this by installing `aiohttp`: |
| 88 | + |
| 89 | +```sh |
| 90 | +# install from PyPI |
| 91 | +pip install groq[aiohttp] |
| 92 | +``` |
| 93 | + |
| 94 | +Then you can enable it by instantiating the client with `http_client=DefaultAioHttpClient()`: |
| 95 | + |
| 96 | +```python |
| 97 | +import os |
| 98 | +import asyncio |
| 99 | +from groq import DefaultAioHttpClient |
| 100 | +from groq import AsyncGroq |
| 101 | + |
| 102 | + |
| 103 | +async def main() -> None: |
| 104 | + async with AsyncGroq( |
| 105 | + api_key=os.environ.get("GROQ_API_KEY"), # This is the default and can be omitted |
| 106 | + http_client=DefaultAioHttpClient(), |
| 107 | + ) as client: |
| 108 | + chat_completion = await client.chat.completions.create( |
| 109 | + messages=[ |
| 110 | + { |
| 111 | + "role": "user", |
| 112 | + "content": "Explain the importance of low latency LLMs", |
| 113 | + } |
| 114 | + ], |
| 115 | + model="llama3-8b-8192", |
| 116 | + ) |
| 117 | + print(chat_completion.id) |
| 118 | + |
| 119 | + |
| 120 | +asyncio.run(main()) |
| 121 | +``` |
| 122 | + |
83 | 123 | ## Using types |
84 | 124 |
|
85 | 125 | Nested request parameters are [TypedDicts](https://docs.python.org/3/library/typing.html#typing.TypedDict). Responses are [Pydantic models](https://docs.pydantic.dev) which also provide helper methods for things like: |
@@ -218,7 +258,7 @@ client.with_options(max_retries=5).chat.completions.create( |
218 | 258 | ### Timeouts |
219 | 259 |
|
220 | 260 | By default requests time out after 1 minute. You can configure this with a `timeout` option, |
221 | | -which accepts a float or an [`httpx.Timeout`](https://www.python-httpx.org/advanced/#fine-tuning-the-configuration) object: |
| 261 | +which accepts a float or an [`httpx.Timeout`](https://www.python-httpx.org/advanced/timeouts/#fine-tuning-the-configuration) object: |
222 | 262 |
|
223 | 263 | ```python |
224 | 264 | from groq import Groq |
|
![stainless-app[bot]](https://avatars.githubusercontent.com/in/378072?v=4&size=40){kind=avatar}
0 commit comments