Inference Gateway 🌐

Welcome to the Inference Gateway repository! This open-source project aims to provide a high-performance gateway that unifies multiple Large Language Model (LLM) providers. Whether you're using local solutions like Ollama or major cloud providers such as OpenAI, Groq, Cohere, Anthropic, Cloudflare, and DeepSeek, Inference Gateway offers a seamless experience.

Table of Contents

1. Features
2. Getting Started
3. Installation
4. Usage
5. Contributing
6. License
7. Contact
8. Releases

Features ✨

• Unified Interface: Interact with various LLM providers through a single API.
• High Performance: Optimized for speed and efficiency.
• Flexibility: Easily switch between local and cloud solutions.
• Extensible: Add support for new providers with minimal effort.
• Open Source: Community-driven development.

Getting Started 🚀

To get started with Inference Gateway, follow these simple steps. You will need to have a compatible environment set up.

Prerequisites

• Python 3.7 or higher
• pip (Python package installer)
• Access to the LLM providers you wish to use

Installation

You can install Inference Gateway using pip. Open your terminal and run:

pip install inference-gateway

Usage 📖

After installation, you can start using Inference Gateway in your projects. Here’s a basic example:

from inference_gateway import Gateway

# Initialize the gateway
gateway = Gateway()

# Call a local model
response_local = gateway.call_local_model("Ollama", "Your prompt here")

# Call a cloud model
response_cloud = gateway.call_cloud_model("OpenAI", "Your prompt here")

print(response_local)
print(response_cloud)

This example shows how to initialize the gateway and make calls to both local and cloud models.

Contributing 🤝

We welcome contributions from the community! If you want to contribute, please follow these steps:

1. Fork the repository.
2. Create a new branch (git checkout -b feature/YourFeature).
3. Make your changes.
4. Commit your changes (git commit -m 'Add some feature').
5. Push to the branch (git push origin feature/YourFeature).
6. Open a Pull Request.

Please make sure to follow our coding standards and guidelines.

License 📜

This project is licensed under the MIT License. See the LICENSE file for details.

Contact 📫

For questions or suggestions, feel free to reach out:

• Author: Your Name
• Email: [email protected]

Releases 📦

To download the latest release of Inference Gateway, visit the Releases section. Here, you can find the latest version, download it, and execute the necessary files to get started.

For more detailed release notes, check the Releases section to stay updated on new features and improvements.

---

Thank you for your interest in Inference Gateway! We look forward to seeing what you build with it.