Skip to content

Commit 455d841

Browse files
diogoncalvesclaramoreiragactions-user
authored
CLI - Proxy & Tracker (#188)
* Feat/cli for tracker and proxy modules (#184) * feat: cli for tracker and proxy * feat: update tracker dockerfile * style: linting * chore: bump versions --------- Signed-off-by: Clara Moreira Gadelho <[email protected]> * [fix] bump prerelease version in pyproject.toml * [fix] bump prerelease version in pyproject.toml * ci: workflow for pushing alpha versions to dockerhub (#187) Signed-off-by: Clara Moreira Gadelho <[email protected]> --------- Signed-off-by: Clara Moreira Gadelho <[email protected]> Co-authored-by: Clara Moreira Gadelho <[email protected]> Co-authored-by: GitHub Actions <[email protected]>
1 parent 455b2c4 commit 455d841

File tree

7 files changed

+141
-7
lines changed

7 files changed

+141
-7
lines changed

‎.github/workflows/upload-dockerhub-dev.yml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
name: Push Prerelease Docker Image
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
module:
7+
description: "Select the module to push (e.g., tracker or proxy)"
8+
required: true
9+
type: choice
10+
options:
11+
- tracker
12+
- proxy
13+
version:
14+
description: "Alpha version to push (e.g., 1.1.0a1)"
15+
required: true
16+
17+
jobs:
18+
push-docker-alpha:
19+
runs-on: ubuntu-latest
20+
steps:
21+
# Checkout the code
22+
- name: Checkout code
23+
uses: actions/checkout@v3
24+
with:
25+
ref: develop
26+
token: ${{ secrets.GH_TOKEN }}
27+
28+
# Set up Docker Buildx
29+
- name: Set up Docker Buildx
30+
uses: docker/setup-buildx-action@v2
31+
32+
# Log in to Docker Hub
33+
- name: Log in to Docker Hub
34+
uses: docker/login-action@v2
35+
with:
36+
username: ${{ secrets.DOCKER_USERNAME }}
37+
password: ${{ secrets.DOCKER_PASSWORD }}
38+
39+
# Build Docker image
40+
- name: Build Docker image
41+
run: |
42+
MODULE=${{ github.event.inputs.module }}
43+
VERSION=${{ github.event.inputs.version }}
44+
IMAGE_NAME="tensoropsai/llmstudio-$MODULE"
45+
echo "Building Docker image: $IMAGE_NAME:$VERSION"
46+
47+
cd ./deploy
48+
make version=$VERSION build-llmstudio-$MODULE
49+
50+
# Push Docker image
51+
- name: Push Docker image
52+
run: |
53+
MODULE=${{ github.event.inputs.module }}
54+
VERSION=${{ github.event.inputs.version }}
55+
IMAGE_NAME="tensoropsai/llmstudio-$MODULE"
56+
echo "Pushing Docker image: $IMAGE_NAME:$VERSION"
57+
58+
docker push $IMAGE_NAME:$VERSION

‎deploy/proxy.Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,6 @@ RUN apt-get clean && apt-get update
66

77
# Install llmstudio
88
ARG LLMSTUDIO_VERSION
9-
RUN pip install 'llmstudio[proxy]'==${LLMSTUDIO_VERSION}
9+
RUN pip install 'llmstudio-proxy'==${LLMSTUDIO_VERSION}
1010

11-
CMD ["llmstudio", "server", "--proxy"]
11+
CMD ["llmstudio-proxy", "server"]

‎deploy/tracker.Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ RUN apt-get clean && apt-get update
66

77
# Install llmstudio
88
ARG LLMSTUDIO_VERSION
9-
RUN pip install 'llmstudio[tracker]'==${LLMSTUDIO_VERSION}
9+
RUN pip install 'llmstudio-tracker'==${LLMSTUDIO_VERSION}
1010
RUN pip install psycopg2-binary
1111

12-
CMD ["llmstudio", "server", "--tracker"]
12+
CMD ["llmstudio-tracker", "server"]

‎libs/proxy/llmstudio_proxy/cli.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import os
2+
import signal
3+
import threading
4+
5+
import click
6+
from llmstudio_proxy.server import setup_engine_server
7+
8+
9+
def handle_shutdown(signum, frame):
10+
print("Shutting down gracefully...")
11+
os._exit(0)
12+
13+
14+
@click.group()
15+
def main():
16+
pass
17+
18+
19+
@main.command()
20+
def server():
21+
signal.signal(signal.SIGINT, handle_shutdown)
22+
23+
setup_engine_server()
24+
25+
print("Press CTRL+C to stop.")
26+
27+
stop_event = threading.Event()
28+
try:
29+
stop_event.wait()
30+
except KeyboardInterrupt:
31+
print("Shutting down server...")
32+
33+
34+
if __name__ == "__main__":
35+
main()

‎libs/proxy/pyproject.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[tool.poetry]
22
name = "llmstudio-proxy"
3-
version = "1.0.2"
3+
version = "1.0.3a0"
44
description = ""
55
authors = ["Diogo Goncalves <[email protected]>"]
66
readme = "README.md"
@@ -19,6 +19,9 @@ llmstudio-core = "^1.0.0"
1919
[tool.poetry.group.dev.dependencies]
2020
llmstudio-core = { path = "../core/", develop = true }
2121

22+
[tool.poetry.scripts]
23+
llmstudio-proxy = "llmstudio_proxy.cli:main"
24+
2225
[build-system]
2326
requires = ["poetry-core"]
2427
build-backend = "poetry.core.masonry.api"

‎libs/tracker/llmstudio_tracker/cli.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import os
2+
import signal
3+
import threading
4+
5+
import click
6+
from llmstudio_tracker.server import setup_tracking_server
7+
8+
9+
def handle_shutdown(signum, frame):
10+
print("Shutting down gracefully...")
11+
os._exit(0)
12+
13+
14+
@click.group()
15+
def main():
16+
pass
17+
18+
19+
@main.command()
20+
def server():
21+
signal.signal(signal.SIGINT, handle_shutdown)
22+
23+
setup_tracking_server()
24+
25+
print("Press CTRL+C to stop.")
26+
27+
stop_event = threading.Event()
28+
try:
29+
stop_event.wait()
30+
except KeyboardInterrupt:
31+
print("Shutting down server...")
32+
33+
34+
if __name__ == "__main__":
35+
main()

‎libs/tracker/pyproject.toml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[tool.poetry]
22
name = "llmstudio-tracker"
3-
version = "1.0.3"
4-
description = "LLMstudio Tracker is the module of LLMstudio that allows monitoring and logging your LLM calls. It supports seamless integration with the LLMstudio environment through configurable tracking servers, allowing for detailed insights into synchronous and asynchronous chat interactions. By leveraging LLMstudio Tracker, users can gain insights on model performance and streamline development workflows with actionable analytics."
3+
version = "1.0.4a0"
4+
description = "LLMstudio Tracker is the module of LLMstudio that allows monitoring and logging your LLM calls. By leveraging LLMstudio Tracker, users can gain insights on model performance and streamline development workflows with actionable analytics."
55
authors = ["Diogo Goncalves <[email protected]>"]
66
readme = "README.md"
77

@@ -16,6 +16,9 @@ uvicorn = "^0.27"
1616
sqlalchemy-bigquery = "^1.12.0"
1717
google-cloud-bigquery-storage = "^2.27.0"
1818

19+
[tool.poetry.scripts]
20+
llmstudio-proxy = "llmstudio_tracker.cli:main"
21+
1922
[build-system]
2023
requires = ["poetry-core"]
2124
build-backend = "poetry.core.masonry.api"

0 commit comments

Comments
 (0)