Skip to content

Commit a732c8d

Browse files
daniasdanias
committed
Add managed local embeddings runtime with packaging and tests
1 parent 93a4cdf commit a732c8d

28 files changed

Lines changed: 1789 additions & 1 deletion

‎.github/workflows/ci.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
jobs:
8+
test:
9+
runs-on: ubuntu-24.04
10+
11+
steps:
12+
- name: Check out repository
13+
uses: actions/checkout@v4
14+
15+
- name: Set up Python
16+
uses: actions/setup-python@v5
17+
with:
18+
python-version: "3.11"
19+
cache: "pip"
20+
21+
- name: Install dependencies
22+
run: |
23+
python -m pip install --upgrade pip
24+
python -m pip install -e ".[dev]"
25+
26+
- name: Compile source tree
27+
run: python -m compileall src tests scripts
28+
29+
- name: Run test suite
30+
run: pytest
31+
32+
- name: Smoke test CLI shape
33+
run: |
34+
bitloops-embeddings --help
35+
bitloops-embeddings describe --model bge-m3

‎.github/workflows/release.yml‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags:
6+
- "v*.*.*"
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: write
11+
12+
jobs:
13+
build:
14+
name: Build ${{ matrix.target }}
15+
runs-on: ${{ matrix.os }}
16+
continue-on-error: ${{ matrix.allow_failure }}
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
include:
21+
- os: ubuntu-24.04
22+
target: x86_64-unknown-linux-gnu
23+
allow_failure: false
24+
run_real_smoke: true
25+
- os: ubuntu-24.04-arm
26+
target: aarch64-unknown-linux-gnu
27+
allow_failure: true
28+
run_real_smoke: false
29+
- os: macos-13
30+
target: x86_64-apple-darwin
31+
allow_failure: false
32+
run_real_smoke: false
33+
- os: macos-14
34+
target: aarch64-apple-darwin
35+
allow_failure: false
36+
run_real_smoke: false
37+
- os: windows-2022
38+
target: x86_64-pc-windows-msvc
39+
allow_failure: false
40+
run_real_smoke: false
41+
42+
steps:
43+
- name: Check out repository
44+
uses: actions/checkout@v4
45+
46+
- name: Set up Python
47+
uses: actions/setup-python@v5
48+
with:
49+
python-version: "3.11"
50+
cache: "pip"
51+
52+
- name: Install dependencies
53+
shell: bash
54+
run: |
55+
python -m pip install --upgrade pip
56+
python -m pip install -e ".[dev]"
57+
58+
- name: Compile source tree
59+
shell: bash
60+
run: python -m compileall src scripts
61+
62+
- name: Real backend smoke before packaging
63+
if: matrix.run_real_smoke
64+
shell: bash
65+
run: python scripts/real_backend_smoke.py --binary bitloops-embeddings
66+
67+
- name: Build packaged archive
68+
id: package
69+
shell: bash
70+
run: |
71+
python scripts/package_release.py \
72+
--target "${{ matrix.target }}" \
73+
--archive-dir build/artifacts \
74+
--github-output "${GITHUB_OUTPUT}"
75+
76+
- name: Smoke test packaged help output
77+
shell: bash
78+
run: |
79+
"${{ steps.package.outputs.bundle_executable }}" --help
80+
81+
- name: Real backend smoke after packaging
82+
if: matrix.run_real_smoke
83+
shell: bash
84+
run: python scripts/real_backend_smoke.py --binary "${{ steps.package.outputs.bundle_executable }}"
85+
86+
- name: Upload packaged artefact
87+
uses: actions/upload-artifact@v4
88+
with:
89+
name: ${{ matrix.target }}
90+
path: ${{ steps.package.outputs.archive_path }}
91+
92+
publish:
93+
if: startsWith(github.ref, 'refs/tags/v')
94+
needs: build
95+
runs-on: ubuntu-24.04
96+
97+
steps:
98+
- name: Download packaged artefacts
99+
uses: actions/download-artifact@v4
100+
with:
101+
path: release-artifacts
102+
merge-multiple: true
103+
104+
- name: Create GitHub Release
105+
uses: softprops/action-gh-release@v2
106+
with:
107+
files: release-artifacts/*

‎README.md‎

Lines changed: 202 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,202 @@
1-
# bitloops-embeddings
1+
# bitloops-embeddings
2+
3+
`bitloops-embeddings` is a managed local embeddings runtime for Bitloops. It provides:
4+
5+
- a one-shot CLI for simple embedding requests
6+
- a long-lived local HTTP server for repeated requests
7+
- release packaging for major desktop and server operating systems
8+
9+
The first release is intentionally operational rather than retrieval-quality-complete. It focuses on a stable interface, model bootstrapping, hello-world inference, and releasable artefacts.
10+
11+
## Runtime model
12+
13+
The initial public model identifier is `bge-m3`.
14+
15+
- Public model id: `bge-m3`
16+
- Upstream model id: `BAAI/bge-m3`
17+
- Backend: `sentence-transformers`
18+
- Device: CPU
19+
- Provisioning: first-run download into a local cache directory
20+
21+
The command and HTTP layers are written against an internal backend registry so additional models or inference backends can be added later without changing the user-facing contracts.
22+
23+
## Requirements
24+
25+
- Python `3.11` or `3.12`
26+
- `pip`
27+
28+
## Local development
29+
30+
Create an environment and install the project with development dependencies:
31+
32+
```bash
33+
python3.12 -m venv .venv
34+
source .venv/bin/activate
35+
python -m pip install --upgrade pip
36+
python -m pip install -e ".[dev]"
37+
```
38+
39+
Run the test suite:
40+
41+
```bash
42+
pytest
43+
```
44+
45+
## CLI usage
46+
47+
Show the available commands:
48+
49+
```bash
50+
bitloops-embeddings --help
51+
```
52+
53+
Generate a single embedding:
54+
55+
```bash
56+
bitloops-embeddings embed --model bge-m3 --input "Hello World"
57+
```
58+
59+
Example response:
60+
61+
```json
62+
{
63+
"model_id": "bge-m3",
64+
"dimensions": 1024,
65+
"embeddings": [[0.123, -0.456, 0.789]],
66+
"runtime": {
67+
"name": "bitloops-embeddings",
68+
"version": "0.1.0"
69+
}
70+
}
71+
```
72+
73+
Write the same JSON response to a file as well:
74+
75+
```bash
76+
bitloops-embeddings embed \
77+
--model bge-m3 \
78+
--input "Hello World" \
79+
--output ./embedding.json
80+
```
81+
82+
Inspect model metadata without loading the model:
83+
84+
```bash
85+
bitloops-embeddings describe --model bge-m3
86+
```
87+
88+
## Server usage
89+
90+
Start the local server:
91+
92+
```bash
93+
bitloops-embeddings serve --model bge-m3
94+
```
95+
96+
Defaults:
97+
98+
- host: `127.0.0.1`
99+
- port: `7719`
100+
- max batch size: `32`
101+
102+
Override the bind target:
103+
104+
```bash
105+
bitloops-embeddings serve --model bge-m3 --host 127.0.0.1 --port 7719
106+
```
107+
108+
### HTTP API
109+
110+
Health:
111+
112+
```bash
113+
curl http://127.0.0.1:7719/health
114+
```
115+
116+
Embed:
117+
118+
```bash
119+
curl -X POST http://127.0.0.1:7719/embed \
120+
-H "content-type: application/json" \
121+
-d '{"texts":["Hello World"]}'
122+
```
123+
124+
Response shape:
125+
126+
```json
127+
{
128+
"model_id": "bge-m3",
129+
"dimensions": 1024,
130+
"embeddings": [[0.123, -0.456, 0.789]],
131+
"runtime": {
132+
"name": "bitloops-embeddings",
133+
"version": "0.1.0"
134+
}
135+
}
136+
```
137+
138+
Error shape:
139+
140+
```json
141+
{
142+
"error": {
143+
"code": "runtime_error",
144+
"message": "..."
145+
}
146+
}
147+
```
148+
149+
## Cache directory resolution
150+
151+
Model cache resolution order:
152+
153+
1. `--cache-dir`
154+
2. `BITLOOPS_EMBEDDINGS_CACHE_DIR`
155+
3. platform default cache directory via `platformdirs`
156+
157+
Examples:
158+
159+
- macOS: `~/Library/Caches/bitloops-embeddings`
160+
- Linux: `~/.cache/bitloops-embeddings`
161+
- Windows: `%LOCALAPPDATA%/bitloops-embeddings/Cache`
162+
163+
## Packaging
164+
165+
Release packaging uses PyInstaller `--onedir` bundles. Each archive contains:
166+
167+
- the launchable runtime bundle
168+
- `README.md`
169+
- `LICENSE`
170+
171+
Create a local packaged artefact:
172+
173+
```bash
174+
python scripts/package_release.py --target x86_64-apple-darwin
175+
```
176+
177+
Run the real-model smoke test against an installed console script or packaged executable:
178+
179+
```bash
180+
python scripts/real_backend_smoke.py --binary bitloops-embeddings
181+
```
182+
183+
## GitHub Actions
184+
185+
The repository includes two workflows:
186+
187+
- `ci.yml`
188+
- installs dependencies
189+
- runs unit and integration tests
190+
- runs compile checks
191+
- validates the CLI help output
192+
- `release.yml`
193+
- builds native bundles for the target matrix
194+
- packages archives
195+
- uploads artefacts
196+
- creates a GitHub Release for `v*.*.*` tags
197+
198+
## Troubleshooting
199+
200+
- The first `embed` or `serve` invocation downloads model files into the local cache. This can take a while on a cold machine.
201+
- If model loading fails, check network access to Hugging Face and confirm the cache directory is writable.
202+
- The runtime does not log input texts by default.

‎pyproject.toml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
[build-system]
2+
requires = ["hatchling>=1.25.0"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "bitloops-embeddings"
7+
description = "Managed local embeddings runtime for Bitloops"
8+
readme = "README.md"
9+
license = { text = "Apache-2.0" }
10+
authors = [
11+
{ name = "Bitloops" },
12+
]
13+
requires-python = ">=3.11,<3.13"
14+
dynamic = ["version"]
15+
dependencies = [
16+
"fastapi>=0.115.0,<1.0.0",
17+
"platformdirs>=4.3.0,<5.0.0",
18+
"pydantic>=2.9.0,<3.0.0",
19+
"sentence-transformers>=3.1.0,<4.0.0",
20+
"typer>=0.12.5,<1.0.0",
21+
"uvicorn>=0.30.6,<1.0.0",
22+
]
23+
24+
[project.optional-dependencies]
25+
dev = [
26+
"httpx>=0.27.2,<1.0.0",
27+
"pyinstaller>=6.11.0,<7.0.0",
28+
"pytest>=8.3.3,<9.0.0",
29+
]
30+
31+
[project.scripts]
32+
bitloops-embeddings = "bitloops_embeddings.cli:main"
33+
34+
[tool.hatch.version]
35+
path = "src/bitloops_embeddings/version.py"
36+
37+
[tool.hatch.build.targets.wheel]
38+
packages = ["src/bitloops_embeddings"]
39+
40+
[tool.pytest.ini_options]
41+
pythonpath = ["src"]
42+
testpaths = ["tests"]

0 commit comments

Comments
 (0)