Skip to content

Commit 4b2fea4

Browse files
committed
Release version 0.6.2
- Refactored transmit module for improved maintainability and testability - Added CI/CD safe smoke tests (no real database connections needed) - Obfuscated all PII and API keys in test data - Added SQLite support to all database operations - Fixed CLI error handling for proper exit codes - Fixed date filtering in SQLite implementation - Fixed CLI mode mapping for transmit functionality - Created comprehensive test suite with 97 passing tests Co-Authored-By: Erik Peterson <erik@cloudzero.com>
1 parent 86279db commit 4b2fea4

19 files changed

+3813
-539
lines changed

CHANGELOG.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,21 +7,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [0.6.2] - 2025-01-29
11+
12+
### Added
13+
- **Transmit module refactoring** for improved maintainability and testability
14+
- Split monolithic `DataTransmitter` into focused components with single responsibilities
15+
- Introduced dependency injection for all components
16+
- Created protocol-based design with `OutputHandler` and `Transmitter` interfaces
17+
- Added test-friendly implementations: `NullOutput`, `MockTransmitter`, `CollectingOutput`
18+
- Comprehensive test suite with 27 new tests demonstrating improved testability
19+
- Full backward compatibility maintained
20+
- **CI/CD safe smoke tests** for all CLI commands
21+
- Updated smoke tests to not require real database connections
22+
- All tests now safe for automated CI/CD pipelines
23+
- **PII obfuscation in test data**
24+
- Anonymized all names, emails, and API keys in test.sqlite
25+
- Updated create_test_sqlite.py script to generate only anonymized test data
26+
- Reviewed entire codebase to ensure no PII is present
27+
28+
### Changed
29+
- Updated CLI to use refactored transmit module (`transmit_refactored.py`)
30+
- Fixed CLI error handling to properly exit with error code when transmit fails
31+
- Improved BatchAnalyzer to correctly group data by date in refactored transmit module
32+
33+
### Fixed
34+
- Fixed date filtering in SQLite implementation for data source strategies
35+
- Fixed CLI mode mapping to correctly handle 'today', 'yesterday', 'date-range' modes
36+
37+
### Testing
38+
- All 97 tests passing including new refactored transmit tests
39+
- SQLite integration fully tested with transmit functionality
40+
- All smoke tests are now CI/CD safe
41+
1042
## [0.6.1] - 2025-01-28
1143

1244
### Fixed
1345
- Fixed import error in `data_processor.py` - `extract_model_name` now correctly imported from `model_name_strategies`
1446
- Added `__main__.py` to enable running package as module: `python -m ll2cz`
1547

1648
### Added
49+
- **SQLite database support** - All commands now accept `sqlite://` connection strings
1750
- Comprehensive import tests (`test_imports.py`) to prevent future import errors
1851
- CLI smoke tests (`test_cli_smoke.py`) to ensure all commands work without crashing
1952
- Tests for all critical imports and circular dependency detection
53+
- `scripts/create_test_sqlite.py` to generate test SQLite databases with sample data
54+
- SQLite-specific tests (`test_sqlite.py`) for database functionality
2055

2156
### Testing
2257
- Verified all CLI commands work correctly: `transmit`, `analyze`, `transform`, `cache`, `config`
2358
- All 21 core tests passing
24-
- All 16 new import and smoke tests passing
59+
- All 24 new tests passing (import, smoke, and SQLite tests)
60+
- SQLite support tested with both test databases and production-like schemas
2561

2662
## [0.6.0] - 2025-01-28
2763

README.md

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@ Transform LiteLLM database data into CloudZero AnyCost CBF format for cost track
44

55
## Features
66

7-
- Extract usage data from LiteLLM PostgreSQL database
7+
- Extract usage data from LiteLLM PostgreSQL or SQLite database
88
- Transform data into CloudZero Billing Format (CBF)
99
- **Cost comparison analysis** between SpendLogs and user tables (v0.4.0+)
1010
- **Dual data source support** with `--source` option: transaction-level SpendLogs or daily aggregated user tables
11+
- **SQLite support** for testing and offline analysis (v0.6.1+)
1112
- Analysis mode with beautiful terminal output using Rich
1213
- Multiple output options: CSV files or direct CloudZero API streaming
1314
- Built with modern Python tools: uv, ruff, pytest, polars, httpx
@@ -84,9 +85,12 @@ ll2cz config edit # Interactively edit configuration
8485
Transform LiteLLM data to CloudZero CBF format:
8586

8687
```bash
87-
# Display data on screen (formatted table)
88+
# Display data on screen (formatted table) - PostgreSQL
8889
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --screen
8990
91+
# Display data on screen - SQLite
92+
ll2cz transform --input "sqlite://path/to/litellm.sqlite" --screen
93+
9094
# Export to CSV file
9195
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --output data.csv
9296
@@ -219,10 +223,28 @@ uv run ruff check --fix src/ tests/
219223
uv build
220224
```
221225

226+
## Testing with SQLite
227+
228+
For testing and development, you can use SQLite instead of PostgreSQL:
229+
230+
```bash
231+
# Create a test SQLite database with sample data
232+
python scripts/create_test_sqlite.py
233+
234+
# Use the test database
235+
ll2cz transmit all --test --input "sqlite://test.sqlite"
236+
```
237+
238+
The test database includes:
239+
- Sample organizations, teams, and users
240+
- API keys with realistic naming
241+
- 30 days of usage data
242+
- Multiple model providers (OpenAI, Anthropic, etc.)
243+
222244
## Requirements
223245

224246
- Python ≥ 3.12
225-
- PostgreSQL database with LiteLLM data
247+
- PostgreSQL or SQLite database with LiteLLM data
226248
- CloudZero API key and connection ID (for streaming mode)
227249

228250
## License

docs/CLI_Reference.md

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,19 +31,25 @@ ll2cz transform [OPTIONS]
3131
```
3232

3333
### Options
34-
- `--input TEXT` - LiteLLM PostgreSQL database connection URL
34+
- `--input TEXT` - LiteLLM database connection URL (PostgreSQL or SQLite)
3535
- `--output TEXT` - Output CSV file name
3636
- `--screen` - Display transformed data on screen in formatted table
3737
- `--limit INTEGER` - Limit number of records for screen output (default: 50)
3838

3939
### Examples
4040
```bash
41-
# Display data on screen (formatted table)
41+
# Display data on screen (formatted table) - PostgreSQL
4242
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --screen
4343

44-
# Export to CSV file
44+
# Display data on screen - SQLite
45+
ll2cz transform --input "sqlite://path/to/litellm.sqlite" --screen
46+
47+
# Export to CSV file - PostgreSQL
4548
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --output data.csv
4649

50+
# Export to CSV file - SQLite
51+
ll2cz transform --input "sqlite://test.sqlite" --output data.csv
52+
4753
# Limit records for screen display
4854
ll2cz transform --screen --limit 25
4955

@@ -69,7 +75,7 @@ ll2cz transmit [OPTIONS] MODE [DATE_SPEC]
6975
- For `all` mode: ignored
7076

7177
### Options
72-
- `--input TEXT` - LiteLLM PostgreSQL database connection URL
78+
- `--input TEXT` - LiteLLM database connection URL (PostgreSQL or SQLite)
7379
- `--cz-api-key TEXT` - CloudZero API key
7480
- `--cz-connection-id TEXT` - CloudZero connection ID
7581
- `--append` - Use 'sum' operation instead of 'replace_hourly'
@@ -127,6 +133,20 @@ ll2cz transmit day --test
127133

128134
# Test specific month with API keys still required
129135
ll2cz transmit month 01-2024 --test --cz-api-key "key" --cz-connection-id "id"
136+
137+
# Test with SQLite database
138+
ll2cz transmit all --test --input "sqlite://test.sqlite"
139+
```
140+
141+
### Database Support
142+
Both PostgreSQL and SQLite databases are supported:
143+
144+
```bash
145+
# PostgreSQL
146+
ll2cz transmit all --input "postgresql://user:pass@host:5432/litellm_db"
147+
148+
# SQLite
149+
ll2cz transmit all --input "sqlite://path/to/database.sqlite"
130150
```
131151

132152
---

0 commit comments

Comments
 (0)