Kolobok: Week 3

content/docs/2025/Kolobok/week3.md

@@ -0,0 +1,175 @@
+---
+title: "Week 3"
+---
+
+# Week 3 Progress Report
+
+## Executive Summary
+
+In Week 3, the Kolobok team achieved major technical integration milestones while simultaneously pushing the boundaries of data engineering and model experimentation. The MVP is now operational for tread depth estimation and spike condition classification across both Telegram and web interfaces. Brand recognition, the third major component, is in active R&D, with promising results achieved using GPT-4o-mini.
+
+This week also marked a shift in engineering philosophy — moving from static databases and hand-constructed logic to a more dynamic, learning-based system design. From synthetic dataset generation in Unity to CLAHE-enhanced unwrapping for OCR, we continuously prioritized robustness, real-world performance, and system modularity. Our architecture now accommodates user feedback and model corrections, reflecting a maturing product vision that emphasizes trust, usability, and ML-aided transparency.
+
+---
+
+## Feature Implementation
+
+### End-to-End Flow
+
+The core pipeline now works across both Telegram and web UI:
+- User submits a photo (tread-side or sidewall)
+- Backend authenticates and processes using deployed ML models
+- Results include tread depth, spike count, and condition (brand OCR pending)
+- Users may correct predictions manually (bot and site)
+
+Only brand/model recognition is pending deployment. The LLM-based OCR pipeline is developed and tested but not yet integrated.
+
+---
+
+## OCR Research and Integration
+
+### Investigation and Evaluation
+
+The team explored and benchmarked six OCR pipelines, focusing on accuracy and preprocessing sensitivity. Models included:
+
+- Tesseract (Google OCR)
+- MMOCR (OpenMMLab) variants: DBNet++, PSENet, PANet, TextSnake
+- GPT-4o-mini (OpenAI Vision Language Model)
+
+Each was tested with raw images, polar unwrapping, and CLAHE enhancement.
+
+### OCR Evaluation Results
+
+| OCR Pipeline | Raw | Unwrapped | CLAHE |
+|--------------|-----|-----------|--------|
+| Tesseract | 4 | 9 | 8 |
+| DBNet++ + ABINet | 6 | 10 | 15 |
+| PSENet + ABINet | 5 | 11 | 14 |
+| TextSnake + ABINet | 7 | 12 | 16 |
+| PANet + ABINet | 3 | 8 | 9 |
+| GPT-4o-mini | 37 | 45 | 45 |
+
+GPT-4o-mini achieved perfect accuracy (45/45) on the benchmark using unwrapped CLAHE-enhanced images. It was selected for integration in Week 4.
+
+---
+
+## ML Pipeline Development
+
+### Tread Depth Estimation
+
+- Ensemble regression with Swin Transformer, DenseNet, ConvNeXt (in development)
+- Unity-generated dataset used for pretraining and edge case augmentation
+- MAE on test set: ~0.6 mm
+- Augmentations: rotations, crops
+
+### Spike Classification
+
+- Binary classifier using ResNet-like CNN
+- Dataset expanded with 6000+ bootstrapped samples
+- Tires without spikes used for hard negatives
+- Final FP + FN on test set: 10
+
+---
+
+## Model Architecture and Configuration
+
+### Regression Model
+- Loss: MSE + MAE monitoring
+- Optimizer: AdamW
+- LR Scheduler: CosineAnnealingLR
+- Batch size: 16
+- Epochs: 40
+
+### Spike Classification
+- Loss: CrossEntropy + hard-negative mining
+- Augmentations: crop, rotate, CLAHE
+
+### Logging
+- TensorBoard: MAE trends, misclassified visualizations, histograms
+- Checkpointing: per-epoch with val metrics
+
+---
+
+## Experimental Insights
+
+| Experiment | Finding | Outcome |
+|------------|---------|---------|
+| CLAHE vs HE | CLAHE outperformed consistently | Standardized CLAHE |
+| Hard negatives | Reduced false positives by ~20% | Included in training |
+| GPT OCR vs MMOCR | GPT superior on real-world samples | Adopted GPT-4o-mini |
+| Ensemble vs single model | 0.2mm better MAE | Final model is stacked ensemble |
+| Unwrapping for OCR | Boosted recognition by 3× | Pipeline requirement |
+
+---
+
+## API, Bot, and Web UI
+
+### Backend (FastAPI)
+- Auth: Bearer token
+- Endpoints: `/analyze/tread`, `/identify_tire`
+- Error codes: 400 (bad image), 401 (auth)
+
+### Telegram Bot
+- Manual correction of predictions
+- Robust state handling
+
+### Web Interface
+- Functional MVP with drag-and-drop upload
+- Connects to same API backend
+- Design aligns with Telegram UX
+
+---
+
+## Data Handling & Privacy
+
+- No user data stored
+- Brand/model database replaced by GPT queries
+- Only temporary logs used for diagnostics
+
+---
+
+## Testing and Feedback
+
+| Issue | Fix |
+|-------|-----|
+| Spike false positives | Added negative tire images |
+| Lighting sensitivity | Unity-based shadow samples |
+| OCR failures | Switched to GPT-4o-mini |
+
+---
+
+## Roadmap
+
+- Integrate GPT-based OCR into full pipeline
+- Conduct small user study (10 users)
+- Add admin dashboard for request analysis
+- Finalize dataset with versioning and backups
+- Fine-tune depth model using correction feedback
+
+---
+
+## Lessons Learned
+
+- Real-world variance must drive training strategy
+- ML + UX = user trust
+- LLMs simplify pipelines previously requiring deep tuning
+- Good error design prevents user frustration
+
+## Team Contributions
+
+| Team Member | Contributions |
+|-------------|---------------|
+| **Nikita Menshikov** | Wrote the report, set [tasks](https://github.com/orgs/IU-Capstone-Project-2025/projects/9), pitched dataset augmentation techniques, setup labelling for new spikes dataset |
+| **Nikita Zagainov** | Conducted OCR [research](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/ed78b4e0b0b67be7354dd563dff40f31b930f34a/ML/research/ocr_research/main.pdf) for brand and parameter recognition, built tire segmentator ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/7e1d76340241c2b0d7eae70280ae11b379062aeb), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bcc83732147af72a135a84e0081dd4a582900dd9), [3](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5e3cecfc353964a78755964403c2ceb53065b0cc)) |
+| **Dmitry Tetkin** | [Modeled](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/9d6bb4c74b963ff3a24c0d905d55409a1d611338/TireDataset/Assets/Report/SyntheticDatasetReport.pdf) synthetic tires in [Unity](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/630504b7e7c95a6aa170bd7cd5975c615b8d6a0e), expanded training data |
+| **Vladislav Strelkov** | Connected endpoints to [tg bot](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d793d9c2fb92914455a948b099a3064ad265c0fa), refined user paths, enhanced it with functionality for the demo |
+| **Darya Stepanova** | Developed the first version of site [frontend](https://www.figma.com/design/llkM7QUkBNkwm0pit8qUhn/KolobokBoard?node-id=5-2&t=C5HOXlHhL0asVBw2-1), basic [back](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/5337ae775e5aebe0b5c78d0287dedf40def59712) and [landing](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/f2849fdf6f740b934fdeb6c14302f839a1a3cb79) |
+| **Sergey Aitov** | [Improved](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/cac19b9d3b80f2c2e5a04f4510201babbd4f99b8) spikes labelling by running pretrained model on it, implemented spikes counter ([1](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/0006e78fd1479492908f15dc8922d9221a5d435c), [2](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/63431f9bfcd3c9bec4a53c665e4f9c81b52f6194))|
+| **Ekaterina Petrova** | Performed research on tire [thread depth](https://github.com/IU-Capstone-Project-2025/Kolobok/blob/032ee4c674269be130f41fff3f2107ef410912ab/ML/research/thread_depth_research/main.pdf) and implemented a [module](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/d961ab52291b071812d864540a5ebd1f6d3aa337) |
+
+---
+
+## Confirmation of the code's operability
+We confirm that the code in the main branch:
+- [x] In working condition
+- [x] Run via method described in README.md

content/docs/2025/Kolobok/week4.md

@@ -0,0 +1,154 @@
+---
+title: "Week 4"
+---
+
+# Week 4 Progress Report: Testing, CI/CD & Deployment Setup
+
+## Executive Summary
+
+This week focused on hardening our system for deployment by implementing comprehensive testing, establishing a CI/CD pipeline, and deploying the full Kolobok application to Yandex Cloud. 
+
+We introduced rigorous testing across backend APIs, ML components, and user-facing interfaces (Telegram bot and web UI). GitHub Actions was configured for CI, allowing for automated linting, test execution, and image building on every PR to `main`. A lightweight CD system was also added, automatically deploying the updated backend to our staging server.
+
+Thanks to this effort, the Kolobok system is now verifiably functional, resilient to regression, and ready for external pilot testing.
+
+---
+
+## Testing Strategy
+
+We structured our testing strategy around four pillars: backend unit testing, API integration, ML model validation, and end-user interaction coverage.
+
+### Backend (FastAPI)
+
+- **Tools**: `unittest`, `httpx`
+- **Coverage**: 100% overall
+- **Tests written**:
+  - Input validation (bad image, invalid tokens)
+  - Response formatting for successful and failed inferences
+  - Token authentication & permissions
+
+### API Integration Tests
+
+- End-to-end test cases simulating:
+  - Base64 image upload → inference → JSON response
+  - Token failure handling
+  - Spike-only and tread-only photo behavior
+- Tools: `requests`, `pytest`, custom mock images
+
+### Telegram Bot & Frontend Testing
+
+#### Telegram Bot
+
+- Covered flows:
+  - Image sent too early
+  - Cancel operation
+  - Prediction correction
+  - Token missing from config
+
+- Manual testing was conducted
+
+#### Web Platform
+
+- Covered flows:
+  - File drag/drop
+  - Predict + correct
+  - Network error fallback
+  - Invalid file warning
+
+- Testing via browser devtools and form mocking
+
+### Machine Learning Validation
+
+- **Tread Depth**:
+  - Average error: 0.62mm (on synthetic test set)
+  - Real-tire test batch: 83% within 1.0mm
+- **Spike Classifier**:
+  - FP+FN: ~7.4
+  - ROC AUC: 0.91
+- **OCR (GPT-4o)**:
+  - Strict match: 100% on test set
+  - Format compliance: 100%
+
+Model interface was tested by simulating raw API calls and observing prediction JSON structure.
+
+---
+
+## CI/CD Setup
+
+### CI Configuration
+
+CI is set up via GitHub Actions:
+
+- Triggers: PR/push to `main`
+- Steps:
+  - Install dependencies
+  - Run formatting/linting
+  - Run tests (`pytest`)
+
+## Environment Setup 
+
+We successfully deployed our application to Yandex Cloud (using free trial plan). 
+
+Note: our service requires high computing power, which implies huge costs. We will power on the service only during demo.
+
+### Staging Details
+
+* Services exposed:
+  * `/api/v1/analyze_tread`
+  * `/api/v1/identify_tire`
+
+### Stack Summary
+
+| Component  | Technology                      |
+| ---------- | ------------------------------- |
+| Backend    | FastAPI + Docker                |
+| Bot        | `python-telegram-bot`           |
+| ML Models  | PyTorch, callable endpoints     |
+| Deployment | GitHub Actions + Docker Compose |
+| Monitoring | Logs + manual probes            |
+
+---
+
+## Code Coverage Report
+
+We have implemented autolinting and autotesting for ML backend and JS frontend parts of our codebase.
+Current state of the codebase coverage:
+
+| Service  | Status       | 
+| ------------ | ------------ |
+| Backend (linting)    | ✅ 100% |
+| Backend (tests)    | ✅ 100%  | 
+| Frontend (tests)      | ✅ 100%  | 
+
+---
+
+## PM Team Vibe Check
+
+| Team Member  | Status       | Note                                            |
+| ------------ | ------------ | ----------------------------------------------- |
+| Nikita M.    | ✅ Engaged    | Coordinating next week’s strategy          |
+| Nikita Z.    | ✅ Motivated  | Finalized model testing and validation          |
+| Vlad S.      | ✅ Energized  | Resolved Docker network bug                     |
+| Sergey A.    | ✅ Positive   | Led coverage push for backend                   |
+| Ekaterina P. | ✅ Focused    | Finished test loop for frontend                 |
+| Darya S.     | ✅ Productive | Validated bot UX with edge cases                |
+| Dmitry T.    | ✅ Curious    | Opened PR for synthetic validation overlay tool |
+
+## Team Contributions
+
+| Team Member            | Contributions                                            |
+| ---------------------- | -------------------------------------------------------- |
+| **Nikita Menshikov**   | Wrote the report, pitched CI/CD planning, ran team vibe check |
+| **Nikita Zagainov**    | Added [tests](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/525d63a61452e9ad0815c060b0779ae282f800d6) to ML pipeline, facilitate [logging](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/399f2d04cc62a2e93aa5acce3fed7f71b2d32367) in the backend, conducted [experiments](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e745eec8031d679f587587bb61d7e1c0d19f0b66) on enhancing tread depth recognition  |
+| **Dmitry Tetkin**      | Assisted [research](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/07d1f331cf621ee1823b4ae1c9b6c321c00fa235) on how synthetic dataset influence precision, helped [integration](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/14c7b72d1a9c209696eedca49963cb8d97fa9925) synthetic data into pipeline, developed MacOS desktop [app](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/bac0805aab35fdb2466d17a27e5a5f7dcddf3903) |
+| **Vladislav Strelkov** | Built full CI [pipeline](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/e1fe85b0771bb09a30fd453578bd7603ee84242b#diff-d398e54a64ae971408747d5cec234a6e2f610c8c8517e98b6fad3da618c97f02), [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/812d7ccc87561480c8e0a0640e50427a625b5d6a) in depth evaluation research          |
+| **Sergey Aitov**       | [Built](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/143feba243b9d5280a7d41c4b306ae0371dc1ad7) OCR MVP                   |
+| **Ekaterina Petrova**  | [Enhanced](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/8bf464be7c85b54369a3e6c526f714fbf1e24ece) bot logging and authentification, [assisted](https://github.com/IU-Capstone-Project-2025/Kolobok/commit/36eb54d806a458b5fe413633c27f090cd4eca42c) in shaping the final version of the pipeline        |
+| **Darya Stepanova**    | Webpage frontend and backend huge [update](https://github.com/IU-Capstone-Project-2025/Kolobok/commits/my-frontend-update/)    |
+
+---
+
+## Confirmation of System Operability
+
+* ✅ All tests passing locally and in CI
+* ✅ Docker builds succeed for all services