Dockerised and added ui for all the models- added github actions/unit tests#1
Open
Ramkumar78 wants to merge 34 commits intoPerceptron42:mainfrom
Open
Dockerised and added ui for all the models- added github actions/unit tests#1Ramkumar78 wants to merge 34 commits intoPerceptron42:mainfrom
Ramkumar78 wants to merge 34 commits intoPerceptron42:mainfrom
Conversation
- Restructured codebase into `app/` (api, core, models) and `frontend/`. - Implemented FastAPI backend with endpoints for data, features, training, and prediction. - Developed React frontend for visualizing predictions and managing the pipeline. - Added Docker support for both backend and frontend. - Added GitHub Actions for CI (Tests, Linting, Security). - Refactored feature engineering to be importable and callable. - Added unit tests for feature generation and API routes.
…3559592349294111
- Added Docker Deployment section to README. - Verified frontend functionality with Playwright. - Fixed MUI v7 breaking changes in frontend code (Grid component). - Added comprehensive unit tests for feature engineering modules (technical, volume, volatility). - Added GitHub Actions workflow for linting and security scanning.
…3559592349294111 Merge branch 'main' into jules-9763559592349294111-f09d91ca
…3559592349294111 Finalize Docker setup, verify frontend, and enhance tests
…53460946646700 Resolve unit test failures and security hardening
…22657878343450 Update README with Docker access instructions
- Added libgomp1 to Dockerfile to fix LightGBM runtime crash. - Created .github/workflows/docker-build-test.yml to test the build and training pipeline in CI.
…72890021 Fix LightGBM Docker crash and add CI pipeline test
- Add `pytest` and `pytest-cov` dependencies for testing. - Fix `_hurst_exponent` calculation in `app/core/regime_dependent_features.py` by removing incorrect 2.0 multiplier. - Add `tests/test_additional_coverage.py` to cover `save_data` and `_hurst_exponent` logic. - Verify all tests pass.
- Updated `app/api/main.py` to support dynamic model selection (`/predict`, `/metrics`, `/train/{model}`).
- Updated `frontend/src/App.tsx` and components to include a model selector and better visualization.
- Added `recharts` for metrics visualization.
- Adapted UI components to Material UI v7 Grid syntax.
- Updated CSP to allow backend connection in dev mode.
…651804970514867864 Support multiple models in UI and Backend
Added `os.makedirs(OUTPUT_DIR, exist_ok=True)` to the main execution block of `modelling_lightgbm.py`, `modelling_xgboost.py`, `modelling_catboost.py`, and `modelling_ensemble.py`. This ensures that the required output directories are created before the scripts attempt to save any files, preventing crashes when running the training via the API.
…6842290686057263031 Fix OSError: Ensure output directories exist before training models
The `modelling_xgboost.py` script was failing to save the `metrics.json` file for the selected model, causing the `/metrics` endpoint to return a 404 Not Found error. This change ensures that the evaluation metrics for the selected model (top 40 features) are correctly serialized and saved to `output/models/xgboost/metrics.json` at the end of the training process. This fixes the issue where users would see a 404 error when accessing `/metrics?model=xgboost` after training.
…8065575854695 Fix 404 error on metrics endpoint for XGBoost model
Added `__init__.py` to `app/` and `tests/` directories. This ensures that `app` is recognized as a package and `pytest` correctly adds the project root to `sys.path` when running tests, allowing imports like `from app.core...` to resolve.
…2796341867463 Fix pytest import error by adding __init__.py
- Create `.github/workflows/ci-coverage.yml` to run tests, coverage, and security scans (Bandit, pip-audit). - The workflow installs system dependencies for TA-Lib and LightGBM (libgomp1). - Create `.coveragerc` to configure coverage settings.
…9510412732 Add CI coverage workflow and configuration
… loading - Fix `IndexError` in `app/api/main.py` for CatBoost predictions by correctly accessing the probability array (`[0][1]`). - Move `SoftVotingEnsemble` class to `app/core/ensemble.py` to ensure it can be imported by both the training script and the API, resolving pickling/loading issues. - Update `app/models/modelling_catboost.py` to save `metrics.json` so the `/metrics` endpoint works correctly. - Update `app/models/modelling_ensemble.py` and `app/api/main.py` to import `SoftVotingEnsemble` from the new core module.
…on-1842224089356292107 Fix CatBoost prediction crash and refactor Ensemble model loading
- Backend: Added `custom_pipeline.py` for end-to-end processing (download, features, train).
- API: Added `/custom/train` and `/custom/status/{job_id}` endpoints.
- Frontend: Added "Custom Asset Lab" tab with input form and status tracking.
- Tests: Added `test_custom_pipeline.py`.
- Fix: Resolved Material UI Grid2 import issue by using standard Grid v2 export.
- Backend: Added pytest, pytest-cov, pytest-bdd. - Backend: Implemented BDD feature files and step definitions for Prediction, Training, and Feature Engineering. - Backend: Added comprehensive unit tests for API and Core logic with coverage > 60% for API. - Frontend: Added Vitest, JSDOM, and React Testing Library. - Frontend: Added component tests for Dashboard, PredictionCard, and TrainingControl. - Frontend: Updated vite.config.ts for test configuration. - Dependencies: Updated requirements.txt and package.json.
…tion-7403265774028534633 Implement Comprehensive Unit Testing Framework (Backend & Frontend)
…18160639686 Add Custom Asset Training Lab
- Migrated all unit test logic to Gherkin feature files and Python step definitions.
- Created `api.feature`, `training.feature`, `math_logic.feature`, `custom_pipeline_flow.feature`, `data_download.feature`, `feature_engineering.feature`, `prediction.feature`, `custom_pipeline.feature`.
- Implemented corresponding step definitions in `tests/step_defs/`.
- Ensured 100% coverage of logic previously in `tests/unit/`, including:
- Detailed mathematical verification (RSI, Hurst, etc.).
- Custom pipeline execution flow.
- Full API endpoint coverage (training, custom jobs, predictions).
- Error handling scenarios (500s, 404s).
- Deleted legacy `tests/unit/` directory after verification.
- Added `pytest-bdd` to `requirements.txt`.
…35002609 Migrate Backend Tests to BDD Framework
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.