Fastapi #94

sfoale · 2025-06-09T15:16:22Z

Summary

• Implement complete FastAPI backend to replace legacy Flask application with modern async architecture
• Add comprehensive test coverage for all FastAPI endpoints with 11 test modules covering new code

Key Changes

New FastAPI Backend (server/)

Modern async FastAPI application with auto-generated OpenAPI documentation
SQLAlchemy 2.0+ models with PostGIS geospatial support
Pydantic schemas for request/response validation
Modular route structure organized by domain (pointing, instrument, alerts, etc.)
JWT authentication system replacing Flask-based auth

Comprehensive Test Suite (tests/fastapi/)

100% endpoint coverage across all API routes
Integration tests with database restore functionality
Test data fixtures for realistic testing scenarios

Kubernetes Integration (gwtm-helm/)

FastAPI deployment templates and configuration
Database initialization scripts for containerized environments
Development tooling with Skaffold for full-stack testing

This FastAPI implementation aims to maintain full API compatibility while providing better performance, automatic documentation, and type safety.

Refactor routes. Add README.

Fingel

I spent some time trying to get this running locally, but was ultimately unsuccessful.

I'm not clear on how the database is managed. There are some manual commands in the file database_postgresql_notes but if that is the de-facto method of getting a database set up, shouldn't it be a .sql file?

There seemed to be some syntax errors in there anyway, ($$to add enum type -> alter type bandpass add value 'other';) so even after copying the commands to a .sql file I was unable to use them to set up the database properly.

Without the database, it seems the tests will not run. Additionally, the tests appear to make HTTP requests to 2 different services. I managed to figure that one of them running on port 8000 was the fastapi service itself. But I don't know what the requests going to port 5000 are supposed to test.

All in all, I think the README needs some help.

Fingel · 2025-06-11T16:45:24Z

requirements.txt

 Flask-Mail==0.10.0
 Flask-SQLAlchemy==2.5.1
 Flask-WTF
+flask-caching


Missing fastapi

… application.

sfoale · 2025-06-12T15:10:07Z

I spent some time trying to get this running locally, but was ultimately unsuccessful.

I'm not clear on how the database is managed. There are some manual commands in the file database_postgresql_notes but if that is the de-facto method of getting a database set up, shouldn't it be a .sql file?

There seemed to be some syntax errors in there anyway, ($$to add enum type -> alter type bandpass add value 'other';) so even after copying the commands to a .sql file I was unable to use them to set up the database properly.

Without the database, it seems the tests will not run. Additionally, the tests appear to make HTTP requests to 2 different services. I managed to figure that one of them running on port 8000 was the fastapi service itself. But I don't know what the requests going to port 5000 are supposed to test.

All in all, I think the README needs some help.

Sorry about that - I've added a bit of signposting from the main readme to the fastapi readme.

With the project in flux with the intention to retire flask in favour of fastapi / svelte I wasn't concentrating so much on the top level README.

In summary, the most straightforward way to get the whole thing up and running - fastapi, database, even the flask app, is to use skaffold - just as
skaffold dev
should bring everything up. Then the tests can be run with a script (provided) - more info in the server/README.

Hope that is a bit clearer now.

Copilot

Pull Request Overview

This PR replaces the legacy Flask backend with a new async FastAPI application, adds Pydantic-based settings and JWT authentication, and integrates full Kubernetes/Helm/Skaffold deployment plus comprehensive endpoint tests.

Introduce a modern FastAPI service in server/ with SQLAlchemy models, Pydantic schemas, and JWT auth
Provide Pydantic settings in server/config.py and Dockerfile for containerization
Add Helm charts (gwtm-helm/) and Skaffold config for local and production deployments

Reviewed Changes

Copilot reviewed 147 out of 147 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
server/core/enums/bandpass.py	New `bandpass` enum values
server/core/enums/init.py	Import list for core enums (duplicate import present)
server/config.py	Pydantic `Settings` class for environment variables
server/auth/auth.py	JWT token creation, decoding, and user validation
server/auth/init.py	Auth package init
server/README.md	FastAPI backend usage and testing instructions
server/Dockerfile	Container build for FastAPI service
gwtm-helm/values.yaml	Helm values for FastAPI deployment
gwtm-helm/values-dev.yaml	Development overrides for Helm
gwtm-helm/templates/ingress.yaml	Ingress paths for `/api/v1`, docs, health
gwtm-helm/templates/fastapi/service.yaml	Kubernetes Service definition for FastAPI
gwtm-helm/templates/fastapi/deployment.yaml	Kubernetes Deployment for FastAPI with init container
gwtm-helm/templates/fastapi/configmap.yaml	ConfigMap for FastAPI environment variables
gwtm-helm/templates/backend/create-database-tables.yaml	Database setup Job
gwtm-helm/start-dev.sh	Script to start Skaffold dev environment
gwtm-helm/skaffold.yaml	Skaffold configuration for both Flask and FastAPI
gwtm-helm/restore-db	Script for restoring database dumps
gwtm-helm/fastapi-README.md	Documentation for FastAPI Helm templates
README.md	Root README with FastAPI quick-start
.dockerignore	Updated ignore rules for Docker build

Comments suppressed due to low confidence (2)

server/core/enums/bandpass.py:3

Class names should use PascalCase. Rename bandpass to Bandpass to follow Python class naming conventions.

class bandpass(IntEnum):

gwtm-helm/templates/fastapi/configmap.yaml:13

CORS_ORIGINS is expected to be a list by the Pydantic settings. Providing a single string may cause parsing errors. Consider formatting it as a JSON array or comma-separated list that your app can split.

CORS_ORIGINS: "{{ .Values.ingress.host }}"

Copilot · 2025-06-16T19:20:57Z

server/core/enums/__init__.py

+from .pointing_status import pointing_status
+from .frequency_units import frequency_units
+from .depth_unit import depth_unit
+from .frequency_units import frequency_units


Duplicate import of frequency_units detected. Remove the redundant import to clean up the module.

Suggested change

from .frequency_units import frequency_units

Copilot · 2025-06-16T19:20:57Z

server/config.py

+    SECRET_KEY: str = Field(os.urandom(16).hex(), env="SECRET_KEY")
+    JWT_SECRET_KEY: str = Field(os.urandom(16).hex(), env="JWT_SECRET_KEY")


Using os.urandom at import time sets the same default each run and may not be re-evaluated per instance. Consider using default_factory=... to generate a new random key for each instantiation.

Suggested change

SECRET_KEY: str = Field(os.urandom(16).hex(), env="SECRET_KEY")

JWT_SECRET_KEY: str = Field(os.urandom(16).hex(), env="JWT_SECRET_KEY")

def _generate_random_key() -> str:

"""Generate a random 16-byte key and return its hexadecimal representation."""

return os.urandom(16).hex()

SECRET_KEY: str = Field(default_factory=_generate_random_key, env="SECRET_KEY")

JWT_SECRET_KEY: str = Field(default_factory=_generate_random_key, env="JWT_SECRET_KEY")

cmccully · 2025-07-02T15:54:23Z

server/core/enums/wavelengthunits.py

@@ -0,0 +1,17 @@
+from enum import IntEnum
+
+class WavelengthUnits(IntEnum):


Astropy has a pretty extensive units package developed. Does it help us to use that?

cmccully · 2025-07-02T15:56:35Z

server/core/enums/bandpass.py

@@ -0,0 +1,35 @@
+from enum import IntEnum
+
+class Bandpass(IntEnum):


This feels very fragile. Should this be more dynamic and stored in the db?

Update PostGIS setup to work with existing extension in postgis schema.

cmccully · 2025-12-17T18:18:37Z

server/config.py

+    # CORS settings
+    CORS_ORIGINS: List[str] = ["*"]
+    CORS_METHODS: List[str] = ["*"]
+    CORS_HEADERS: List[str] = ["*"]


Should these be set to something that isn't * for prod?

cmccully · 2025-12-17T18:19:32Z

server/core/enums/depthunit.py

+class DepthUnit(IntEnum):
+    """Enumeration for depth units."""
+
+    ab_mag = 1


Should this also be in the db?

cmccully · 2025-12-17T18:21:27Z

server/db/database.py

+    db = SessionLocal()
+    try:
+        yield db
+    finally:


I'm a little unclear on when this finally happens. I think this may prematurely close the db connection. Should this be more of a context manager?

cmccully · 2025-12-17T18:22:41Z

server/db/models/__init__.py

@@ -0,0 +1,10 @@
+from .users import Users, UserGroups, Groups, UserActions


Why do we import these things here in the init if we don't expose them?

cmccully · 2025-12-17T18:24:39Z

server/db/models/candidate.py

+from server.core.enums.depthunit import DepthUnit as depth_unit_enum
+from geoalchemy2 import Geography
+from sqlalchemy.ext.hybrid import hybrid_property
+from ..database import Base


Importing this effectively global variable makes me nervous. I might advocate for having a get session function and leaving the separation of responsibilities to not touch that variable.

cmccully · 2025-12-17T18:25:36Z

server/db/models/candidate.py

+        self.warnings = []
+
+
+class GWCandidate(Base):


Ok. I'm seeing what you are doing now. I guess this is fine.

cmccully · 2025-12-17T18:27:00Z

server/db/models/doi_author.py

+        Returns:
+            Tuple of (success, creators_list)
+        """
+        from sqlalchemy import and_


Imports in methods make me nervous.

cmccully · 2025-12-17T18:34:04Z

server/routes/pointing/test_refactoring.py

@@ -0,0 +1,11 @@
+"""Test endpoint for refactored pointing routes."""


Does this file need to persist?

sfoale added 9 commits May 8, 2025 16:17

Adding fastapi server. Work in progress.

fe3c8e7

Add test data and instrument endpoint tests.

92ba6f6

Add pointing tests. Work in progress

4da60f9

Add candidate and gw_alert tests.

08e90ad

Add more tests.

4fd6a0b

More tests completed.

260e930

All tests complete.

76d5abc

Complete test coverage of fastapi endpoints.

93ba921

Refactor routes. Add README.

Merge branch 'master' into fastapi

f94b067

sfoale requested review from Fingel, cmccully and swyatt7 June 9, 2025 15:16

Missing test.

f7045c5

Fingel reviewed Jun 11, 2025

View reviewed changes

requirements.txt

Flask-Mail==0.10.0

Flask-SQLAlchemy==2.5.1

Flask-WTF

flask-caching

Copy link

Contributor

Fingel Jun 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing fastapi

Update README with better signposting for running and testing fastapi…

4d0c79d

… application.

sfoale requested a review from Fingel June 12, 2025 15:10

Correct paths

dde0893

cmccully requested a review from Copilot June 16, 2025 19:19

Copilot AI reviewed Jun 16, 2025

View reviewed changes

Copilot review comments.

b3e0821

cmccully reviewed Jul 2, 2025

View reviewed changes

sfoale added 7 commits July 3, 2025 06:24

Add a github action to run the fastapi tests.

a2470c2

PEP 8 formatting with black.

3203d15

fix skaffold run in github action

4a29345

Use docker compose for the testing github action.

6fdabfb

Use docker compose.

0ab1cb7

Initialiase db for github action tests.

8142eed

Create db tables from models with sqlalchemy.

71cda89

sfoale added 8 commits July 3, 2025 09:28

Python formatting.

7da8951

Add missing model imports.

e927869

Enable postgis extension in the github actions

25abaa0

Debug actions failures

233fe9e

Fix the fastapi models.

c7632f4

Debug actions failures

bf11172

Remove debug steps that were polluting test code.

a230f64

Update GitHub Action to use FastAPI database initialisation.

e06350e

sfoale force-pushed the fastapi branch from 9871ad2 to e06350e Compare July 3, 2025 12:39

sfoale added 8 commits July 3, 2025 14:00

Fix instrument_name field length from 25 to 64 characters.

e43f7cf

Update PostGIS setup to work with existing extension in postgis schema.

Clean up github action.

daa0528

Remove Redis dependency from GitHub Actions workflow

6e4e95b

Pin dependency versions to speed up docker build.

41a17d5

Pin dependency versions to speed up docker build.

867e5ec

Pin dependency versions to speed up docker build.

76939c3

Revert.

8ace7c0

PEP 8 check.

a67b246

cmccully reviewed Dec 17, 2025

View reviewed changes

		SECRET_KEY: str = Field(os.urandom(16).hex(), env="SECRET_KEY")
		JWT_SECRET_KEY: str = Field(os.urandom(16).hex(), env="JWT_SECRET_KEY")

-    SECRET_KEY: str = Field(os.urandom(16).hex(), env="SECRET_KEY")
-    JWT_SECRET_KEY: str = Field(os.urandom(16).hex(), env="JWT_SECRET_KEY")
+    def _generate_random_key() -> str:
+        """Generate a random 16-byte key and return its hexadecimal representation."""
+        return os.urandom(16).hex()
+    SECRET_KEY: str = Field(default_factory=_generate_random_key, env="SECRET_KEY")
+    JWT_SECRET_KEY: str = Field(default_factory=_generate_random_key, env="JWT_SECRET_KEY")

		@@ -0,0 +1,17 @@
		from enum import IntEnum

		class WavelengthUnits(IntEnum):

		@@ -0,0 +1,35 @@
		from enum import IntEnum

		class Bandpass(IntEnum):

		@@ -0,0 +1,10 @@
		from .users import Users, UserGroups, Groups, UserActions

		@@ -0,0 +1,11 @@
		"""Test endpoint for refactored pointing routes."""

Fastapi #94

Are you sure you want to change the base?

Fastapi #94

Uh oh!

Conversation

sfoale commented Jun 9, 2025

Summary

Key Changes

New FastAPI Backend (server/)

Comprehensive Test Suite (tests/fastapi/)

Kubernetes Integration (gwtm-helm/)

Uh oh!

Fingel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sfoale commented Jun 12, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants