Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
498 lines (376 loc) · 23.5 KB

ARCHITECTURE.md

File metadata and controls

498 lines (376 loc) · 23.5 KB
Raw

Learn.tg - Project Architecture

Overview

Learn.tg is a live, production gamified educational platform making learning engaging and rewarding. Students complete quality content through interactive guides and crossword puzzles, earning cryptocurrency rewards for correct answers. The platform is currently operational at https://learn.tg with multiple courses across different subjects.

Current Game Implementation

The platform currently features crossword puzzles as the primary interactive assessment method. Each guide concludes with a crossword puzzle that tests the material learned. Future development plans include expanding to additional game types while maintaining the core educational value.

Guide Format and Parsing

Course guides are Markdown files in resources/{lang}/{prefijoRuta}/guide*.md. They contain comprehension questions as numbered lists ending with (answer):

1. The Celo native cryptocurrency is called ___ . (CELO)

The parser (lib/remarkFillInTheBlank.mjs) extracts questions and answers using regex /^(.*___.*)\s+\(([^)]+)\)\s*$/s. Key behaviors:

  • Questions with ___ become crossword clues (max 5 shown, randomly selected from all available).
  • The answer in (...) is captured — [^)]+ allows parentheses in the question body (e.g. formulas) without interference.
  • Answers are stored in billetera_usuario.answer_fib as pipe-separated values (|) and validated against user submissions.
  • Guide order is controlled by nombrecorto in cor1440_gen_actividadpf (text sort). The sufijoRuta column must match the filename.

See Guide Writing Conventions for the full style guide.

Credentials (SBTs)

When a student completes 100% of guides in a course, an SBT (Soul-Bound Token) is minted via lib/credentials.ts → mintCourseCredential(). The SBT is non-transferable proof of learning, displayed on the user's public profile. Emission records are cached in credential_emission and metadata in credential_metadata.

System Architecture Diagram

This diagram illustrates the flow of information and actions between the user, the different parts of the application, and the blockchain.

graph TD
    subgraph User
        A[User's Wallet]
        B[Browser]
    end

    subgraph Platform
        C[Frontend - Next.js/React]
        D[Backend - Rails]
        E[Smart Contract - Solidity/Celo]
        F[Backend - Next.js API]
    end

    subgraph Blockchain
        G[Celo Network]
    end

    A -- 1. Connects & Signs (SIWE) --> C
    B -- 2. Interacts with Content --> C
    C -- 3. Fetches Guides/Data (JWT Auth) --> D
    D -- 4. Returns Content --> C
    C -- 5. Submits Answers/Actions --> F
    F -- 6. Validates, Records Events, Triggers Reward --> E
    E -- 7. Executes Transaction --> G
    G -- 8. Sends USDT, SLEARN, and CELO Rewards --> A

    style E fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px
    style C fill:#9f9,stroke:#333,stroke-width:2px
    style F fill:#9f9,stroke:#333,stroke-width:2px
Loading

Service Ports

Service Port Purpose
Next.js dev server 4000 UI + API routes (local development)
Rails API 3500 Admin backend (/learntg-admin)
Next.js API (HTTPS) 9001 API proxied to live server for quickstart mode

Architecture Stack

1. Major Backend: Rails (servidor/)

  • Framework: Ruby on Rails (v8.0)
  • Database: PostgreSQL (>= 16.2) with unaccent extension
  • Purpose: Course management, guide organization, user data persistence, teacher administration.
  • Based on: MSIP and cor1440_gen frameworks
  • Authentication: Receives and validates tokens from the Next.js frontend (the same CSRF token used for SIWE, stored in billetera_usuario.token).

2. Frontend and minor backend: Next.js (apps/nextjs/)

  • Framework: Next.js with React + TypeScript.
  • UI Components: Utilizes Radix UI for building a flexible and accessible component library. Shared utilities and CLI tooling from @pasosdejesus/m (i18n, WBA metrics, E2E test harness, debug console, Kysely mocks, shadcn components, blockchain helpers).
  • Purpose: User interface, content delivery, and user authentication.
  • Authentication: Implements Sign-In With Ethereum (SIWE) with a custom window.ethereum-based wallet layer (no RainbowKit, no wagmi). See SIWE Auth Flow and Wallet Auth for details.
  • Note: For a detailed technical breakdown of the Next.js application, refer to the README.md file within the apps/nextjs directory.

3. Smart Contracts: Hardhat (apps/hardhat/)

  • Language: Solidity (^0.8.24)
  • Network: Celo (mainnet) & Celo Sepolia (testnet)
  • Contracts:
    • LearnTGVaultsV4.sol: Manages USDT and SLEARN scholarship rewards for crossword puzzle completions (active).
    • CeloUBI.sol: Manages periodic claims of Universal Basic Income (UBI) in CELO.
    • SLEARN.sol: ERC-20 utility token, 2 decimals, restricted transfers.
    • PasosDeJesusCredentials.sol: Course completion SBTs (Soul-Bound Tokens).

Authentication & Communication Flow

  1. Frontend (Next.js): A user connects their wallet and signs a message (SIWE).
  2. Next.js API: Validates the signature and generates a JWT token. This token is stored and used for subsequent authenticated actions.
  3. Backend Communication:
    • For fetching course data and content, the frontend sends the JWT in the Authorization header to the Rails backend.
    • For submitting answers and triggering rewards, the frontend sends the answers along with the JWT to a specific Next.js API route (/api/check-crossword).

Key Point: The platform uses a hybrid backend approach. The Rails server acts as the primary administrative and content backend, while the Next.js server handles real-time, user-specific actions like answer validation and blockchain interactions.

Reward System

The platform features two distinct reward mechanisms, demonstrating our principle of amor through tangible provision.

1. Activity Rewards (USDT for Crosswords)

  • Trigger: A student submits a crossword answer via the /api/check-crossword endpoint.
  • Process:
    1. The Next.js backend validates the answer.
    2. If correct, it calls the payScholarship() function on the LearnTGVaultsV4.sol contract.
    3. The contract verifies on-chain that the user has a profileScore of at least 50, has not already been rewarded for the guide, and has respected the 24-hour cooldown period. The profileScore breakdown and scholarship formula are documented in the user-facing course: Web3 & UBI — Guide 2b.
    4. If checks pass, the contract calculates and transfers USDT and SLEARN rewards to the student's wallet.
    5. When 100% of guides in a course are completed, mintCourseCredential() issues an SBT via PasosDeJesusCredentials.sol.

2. Universal Basic Income (UBI) Claims in CELO

  • Trigger: A user initiates a UBI claim via the /api/claim-celo-ubi endpoint.
  • Process:
    1. The Next.js backend verifies the user's eligibility (e.g., wallet, profileScore, potential cooldowns).
    2. It then calls the claim() function on the CeloUBI.sol contract.
    3. The contract validates the claim conditions (such as cooldown periods) on-chain.
    4. Upon successful validation, it transfers a set amount of CELO to the user's wallet.

Progress and Scoring System

The platform tracks user progress and scores through two main database tables:

guide_usuario (per-guide progress)

  • points: 1 if the guide was answered correctly, 0 otherwise
  • amountpaid: Amount rewarded for this guide — USDT if paid, otherwise SLEARN (0 if not yet paid)
  • profilescore: User's profile score at the time of submission

Global user scores (in usuario table)

  • learningscore: Total points across all guides and courses
  • profilescore: Metric of user engagement (≥50 required for rewards)

Progress calculation

When a user answers a guide correctly:

  1. guide_usuario.points is set to 1
  2. updateUserAndCoursePoints() recalculates the user's total learningscore
  3. Progress percentages are recomputed based on completed vs total guides

The frontend displays progress using a three-color arc showing completion and payment percentages.

Admin Dashboard

Verifiers access /en/admin to manage users and churches. Key features:

  • Pending Verifications: Users who requested interviews
  • Recent Users / Churches: Quick access to recently modified records
  • User Edit Modal: Edit profile fields, verify data (checkboxes), assign churches, schedule interviews
  • Church Edit Modal: Edit church details, view members, delete (soft-delete)
  • Calendar: Verifier availability managed via CalDAV (Radicale)

See Admin API for endpoint details.

Additional Database Tables

church (places of worship)
  • Managed by verifiers via /api/admin/churches
  • Linked to users via usuario.church_id
  • Tracks pastor info, location, denomination, registration
credential_emission + credential_metadata (SBT cache)
  • credential_emission: Records SBT minting events (user, course, tokenId, txHash)
  • credential_metadata: Cached on-chain metadata (name, type, image URL, premium status)

Metrics and Analytics System

In line with our principle of transparency, the project uses a robust, server-side-only metrics system to ensure data integrity and prevent client-side event manipulation.

  • Core Principle: The backend is the single source of truth. Events are not tracked by the client; they are recorded by the server as a side-effect of API calls being successfully processed. For example, a guide_view event is recorded when the /api/guide endpoint serves the guide content.
  • Implementation: The logic is handled within the Next.js backend. For detailed implementation, data flow, and the philosophy behind this approach, please refer to the README.md file in the apps/nextjs directory.

The existing Metrics Dashboard (/metrics) visualizes this reliable, server-recorded data.

Database Schema

The PostgreSQL database is shared between the Rails backend and Next.js backend. Rails manages the core schema through ActiveRecord migrations, while Next.js accesses the same tables directly via the Kysely ORM for performance-critical operations like answer validation and progress tracking.

Core Rails Tables (PostgreSQL)

cor1440_gen_proyectofinanciero (courses)

  • id: Primary key, referenced as courseId throughout the platform
  • titulo: Course title displayed to users
  • subtitulo: Optional subtitle
  • prefijoRuta: URL path prefix (e.g., "gooddollar")
  • idioma: Language ("en" or "es")
  • Additional administrative fields for project management

cor1440_gen_actividadpf (guides)

  • id: Primary key, referenced as actividadpf_id in related tables
  • proyectofinanciero_id: Foreign key to cor1440_gen_proyectofinanciero
  • titulo: Guide title
  • nombrecorto: Short name used for ordering
  • sufijoRuta: URL path suffix (e.g., "guide1") - must be non-empty for published guides
  • descripcion: Optional description

Guide Ordering and Mapping

  • Guides are filtered by proyectofinanciero_id = {courseId} and sufijoRuta IS NOT NULL AND sufijoRuta <> ''
  • In the crossword validation API (/api/check-crossword), guides are ordered by nombrecorto
  • Guide numbers (1-indexed) correspond to the position in this ordered list
  • The frontend uses sufijoRuta to construct URLs: /{lang}/{prefijoRuta}/{sufijoRuta}
  • Note: While the validation API orders by nombrecorto, other parts of the system (like the Rails backend API) may use different ordering. The guideId parameter in API requests is 1-indexed (position in ordered list), but the smart contract receives actividadpf_id (database ID from cor1440_gen_actividadpf.id). The backend translates between them via guides.rows[guideId - 1].id.

User Management Tables

usuario (users)
  • id: Primary key
  • profilescore: Engagement metric (≥50 required for blockchain rewards)
  • learningscore: Total points across all guides and courses
  • updated_at: Last update timestamp
billetera_usuario (wallet linking)
  • billetera: Wallet address (case-insensitive)
  • usuario_id: Foreign key to usuario
  • token: JWT token for API authentication
  • answer_fib: Stores correct crossword answers (pipe-separated |) provided by the Rails backend after guide viewing
guide_usuario (per-guide progress)
  • usuario_id, actividadpf_id: Composite primary key
  • points: 1 if the guide was answered correctly, 0 otherwise
  • amountpaid: USDT amount rewarded for this guide (0 if not paid)
  • profilescore: Snapshot of user's profilescore at submission time
userevent (user events)
  • id: Primary key
  • usuario_id: Optional foreign key to usuario
  • event_type: Type identifier (guide_view, game_start, etc.)
  • event_data: JSON payload with event-specific data
  • created_at: Timestamp of event
transaction (blockchain & point transactions)

Single source of truth for all value movements — both on-chain (USDT, SLEARN, CELO) and off-chain (Learning Points). Every operation that changes a user's balance or records a blockchain interaction creates an entry here.

  • id: Primary key
  • usuario_id: Foreign key to usuario
  • wallet: Wallet address that signed the transaction
  • type: Operation type — scholarship (crossword reward), donation (user gave value), donation_reward (SLEARN cashback for donating), pay-course (premium course payment), ubi-claim (CELO basic income), conversion (SLEARN ↔ Learning Points)
  • crypto: Asset — usdt, slearn, celo
  • amount: Human-readable amount (e.g., 10.00 USDT, 5.50 SLEARN)
  • balance_impact: Net effect on user's balance — negative for outflows (donations), positive for inflows (rewards, scholarships)
  • hash: Blockchain transaction hash (null for off-chain operations like Learning Points)
  • categoria: Logical grouping (e.g., 'donation' groups all donation types)
  • subcategoria: Detail within category — 'course' (course donation), 'cluster' (cluster donation), 'country' (country donation)
  • metadata: JSON payload with operation-specific data (courseId, clusterId, processPaymentHash, etc.)
  • date, created_at, updated_at: Timestamps
  • descripcion: Optional human description
  • synced: Whether synced with blockchain state

Constraints:

  • type CHECK: scholarship | donation | donation_reward | pay-course | ubi-claim | conversion
  • hash UNIQUE (prevents replay attacks)

Usage in leaderboard: Leaderboard metrics aggregate this table — SLEARN net balance via SUM(balance_impact), scholarships via SUM(amount) WHERE type='scholarship', donations via SUM(amount) WHERE type='donation'. The leaderboard does not filter by subcategoria, showing total user donation activity regardless of destination.

Data Flow Notes

  1. User wallet connection creates/updates billetera_usuario with a signed token
  2. Course and guide data is fetched from Rails APIs (which query the above tables)
  3. Crossword answers are validated against billetera_usuario.answer_fib
  4. Progress is tracked in guide_usuario
  5. Blockchain rewards update guide_usuario.amountpaid
  6. User events (guide views, game completions, etc.) are recorded in userevent via the metrics tracking system
  7. All blockchain transactions and value movements are recorded in transaction — single source of truth for leaderboard, transparency dashboard, and user transaction history

Cache Update Triggers

Policy: Derived/cached data (scores, aggregates, rankings) must be kept fresh via database triggers, not periodic cron jobs or application-level timers. A cache row is stale only between the start and commit of the triggering transaction — never longer.

Design Rules

  1. Triggers fire on data change, not on a clock. Every INSERT/UPDATE/DELETE on source tables recalculates affected caches immediately.
  2. Triggers are PostgreSQL functions defined in migrations alongside the tables they observe.
  3. Application code never writes to cache tables directly — only triggers do.
  4. Cache tables are read-only from the application layer.

Example: Cluster Score Cache (R-#154)

-- Trigger on transaction table: recalculate cluster score when SLEARN/USDT changes
CREATE OR REPLACE FUNCTION update_cluster_score_from_transaction()
RETURNS TRIGGER AS $$
BEGIN
  -- Recalculate affected cluster score cache
  PERFORM refresh_cluster_score(NEW.usuario_id);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER trg_transaction_cluster_score
  AFTER INSERT OR UPDATE ON transaction
  FOR EACH ROW
  EXECUTE FUNCTION update_cluster_score_from_transaction();

When to use triggers vs application code

Scenario Use
Aggregate score/rank from multiple tables Trigger
Single-row derived value (e.g., balance = SUM(...)) Trigger
Complex multi-step business logic with external APIs Application code (call trigger-compatible functions)
Real-time notifications (WebSocket, push) Application code after DB commit

Trigger ordering

If multiple triggers fire on the same event, they execute in alphabetical order by trigger name. Prefix triggers with a sequence number when order matters: trg_01_..., trg_02_....

Data Privacy Policy

Communications between users, and between pdJ and users, are confidential by default.

Rules

  1. Communications are not stored in the database unless explicitly required by a REQ.
  2. If stored, they must be encrypted using the pattern defined in R-#153 (AES-256-GCM with wallet-derived DEK for users, ECIES for pdJ access).
  3. API responses never include message content — only metadata (timestamps, status, participant IDs).
  4. Private notes and internal communications use the messaging system (R-#162), not free-text columns on entity tables.
  5. User-facing text fields (cluster names, course descriptions, public profiles) may be plaintext — they are intentionally public.

Sensitive data that MUST be encrypted or excluded

Data Handling
Pastor names, WhatsApp, ID documents Encrypted (R-#153)
Church locations in persecution contexts Encrypted or stored as country-only
GD contact notes, internal pdJ notes R-#162 messaging (Phase 2), not in entity tables
Private messages between users R-#162 encrypted messages
Donor identities (if anonymous) Wallet address only, no personal data

What API endpoints expose

// ⏳ Planned (REQ/157) — not yet implemented
GET /api/gd/contact/:clusterId
→ { cluster_sent_at, pdj_sent_at, gd_responded_at, released_at, status }

// ❌ Never — message content or notes
// Use R-#162 messaging endpoints instead

Quantum Readiness

Layer Algorithm Quantum resistance
Data at rest AES-256-GCM ✅ Grover's reduces to ~128-bit effective — still infeasible
User DEK derivation ECDSA (secp256k1) via personal_sign ❌ Shor's can recover private key → reconstruct DEK
pdJ DEK encryption ECIES over secp256k1 ❌ Shor's breaks ECC → DEK exposed

Impact: If a cryptographically relevant quantum computer emerges (~4000+ logical qubits), all secp256k1-based operations are compromised. This is not specific to learn.tg — it would break all Ethereum wallets, all smart contracts, and the entire Celo network simultaneously. The mitigation path is two-fold: (1) Ethereum's migration to post-quantum signatures for on-chain operations, and (2) R-#166 introduces SPHINCS+ (NIST-standardized, hash-based) at the off-chain encryption layer for learn.tg, which is independent of Ethereum's timeline. A CRQC is widely estimated at 10-20+ years out by NIST, NSA, and academic consensus.

Key Technologies

Component Tech Purpose
Major Backend Rails + PostgreSQL Course/user management, admin interface
Smart Contracts Solidity + Hardhat USDT reward distribution, vault management
Frontend & API Next.js + React + TypeScript UI, content delivery, SIWE auth, answer validation
Blockchain Celo USDT transfers, decentralized rewards
Content Markdown Guide storage, version control
Metrics & Analytics Next.js + Recharts + PostgreSQL User engagement tracking, performance metrics, dashboard visualization
Wallet OKX/Metamask/etc User authentication and reward receipt

Smart Contract Addresses

Contract addresses are stored as JSON files in apps/hardhat/deployments/{Contract}/{network}.json, not in .env variables. This is the single source of truth for all environments.

apps/hardhat/deployments/
  SLEARN/{network}.json
  LearnTGVaults/V4/{network}.json
  ClusterFunds/{network}.json
  MockUSDT/{network}.json
  CeloUbi/{network}.json

Reading addresses in scripts (apps/hardhat/scripts/):

import * as path from "path"
import * as fs from "fs"
const network = process.env.NEXT_PUBLIC_NETWORK || "celoSepolia"
const file = path.join(__dirname, "..", "deployments", "ClusterFunds", `${network}.json`)
const { address } = JSON.parse(fs.readFileSync(file, "utf8"))

Reading addresses in Next.js backend (apps/nextjs/):

Use @pasosdejesus/m/blockchain/deployments:

import { readDeployment } from "@pasosdejesus/m/blockchain/deployments"
const network = process.env.NEXT_PUBLIC_NETWORK || "celoSepolia"
const deployment = readDeployment(network,
  path.join(process.cwd(), "..", "hardhat", "deployments"),
  { contract: "ClusterFunds" }
)
const address = deployment?.address

Do NOT add NEXT_PUBLIC_*_ADDRESS environment variables for contract addresses. Environment variables are for secrets and network configuration (RPC URLs, API keys), not for derived data like deployment addresses.

Development & Deployment

  • Start order: Rails backend first, then Next.js frontend
  • Environment: adJ 7.8 recommended (includes all dependencies)
  • Testing: Hound CI, CodeClimate integration
  • Code Quality: Automated linting and security checks
  • Live Deployment: Running at https://learn.tg

Admin API Authentication

All endpoints under /api/admin/* that expose or modify sensitive data MUST require authentication. The admin dashboard's client-side wallet check is NOT sufficient — API endpoints must independently verify the caller.

Rules

  1. Every admin endpoint must read NEXT_PUBLIC_VERIFIER_WALLET and reject requests that don't include a valid verifier wallet address.
  2. The wallet address must be sent as a query parameter (?wallet=0x...) or in the request body.
  3. Calendar endpoints (/api/admin/calendar/*) are equally sensitive — they expose verifier schedules and must be protected.
  4. Read-only endpoints (GET /api/admin/users, etc.) are also sensitive (user emails, phone numbers, verification status) and require auth.

Implementation pattern

const VERIFIER_WALLETS = (process.env.NEXT_PUBLIC_VERIFIER_WALLET || '')
  .split(',').map(w => w.trim().toLowerCase()).filter(Boolean)

function isVerifier(wallet: string): boolean {
  return VERIFIER_WALLETS.length > 0 && VERIFIER_WALLETS.includes(wallet.toLowerCase())
}

Exceptions

  • GET /api/admin/check-verifier is intentionally public — it lets the frontend verify if a wallet is a verifier without exposing data.