Add Logging, Rate Limiting, File Utils, Notifications, and Auth with Role-Based Access #134
Closed as not planned
Description
Production-Ready Backend Template (TypeScript + Express)
Implement complete production utilities and middleware including logging, rate limiting, notification system, auth with roles, and a file system utility that supports reading XLSX/PDF, all within a structured TypeScript backend using Express.
Features
1. Logger Utility
- Centralized logging system recording info, warnings, and errors.
- Logs saved daily in separate files (
error.log,success.log,warning.log) insidesrc/log/. - Color-coded, timestamped console logs during development.
- Persistent logs for monitoring and troubleshooting in production.
2. Global Request Logging Middleware
- Logs every HTTP request with method, URL, status code, and response time.
- Captures client info such as browser user-agent and IP address.
- Integrates with Logger Utility for consistent, organized logs.
- Helps monitor API traffic and detect potential issues or abuse.
3. Rate Limiting Middleware
- Applies a limit on how many requests a single client (identified by IP address) can make in a certain time frame (e.g., 100 requests every 15 minutes).
- Protects the backend from abuse such as excessive requests, brute force attacks, or accidental overload.
- Enforces stricter limits on sensitive endpoints like login and signup to improve security against attacks like credential stuffing.
- When a client exceeds the allowed request limit, the middleware responds with a clear and user-friendly error message explaining that too many requests have been made and to try again later.
- Helps keep the server stable and available by preventing overuse and potential denial-of-service scenarios.
- Can be customized per route or applied globally depending on the backend needs.
4. File System Utility
- Provides safe and efficient file operations including reading, writing, and creating directories when needed.
- Supports reading and parsing Excel files (XLSX), making it easy to extract data from spreadsheets.
- Extracts text content from PDF files for processing or analysis.
- Handles various file types including text files, images, videos, and more, ensuring flexibility for different use cases.
- Uses Multer middleware for handling file uploads in Express, supporting multiple file types.
- Automatically organizes uploaded files into folders based on their type:
- Images (e.g., jpg, png, gif) are stored in an
images/folder. - Videos (e.g., mp4, avi) go into a
videos/folder. - Text and document files (e.g., txt, pdf, docx) are saved in a
documents/folder.
- Images (e.g., jpg, png, gif) are stored in an
- Includes helper functions to:
- Create necessary folders if they don’t exist, ensuring a proper folder structure.
- Generate unique file names to avoid conflicts.
- Implements robust error handling to gracefully manage issues such as missing files, corrupted data, or permission errors without crashing the application.
- Supports both synchronous and asynchronous file operations using modern Node.js APIs.
- Designed to be reusable and extensible for any file processing needs within the backend.
- Ensures uploaded files are validated and securely stored, preventing unauthorized file types or paths.
5. Notification System
-
Provides one easy way to send messages through many platforms.
-
Email:
Sends emails like account verification, password reset, and alerts using Nodemailer. -
SMS & WhatsApp:
Sends text and WhatsApp messages using services like Twilio to reach users quickly. -
Telegram:
Sends messages through Telegram bots to notify users or groups. -
Facebook & Instagram:
Supports sending messages via Facebook Messenger and Instagram Direct (set up for future use). -
Discord:
Sends notifications to Discord channels or users for community or team updates. -
Slack:
Sends messages to Slack channels to keep teams informed. -
More Channels:
Can be expanded to support other platforms like push notifications, Microsoft Teams, and more. -
Easy to use functions that you can call from anywhere in the app to send notifications.
-
Keeps message formats consistent and handles errors to make sure messages are sent reliably.
-
Supports message templates so you can send nice, formatted messages every time.
-
Tracks if messages were sent successfully and logs errors for easy monitoring.
6. Authentication & Role-Based Access Control (RBAC)
- JWT-based authentication for secure login and session management.
- Issues tokens on login; validates tokens on protected routes.
- Middleware restricts access by user roles (admin, user, moderator).
- Attaches user info (roles, permissions) to requests.
- Ensures sensitive operations are accessible only to authorized users.
7. Global Error Handling Middleware
- Centralized error handling to avoid crashes and unexpected behavior.
- Logs detailed error information for debugging.
- Sends consistent, user-friendly error responses to clients.
- Gracefully handles uncaught exceptions and promise rejections.
8. Application Setup (app.ts)
- Configured with security middlewares like CORS and Helmet.
- Parses JSON request bodies.
- Global request logging and rate limiting applied.
- API routes versioned under
/api/v1. - Global error handling middleware attached last.
- Designed for modularity and ease of extension.
9. Server Startup (server.ts)
- Loads configuration from environment variables.
- Starts Express server on configured port.
- Logs startup information.
- Handles unhandled promise rejections and uncaught exceptions gracefully.
10. Centralized Configuration
- Sensitive info and settings managed via environment variables (
.env). - Supports multiple environments: development, staging, production.
- Typed config access across the codebase for safety and clarity.
Why Use This Setup?
- Security: Protect backend with authentication, rate limiting, and security headers.
- Reliability: Central error handling and persistent logging improve uptime.
- Maintainability: Modular, clean codebase that's easy to understand and extend.
- Scalability: Ready to handle growing user traffic and data safely.
- User Communication: Multi-channel notifications improve engagement and alert users effectively.
Acceptance Criteria
- Logger utility implemented and integrated.
- Request logging middleware active and logs all requests.
- File system utility supports XLSX and PDF processing, with organized file uploads.
- Rate limiting applied globally and on sensitive routes.
- Notification system supports email, SMS, WhatsApp, and is extensible.
- JWT authentication and role-based access middleware working correctly.
- Global error handler catches and responds to all errors gracefully.
- Express app configured securely with necessary middleware.
- Server starts using environment-driven configuration and logs startup info.
- Sensitive information managed securely via environment variables.
Log File Structure
Logs are stored inside the src/log/ directory, separated by purpose for easier maintenance:
error.log— Records errors and critical issues.success.log— Tracks successful operations and info messages.warning.log— Stores warnings and important notices.
Metadata
Metadata
Assignees
Type
Projects
Status