CAPTCHA Recognition Using LSTM and CNN

This project implements a CAPTCHA recognition system using a Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) network. The model is trained to recognize CAPTCHA images, which are 5-8 length strings of alphanumeric characters generated by the python captcha package. The system generates training data, processes CAPTCHA images, and uses Connectionist Temporal Classification (CTC) loss to handle sequence prediction.

Dependencies

• Python 3.9.6
• PyTorch 2.2.2
• Pandas 2.2.3
• Pillow 11.0.0
• torchvision 0.17.2
• captcha 0.6.0

Project Structure

• main.py - Main entry point for running all operations (generate, train, evaluate)
• generate.py - Generates CAPTCHA training and test data
• training.py - Contains the model training logic
• evaluate.py - Evaluates model performance on test data
• setup.py - Contains model architecture and training parameters
• helpers.py - Utility functions for training and evaluation

Usage

The project can be run using main.py with different commands and flags:

1. Run Complete Pipeline:

   python main.py

   This will:

   • Generate training data
   • Train the model
   • Generate test data
   • Evaluate the model

2. Generate Data Only:

   python main.py g

   Flags:

   • -ge - Extends existing dataset
   • -gr - Skip generation count prompt

3. Train Model Only:

   python main.py t

   Flags:

   • -tv - Verbose output during training
   • -ts - Save training output to file
   • -tr - Resume training from saved model state

4. Evaluate Model Only:

   python main.py e

   Flags:

   • -ev - View each CAPTCHA and model attempt
   • -es - Save evaluation results to file
   • -eg - Generate new test data before evaluation

Data Generation

The system generates CAPTCHA images with the following characteristics:

• 5-8 length strings of alphanumeric characters
• Image dimensions: 280x90 pixels
• Padded with underscores to 8 characters
• Stored in Training_Data/ or Test_Data/ directories
• Labels stored in Training_Data_Mapping and Test_Data_Mappings.csv

Model Architecture

• CNN Layers: Two convolutional layers followed by max-pooling
• LSTM Layer: Captures sequential relationships in the CAPTCHA string
• Fully Connected Layer: Outputs final predictions
• Advanced Features:
  • Early stopping mechanism
  • Learning rate scheduling
  • Gradient clipping
  • Model state saving and loading

Training

The model training process includes:

• Batch processing with configurable batch size
• Learning rate scheduling for optimal convergence
• Early stopping to prevent overfitting
• Model and optimizer state saving for training continuation
• Training progress monitoring and logging options

Evaluation

The model's performance is evaluated using:

1. Character-Level Accuracy: Percentage of individual characters correctly predicted
2. Full-String Accuracy: Percentage of perfectly predicted CAPTCHA strings

Evaluation results can be:

• Displayed in real-time with visual inspection (-ev flag)
• Saved to a file (-es flag)
• Generated using fresh test data (-eg flag)

Model Files

After training, the following files are saved:

• captcha_recognition_model.pth - Trained model state
• captcha_optimizer.pth - Optimizer state for training continuation

Flag Combinations

Flags with the same prefix can be combined for more concise commands. For example, instead of -ge -gr you can use -ger

This works for any combination of valid flags under the same prefix:

• Generate flags (-g): e, r
• Training flags (-t): v, s, r
• Evaluation flags (-e): v, s, g