Name	Name	Last commit message	Last commit date
parent directory ..
agents	agents
config	config
evaluation	evaluation
game	game
models	models
tests	tests
training	training
utils	utils
PR_DESCRIPTION.md	PR_DESCRIPTION.md
README.md	README.md
dqn_training_progress.png	dqn_training_progress.png
requirements.txt	requirements.txt

🐦 FlapAI: Flappy Bird Learns to Fly

An AI-powered Flappy Bird game where neural networks learn to survive using Neuroevolution (NEAT) and Deep Q-Learning (DQN)

🎯 Overview

FlapAI is an advanced implementation of the classic Flappy Bird game enhanced with artificial intelligence. The project demonstrates two cutting-edge AI approaches:

Neuroevolution (NEAT) - Evolutionary algorithm that evolves neural networks
Deep Q-Learning (DQN) - Reinforcement learning with deep neural networks

The AI agents learn to play Flappy Bird by observing the game state and making decisions to maximize their survival time and score.

🎮 Demo

Watch AI agents learn to play Flappy Bird in real-time:

NEAT Agent: Evolves through generations, improving survival strategies
DQN Agent: Learns optimal actions through trial and error
Random Agent: Baseline for comparison
Human Agent: Manual control for testing

🚀 Features

🎯 Core Features

Complete Flappy Bird Game: Full game implementation with physics, collision detection, and scoring
Dual AI Approaches: NEAT and DQN implementations for different learning strategies
Real-time Visualization: Watch AI agents play and learn in real-time
Performance Tracking: Comprehensive statistics and progress monitoring
Model Persistence: Save and load trained models
Headless Training: Fast training without graphics for efficiency

🧠 AI Capabilities

State Encoding: Intelligent game state representation for AI input
Fitness Functions: Sophisticated evaluation metrics for NEAT
Experience Replay: Efficient learning with DQN
Population Management: Advanced NEAT population handling
Epsilon-Greedy Policy: Balanced exploration vs exploitation

📊 Analysis Tools

Training Progress Plots: Visualize learning curves
Performance Comparison: Compare different AI approaches
Statistics Dashboard: Detailed metrics and analytics
Agent Evaluation: Comprehensive testing framework

📦 Installation

Prerequisites

Python 3.8 or higher
pip (Python package installer)

Step-by-Step Installation

Clone the repository

git clone https://github.com/yourusername/FlapAI.git
cd FlapAI

Create a virtual environment (recommended)

python -m venv venv

# On Windows
venv\Scripts\activate

# On macOS/Linux
source venv/bin/activate

Install dependencies
```
pip install -r requirements.txt
```

🐛 Troubleshooting

Windows Users:

# If pygame installation fails, try:
pip install pygame --pre

Python 3.13 Users:

# Install packages individually if needed
pip install numpy matplotlib pytest pytest-cov
pip install pygame
pip install neat-python torch

🎮 Quick Start

1. Train a NEAT Agent

python training/train_neat.py --generations 50 --population-size 50

2. Train a DQN Agent

python training/train_dqn.py --episodes 1000 --epsilon 0.9

3. Watch Trained Agents Play

python evaluation/eval_visualize.py --agent-type neat --model-path models/best_neat_gen_final.pkl

4. Play Manually

python game/flappy_bird.py --human

📚 Usage Guide

🧬 NEAT Training

NEAT (Neuroevolution of Augmenting Topologies) evolves neural networks through generations:

from training.train_neat import train_neat

# Train NEAT agent
agent = train_neat(
    generations=50,
    population_size=50,
    fitness_threshold=1000,
    config_file="config/neat-config.txt"
)

Key Parameters:

generations: Number of evolutionary generations
population_size: Size of the neural network population
fitness_threshold: Target fitness score to stop training
config_file: NEAT configuration file path

🧠 DQN Training

Deep Q-Learning uses experience replay and target networks:

from training.train_dqn import train_dqn

# Train DQN agent
agent = train_dqn(
    episodes=1000,
    epsilon=0.9,
    epsilon_decay=0.995,
    learning_rate=0.001
)

Key Parameters:

episodes: Number of training episodes
epsilon: Initial exploration rate
epsilon_decay: Rate of exploration decay
learning_rate: Neural network learning rate

🎯 Agent Evaluation

Compare different AI approaches:

from evaluation.eval_visualize import compare_agents

# Compare multiple agents
results = compare_agents([
    ("NEAT", "models/best_neat.pkl"),
    ("DQN", "models/best_dqn.pth"),
    ("Random", None)
])

🎮 Game Interaction

from game.flappy_bird import FlappyBirdGame
from agents.neat_agent import NEATAgent

# Create game and agent
game = FlappyBirdGame(headless=False)
agent = NEATAgent.load("models/best_neat.pkl")

# Run agent in game
while not game.done:
    state = game.get_state()
    action = agent.get_action(state)
    game.step(action)

🧠 How AI Works

🧬 NEAT Algorithm

Neuroevolution of Augmenting Topologies is an evolutionary algorithm that:

Initializes Population: Creates diverse neural networks
Evaluates Fitness: Tests each network on the game
Selection: Keeps best-performing networks
Mutation: Adds new connections and nodes
Crossover: Combines traits from parents
Speciation: Groups similar networks together

Fitness Function:

fitness = survival_time + (score * 10) + (pipes_passed * 5)

🧠 DQN Algorithm

Deep Q-Learning uses reinforcement learning:

State Observation: Bird position, velocity, pipe locations
Action Selection: Epsilon-greedy policy (jump or not)
Experience Storage: Replay buffer for learning
Q-Value Update: Neural network training
Target Network: Stable learning with separate target

State Representation:

state = [
    bird_y_normalized,
    bird_velocity_normalized,
    pipe_x_normalized,
    gap_y_normalized,
    gap_size_normalized,
    distance_to_pipe_normalized,
    bird_alive
]

🎯 State Encoding

The game state is encoded into numerical inputs:

Feature	Description	Range
Bird Y	Bird's vertical position	[0, 1]
Velocity	Bird's vertical velocity	[-1, 1]
Pipe X	Distance to next pipe	[0, 1]
Gap Y	Vertical position of gap	[0, 1]
Gap Size	Size of the gap	[0, 1]
Distance	Distance to pipe	[0, 1]
Alive	Bird alive status	{0, 1}

🏗️ Architecture

📁 Project Structure

FlapAI/
├── game/                    # Game engine
│   └── flappy_bird.py      # Main game implementation
├── agents/                  # AI agents
│   ├── base_agent.py       # Abstract base class
│   ├── neat_agent.py       # NEAT implementation
│   └── dqn_agent.py        # DQN implementation
├── training/                # Training modules
│   ├── train_neat.py       # NEAT trainer
│   └── train_dqn.py        # DQN trainer
├── evaluation/              # Evaluation tools
│   └── eval_visualize.py   # Visualization and comparison
├── utils/                   # Utilities
│   └── state_encoder.py    # State encoding utilities
├── config/                  # Configuration files
│   └── neat-config.txt     # NEAT parameters
├── tests/                   # Test suite
│   ├── test_game.py        # Game tests
│   ├── test_agents.py      # Agent tests
│   └── test_training.py    # Training tests
├── assets/                  # Game assets (images, sounds)
├── models/                  # Trained model storage
├── requirements.txt         # Python dependencies
├── README.md               # This file
└── LICENSE                 # MIT License

🔧 Core Components

🎮 Game Engine (`game/flappy_bird.py`)

Classes:

Bird: Handles bird physics and collision
Pipe: Manages pipe generation and movement
FlappyBirdGame: Main game loop and state management

Key Methods:

game.step(action)           # Advance game by one frame
game.get_state()            # Get current game state
game.reset()                # Reset game to initial state
game.render()               # Draw game to screen

🧠 AI Agents (`agents/`)

Base Agent Interface:

class BaseAgent(ABC):
    def get_action(self, state): pass
    def update(self, state, action, reward, next_state, done): pass
    def save(self, path): pass
    def load(self, path): pass

NEAT Agent:

Neural network evolution
Population management
Fitness evaluation
Genome persistence

DQN Agent:

Deep neural network
Experience replay buffer
Epsilon-greedy exploration
Target network updates

🏋️ Training Modules (`training/`)

NEAT Trainer:

Population evaluation
Generation advancement
Statistics tracking
Best agent saving

DQN Trainer:

Episode management
Experience replay
Network training
Progress visualization

📊 Performance & Results

🏆 Training Performance

NEAT Results:

Best Score: 50+ pipes passed
Training Time: 2-5 minutes for 50 generations
Population Size: 50-100 individuals
Convergence: 20-30 generations

DQN Results:

Best Score: 30+ pipes passed
Training Time: 5-10 minutes for 1000 episodes
Learning Rate: 0.001
Epsilon Decay: 0.995

📈 Learning Curves

NEAT Learning Pattern:

Generation 1-10: Random behavior, low survival
Generation 11-20: Basic avoidance, short survival
Generation 21-30: Improved timing, longer survival
Generation 31+: Optimal strategies, high scores

DQN Learning Pattern:

Episodes 1-100: Random exploration, frequent crashes
Episodes 101-500: Basic pattern recognition
Episodes 501-800: Improved decision making
Episodes 801+: Optimal policy convergence

🎯 Performance Metrics

Metric	NEAT	DQN	Random
Best Score	50+	30+	5
Avg Survival	15s	12s	3s
Learning Speed	Fast	Medium	N/A
Memory Usage	Low	Medium	Low
Training Time	2-5min	5-10min	N/A

🧪 Testing

🧪 Running Tests

# Run all tests
python -m pytest tests/ -v

# Run specific test categories
python -m pytest tests/test_game.py -v
python -m pytest tests/test_agents.py -v
python -m pytest tests/test_training.py -v

# Run with coverage
python -m pytest tests/ --cov=. --cov-report=html

📊 Test Coverage

Game Tests: 32 tests covering game mechanics
Agent Tests: 42 tests covering AI implementations
Training Tests: 92 tests covering training processes
Total Coverage: 95%+ code coverage

🐛 Test Categories

Unit Tests:

Individual class functionality
Method behavior verification
Edge case handling

Integration Tests:

Agent-game interaction
Training pipeline validation
Model persistence

Performance Tests:

Training speed benchmarks
Memory usage monitoring
Scalability validation

⚙️ Configuration

🧬 NEAT Configuration (`config/neat-config.txt`)

[NEAT]
fitness_criterion     = max
fitness_threshold     = 1000
pop_size             = 50
reset_on_extinction  = False
no_fitness_termination = False

[DefaultGenome]
# node activation options
activation_default      = tanh
activation_mutate_rate = 0.0
activation_options     = tanh

# node add/remove rates
node_add_prob           = 0.2
node_delete_prob        = 0.2

# node connection options
connection_add_prob    = 0.5
connection_delete_prob = 0.5

# network parameters
num_hidden     = 0
num_inputs     = 7
num_outputs    = 1

🧠 DQN Configuration

# Default DQN parameters
DQN_CONFIG = {
    'learning_rate': 0.001,
    'epsilon': 0.9,
    'epsilon_decay': 0.995,
    'epsilon_min': 0.01,
    'gamma': 0.99,
    'memory_size': 10000,
    'batch_size': 32,
    'target_update': 100
}

🎮 Game Configuration

# Game parameters
GAME_CONFIG = {
    'width': 800,
    'height': 600,
    'fps': 60,
    'gravity': 0.5,
    'jump_velocity': -8,
    'pipe_gap': 150,
    'pipe_frequency': 150
}

🔧 Advanced Usage

🎯 Custom Fitness Functions

def custom_fitness_function(agent, game):
    """Custom NEAT fitness function"""
    survival_time = game.frame_count / 60  # seconds
    score = game.score
    pipes_passed = game.pipes_passed
    
    # Reward survival and score, penalize crashes
    fitness = survival_time + (score * 10) + (pipes_passed * 5)
    
    # Bonus for efficient flying (less flapping)
    if hasattr(agent, 'jump_count'):
        efficiency_bonus = max(0, 100 - agent.jump_count)
        fitness += efficiency_bonus
    
    return fitness

🧠 Custom Neural Network Architectures

class CustomDQNNetwork(nn.Module):
    def __init__(self, input_size=7, hidden_size=64, output_size=2):
        super().__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, output_size)
        self.dropout = nn.Dropout(0.2)
        
    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.dropout(x)
        x = F.relu(self.fc2(x))
        x = self.dropout(x)
        return self.fc3(x)

📊 Custom Evaluation Metrics

def evaluate_agent_performance(agent, game, episodes=10):
    """Custom evaluation function"""
    scores = []
    survival_times = []
    efficiency_scores = []
    
    for episode in range(episodes):
        game.reset()
        total_reward = 0
        jumps = 0
        
        while not game.done:
            state = game.get_state()
            action = agent.get_action(state)
            game.step(action)
            
            if action == 1:  # Jump
                jumps += 1
            total_reward += game.reward
        
        scores.append(game.score)
        survival_times.append(game.frame_count / 60)
        efficiency_scores.append(game.score / max(jumps, 1))
    
    return {
        'avg_score': np.mean(scores),
        'avg_survival': np.mean(survival_times),
        'avg_efficiency': np.mean(efficiency_scores),
        'max_score': max(scores)
    }

🎮 Custom Game Modifications

class CustomFlappyBirdGame(FlappyBirdGame):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.wind_effect = 0
        self.wind_direction = 1
    
    def update_wind(self):
        """Add wind effect to bird movement"""
        self.wind_effect += 0.1 * self.wind_direction
        if abs(self.wind_effect) > 2:
            self.wind_direction *= -1
    
    def step(self, action):
        self.update_wind()
        # Apply wind effect to bird
        self.bird.velocity += self.wind_effect * 0.1
        super().step(action)

👨‍💻 Author

SK8-infi

GitHub: @SK8-infi

⭐ If you find this project helpful, please give it a star!

🐛 Found a bug? Please report it in the issues section.

💡 Have a suggestion? We'd love to hear from you!

FilesExpand file tree

FlapAI

Directory actions

More options