axe DevTools for Web CLI Pre-commit Hook Testing Project

A React web application project designed to test and demonstrate how to use the --exit argument in axe DevTools for Web CLI to enforce accessibility quality gates in Git pre-commit hooks.

This approach can also be adapted for use in other parts of the CI/CD pipeline to enforce accessibility standards automatically.

Project Overview

This project serves as a testing ground for Git pre-commit hooks that:

1. Automatically starts a React development server if not already running
2. Runs accessibility tests using axe-core CLI against the running application
3. Generates comprehensive reports in HTML, CSV, and XML formats
4. Blocks commits if accessibility violations are found

Prerequisites

Before setting up this project, ensure you have the following installed:

• Node.js (version 14 or higher)
• npm or yarn package manager
• axe DevTools for Web CLI - See Installing axe DevTools for Web CLI
• Git for version control

Project Setup

1. Clone the repository:

   git clone https://github.com/jeremy-morgan-deque/axe-cli-pre-commit
   cd axe-cli-pre-commit

2. Install dependencies:

   npm install

3. Install axe DevTools for Web CLI: Follow the Installing axe DevTools for Web CLI for your platform

   Note: You may need to adjust the path to the axe executable in the script. The example below assumes it is named axe and is in your path.

Pre-commit Hook Setup

Automatic Setup

The pre-commit hook is already configured in this repository. If you need to set it up manually or on a new system:

1. Ensure the hook file exists:

   ls -la .git/hooks/pre-commit

2. Make the hook executable:

   chmod +x .git/hooks/pre-commit

Manual Setup (for new repositories)

If you want to use this pre-commit hook in a different project:

1. Create the hook file:

   touch .git/hooks/pre-commit
   chmod +x .git/hooks/pre-commit

2. Copy the hook content (see Pre-commit Hook Code section below)

Pre-commit Hook Code

The complete pre-commit hook (/.git/hooks/pre-commit):

#!/bin/bash

###
### Start React dev server
###

echo "Pre-commit hook: Checking if React dev server is running..."

# Check if server is running on port 5173
if lsof -Pi :5173 -sTCP:LISTEN -t >/dev/null ; then
    echo "React dev server is already running on port 5173"
else
    echo "React dev server is not running. Starting it now..."
    
    # Start the dev server in the background
    npm run dev > /dev/null 2>&1 &
    
    # Wait a moment for the server to start
    sleep 3
    
    # Check if it started successfully
    if lsof -Pi :5173 -sTCP:LISTEN -t >/dev/null ; then
        echo "React dev server started successfully on port 5173"
    else
        echo "Failed to start React dev server"
        echo "Please check your project setup and try again"
        exit 1
    fi
fi

###
### Start axe CLI tests
###

echo "Running accessibility tests with axe CLI..."

# make sure the results directory is clean
rm -rf ./axe-results-pre-commit/*

# Individual test URL (This could also be a loop for multiple URLs)
TEST_URL="http://localhost:5173"

#
# Run axe CLI test
#
# Note: --exit (or -q) is used to generate the exit code needed for the quality gate
#       > /dev/null 2>&1 is appended to clean up the stack trace output from the axe CLI
#
axe $TEST_URL --dir ./axe-results-pre-commit/json --exit > /dev/null 2>&1

# store the error code
AXE_EXIT_CODE=$?

# Debug error code (commented out for now)
#echo "AXE_EXIT_CODE: $AXE_EXIT_CODE"

# Check exit code for a11y issues
case $AXE_EXIT_CODE in
    1)
        echo "Accessibility errors: $TEST_URL"
        ;;
    *)
        echo "ERROR: Issue with axe CLI, exiting..."
        echo "URL: $TEST_URL"
        exit 2
        ;;
esac

###
### Post AXE CLI tests
###

# Write HTML, CSV, and jUnit XML reports from all axe CLI tests JSON files
axe reporter ./axe-results-pre-commit/json ./axe-results-pre-commit/reports --format=html,csv,xml

# Handle error conditions
if [ $AXE_EXIT_CODE -ne 0 ]; then
    echo "Accessibility errors found - blocking commit"
    exit 1
fi

exit 0

How It Works

Pre-commit Process

1. Server Check: The hook checks if a React dev server is running on port 5173
2. Server Start: If no server is running, it starts one automatically
3. Accessibility Testing: Runs axe-core CLI tests against the running application
4. Report Generation: Creates detailed reports in multiple formats
5. Commit Decision: Allows or blocks the commit based on test results

Exit Codes

• 0: All tests passed, commit proceeds
• 1: Accessibility violations found, commit blocked
• 2: Technical error with axe CLI, commit blocked

Running the Application

Development Server

npm run dev

The app will be available at http://localhost:5173

Available Scripts

• npm run dev - Start development server
• npm run build - Build for production
• npm run preview - Preview production build locally

Testing the Pre-commit Hook

1. Make a change to any file in the repository
2. Stage the change:

   git add .

3. Attempt to commit:

   git commit -m "Test commit"

4. Observe the hook output - it will automatically run accessibility tests

Accessibility Reports

When accessibility tests run, reports are generated in:

• JSON: ./axe-results-pre-commit/json/ - Raw test data
• HTML: ./axe-results-pre-commit/reports/ - Human-readable reports
• CSV: ./axe-results-pre-commit/reports/ - Spreadsheet format
• XML: ./axe-results-pre-commit/reports/ - jUnit format for CI/CD

Project Structure

├── .git/hooks/pre-commit     # Pre-commit hook script
├── .gitignore               # Git ignore rules
├── index.html               # Main HTML file
├── package.json             # Dependencies and scripts
├── vite.config.js           # Vite configuration
├── src/
│   ├── App.jsx              # Main React component
│   └── index.jsx            # Application entry point
└── axe-results-pre-commit/  # Generated accessibility reports
    ├── json/                # Raw JSON test results
    └── reports/             # Formatted reports (HTML, CSV, XML)

Troubleshooting

Common Issues

1. Hook not executing: Ensure the hook file is executable (chmod +x .git/hooks/pre-commit)
2. axe CLI not found: Install axe DevTools for Web CLI following the Installing axe DevTools for Web CLI
3. Server won't start: Check if port 5173 is already in use
4. Permission errors: Ensure you have write permissions in the project directory

Debugging

Uncomment the debug line in the pre-commit hook to see exit codes:

echo "AXE_EXIT_CODE: $AXE_EXIT_CODE"

Technologies Used

• React 18 - JavaScript library for building user interfaces
• Vite - Fast build tool and development server
• axe DevTools for Web CLI - Accessibility testing automation
• Bash - Shell scripting for Git hooks
• Git Hooks - Automated workflow triggers