MicroGPT

Pure Python GPT-style transformer for generating 16x16 pixel faces.

Trains on Olivetti faces dataset and generates new face samples using autoregressive sampling.

Model

• 2-layer transformer with 4-head attention
• ~12.8K parameters
• Trained for 1000 steps (~12 minutes)

Output

• Training loss curve
• 16 generated faces
• Real vs generated comparison