FPGA-EEG with FINN

This repository demonstrates how to train quantized version of the Deep4 EEG model from Braindecode and deploy it onto FPGA hardware using FINN.

Prerequisites

Before starting, make sure you have:

• Vivado 2022.2 installed
• Docker installed

Set the following environment variables in your shell/profile:

• FINN_XILINX_PATH=/your/path/to/Xilinx
• FINN_XILINX_VERSION=2022.2

On my system the Xilinx directory was found at /tools/Xilinx. This project has only been tested with the 2022.2 version of Vivado.

Setup

Clone the repo and pull submodules:

git clone https://github.com/danielflood/fpga-eeg.git
cd fpga-eeg
git submodule update --init --recursive

Getting the Dataset

This project uses the EEG motor imagery (MI) dataset collected by [2]. The original raw data can be accessed here. Following the preprocessing pipeline from [3], we focus only on the motor imagery tasks.

To download the dataset, start the Docker container:

./run-docker.sh

Then run the download script:

python ./scripts/get_dataset.py

Once downloaded, preprocess the data:

python ./scripts/preprocess_data.py

After preprocessing completes, you can safely remove the raw_data/ directory to save disk space:

Running the Project

Start the FINN Docker environment with:

 ./run-docker.sh notebook

This will build the Docker image (if not already built) and launch Jupyter Lab from the notebooks directory.

From there, you can follow the flow:

1. Train the quantized Deep4 model.
2. Synthesize the trained model into a bitfile for FPGA deployment.
3. Validate the bitfile by running it on a Zynq device using PYNQ.

Repository Layout

• docker/Dockerfile.project - Dockerfile that wraps the base image and installs requirements.txt
• lib/finn/ – FINN submodule (pinned to a specific commit).
• notebooks/ – Jupyter notebooks for training and synthesis.
• run-docker.sh – Script to launch the Dockerized FINN environment.
• src/ – Supporting code (i.e. quantized model)

Notes

• Hardware builds may take significant time.
• You’ll need a supported FPGA development board (e.g. PYNQ-Z2) for deployment.

Contributing

I would really like help with:

• Running the project from a clean install and fixing any issues.
• Improving the documentation so that the tutorial is easier to follow.

Feel free to open issues and pull requests if you have any ideas on how I can improve this project.

Known Issues

• This project uses a very old version of Braindecode. You will likely get many deprecation warnings.
• In the past I've gotten a Vivado stack overflow error when running synthesis. I got around this by changing line 260 in ./lib/finn/src/finn/transformation/fpgadataflow/make_zynq_proj.py to:

  f.write("vivado -stack 10000 -mode batch -source %s\n" % ipcfg)

References

1. Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F. & Ball, T. (2017).
   Deep learning with convolutional neural networks for EEG decoding and visualization.
   Human Brain Mapping, Aug. 2017. https://doi.org/10.1002/hbm.23730
2. Lee, M.-H., Kwon, O.-Y., Kim, Y.-J., Kim, H.-K., Lee, Y.-E., Williamson, J., Fazli, S., & Lee, S.-W. (2019).
   Supporting data for "EEG Dataset and OpenBMI Toolbox for Three BCI Paradigms: An Investigation into BCI Illiteracy"
   [Data set]. GigaScience Database. https://doi.org/10.5524/100542
3. K. Zhang, N. Robinson, S.-W. Lee, C. Guan (2021)
   Adaptive transfer learning for EEG motor imagery classification with deep Convolutional Neural Network.
   Neural Networks. https://doi.org/10.1016/j.neunet.2020.12.013

Licence

MIT License — see LICENSE for details.