warp-transducer

A fast parallel implementation of RNN Transducer (Graves 2013 joint network), on both CPU and GPU.

GPU implementation is now available for Graves2012 add network.

GPU Performance

Benchmarked on a GeForce GTX 1080 Ti GPU.

T=150, L=40, A=28	warp-transducer
N=1	8.51 ms
N=16	11.43 ms
N=32	12.65 ms
N=64	14.75 ms
N=128	19.48 ms

T=150, L=20, A=5000	warp-transducer
N=1	4.79 ms
N=16	24.44 ms
N=32	41.38 ms
N=64	80.44 ms
N=128	51.46 ms

Interface

The interface is in include/rnnt.h. It supports CPU or GPU execution, and you can specify OpenMP parallelism if running on the CPU, or the CUDA stream if running on the GPU. We took care to ensure that the library does not preform memory allocation internally, in oder to avoid synchronizations and overheads caused by memory allocation. Please be carefull if you use the RNNTLoss CPU version, log_softmax should be manually called before the loss function. (For pytorch binding, this is optionally handled by tensor device.)

Compilation

warp-transducer has been tested on Ubuntu 16.04 and CentOS 7. Windows is not supported at this time.

First get the code:

git clone https://github.com/HawkAaron/warp-transducer
cd warp-transducer

create a build directory:

mkdir build
cd build

if you have a non standard CUDA install, add -DCUDA_TOOLKIT_ROOT_DIR=/path/to/cuda option to cmake so that CMake detects CUDA.

Run cmake and build:

cmake -DCUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME ..
make
sudo make install #optional

if it logs

-- cuda found TRUE
-- Building shared library with no GPU support

please run rm CMakeCache.txt and cmake again.

The C library should now be built along with test executables. If CUDA was detected, then test_gpu will be built; test_cpu will always be built.

Troubleshooting: cuda_runtime_api.h: No such file or directory

If you encounter an error like this during installation:

fatal error: cuda_runtime_api.h: No such file or directory

This usually means the CUDA headers can't be found by the compiler. If your system has CUDA installed (e.g., version 12.9) and the file cuda_runtime_api.h exists somewhere like:

/usr/local/cuda-12.9/targets/x86_64-linux/include/cuda_runtime_api.h

Then you can fix this by manually setting the environment variables to point to the correct CUDA paths:

export CUDA_HOME=/usr/local/cuda-12.9
export CFLAGS="-I$CUDA_HOME/targets/x86_64-linux/include"
export LDFLAGS="-L$CUDA_HOME/targets/x86_64-linux/lib"
export PATH="$CUDA_HOME/bin:$PATH"
export LD_LIBRARY_PATH="$CUDA_HOME/lib64:$LD_LIBRARY_PATH"

Test

To run the tests, make sure the CUDA libraries are in LD_LIBRARY_PATH (DYLD_LIBRARY_PATH for OSX).

Contributing

We welcome improvements from the community, please feel free to submit pull requests.

Reference

• Sequence Transduction with Recurrent Neural Networks
• SPEECH RECOGNITION WITH DEEP RECURRENT NEURAL NETWORKS
• Baidu warp-ctc
• Awni implementation of transducer