Potential Minor Speedups

[PyWoody]

Hi,

I've been using faster-whisper with great results and I wanted to share the minor tweaks I've made to the library for some slight speed increases. The speed increases are anywhere from 1-5% without any slowdowns. The larger the audio file with a larger amount of text will give bigger speed increases. You can see the tweaks in my branch here. I'd be glad to make a PR of any of the tweaks you want integrated but feel free to do them yourself as they're so minor.

The rundown of changes is essentially: only assigning dtype once as assigning things in Python is costly, creating dicts with {} as it is always faster than with dict(), and converting suppress_tokens to a set right off the bat. The last one removed the list assertion check but GIGO and assertions are ignored in optimized modes anyway.

[Purfview]

only assigning dtype once as assigning things in Python is costly

The change is only 22 ms faster with 2 h audio.

creating dicts with {} as it is always faster than with dict()

You'd need zillions of iterations to even notice a difference. This lib is not a good case for it.
I think a similar patch was rejected before.

and converting suppress_tokens to a set right off the bat

At a quick glance, your change breaks the original functionality.

Anyway, I benchmarked your fork with 20 runs on 2 h audio.
Result: Your fork is 0.0553% slower

[PyWoody]

Anyway, I benchmarked your fork with 20 runs on 2 h audio.
Result: Your fork is 0.0553% slower

That's surprising. I've tested and re-tested on my machine (M1 Mac) and on a Ubuntu server and I consistently got 1-5% speedups without ever once being slower. Ain't that just the way it always goes, though, huh? I wouldn't have opened the issue if I thought there were chances of slowdowns.

Oh, well, I'll continue using my fork on my local. Just thought I should pass it along.

At a quick glance, your change breaks the original functionality.

You're right. I should have done suppress_tokens = set(suppress_tokens) if suppress_tokens else set(). I overlooked None or empty strings being options. As it is now, the function does a lot of list-like operations then converts it to a set, sorts it, then converts it to a tuple at the end. I just short-circuited converting it to a set.

Looking at the type hints, I think they might be backwards for the function? It claims suppress_tokens: Tuple[int] yet does an assertion check for list and claims it returns Optional[List[int]] when it actually returns a tuple. I'm not really a big type-hints person so I may have my own wires crossed there.

Feel free to close out the issue.

[Purfview]

Tested on another system (Ubuntu server)
Your fork is 0.234% slower

Maybe something to do with different settings. I used this:

from faster_whisper.transcribe import WhisperModel, BatchedInferencePipeline
import time

model = WhisperModel("tiny", device="cuda", compute_type="float16")
model = BatchedInferencePipeline(model=model)

durations = []

for run in range(1, 11):
    start_time = time.time()
    
    segments, _ = model.transcribe("audio_2h.wav", beam_size=1, word_timestamps=True, without_timestamps=False)
    for segment in segments:
        pass

    durations.append(time.time() - start_time)
    del segments

avg = sum(durations) / len(durations)
print("Average runtime over %d runs: %.10f seconds" % (len(durations), avg))

[PyWoody]

I made a few tweaks: used the "hotwords.mp3" file in the test data, used time.perf_counter instead of time.time, changed it to use cpu and int8 as I don't have a CUDA environment setup at the moment, and increased it to 100 iterations to remove potential noise.

import time

from faster_whisper.transcribe import WhisperModel, BatchedInferencePipeline

model = WhisperModel("tiny", device="cpu", compute_type="int8")
model = BatchedInferencePipeline(model=model)

durations = []

for run in range(1, 101):
    start_time = time.perf_counter()
    
    segments, _ = model.transcribe(
        'faster-whisper/tests/data/hotwords.mp3',
        beam_size=1,
        word_timestamps=True,
        without_timestamps=False
    )
    for segment in segments:
        pass

    durations.append(time.perf_counter() - start_time)
    del segments

avg = sum(durations) / len(durations)
print("Average runtime over %d runs: %.10f seconds" % (len(durations), avg))

master: Average runtime over 100 runs: 1.0179613251 seconds
speedup: Average runtime over 100 runs: 1.0148073129 seconds

I've run it a few times and it's been about the same every time.

Regardless, it does not seem that these changes are wanted, so I'm fine with closing the comment and moving on.