Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new-contrib: Audio Whisper API with Local Device Microphones #1271

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
+811 −11,016
Open

new-contrib: Audio Whisper API with Local Device Microphones #1271

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
9c75686
new-contrib: submission - Audio Whisper API with Device Microphones
CarlKho-Minerva Jul 6, 2024
117523f
chore: Updated yaml info + and typo correcting
CarlKho-Minerva Jul 6, 2024
687d905
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Jul 9, 2024
aced6cd
feat: Heavily revise article for device microphone transcription in W…
CarlKho-Minerva Aug 24, 2024
c2eaf92
chore: update authors and registry yaml files
CarlKho-Minerva Aug 24, 2024
7b53a86
chore: *correctly* update yaml files
CarlKho-Minerva Aug 24, 2024
3ab74f5
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Aug 24, 2024
5318114
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Aug 28, 2024
64a9070
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Sep 12, 2024
7786f07
indent fix
CarlKho-Minerva Sep 14, 2024
75e8e52
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Sep 21, 2024
587df2b
Merge branch 'main' into carl-kho/Whisper_API-device_mic_transcription
CarlKho-Minerva Sep 26, 2024
e81cc13
fix: clean up merged sections and remove conflict markers in registry…
CarlKho-Minerva Nov 27, 2024
e5d4800
Merge branch 'carl-kho/Whisper_API-device_mic_transcription' of https…
CarlKho-Minerva Nov 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions authors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,8 @@ justonf:
name: "Juston Forte"
website: "https://www.linkedin.com/in/justonforte/"
avatar: "https://avatars.githubusercontent.com/u/96567547?s=400&u=08b9757200906ab12e3989b561cff6c4b95a12cb&v=4"

carlkho-minerva:
name: "Carl Vincent Ladres Kho"
website: "https://www.carlkho.com/"
avatar: "https://avatars.githubusercontent.com/u/106736711?v=4"
500 changes: 500 additions & 0 deletions examples/Whisper_transcribe_device_microphone.ipynb
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless the reader speaks Filipino they can't test this part out – how about translating from a more common second language like Spanish?

Also, an indefinite record makes many notebooks crash – set a 5-10 second limit as well.

# Demo: Transcribe lengthy Filipino speech and translate into English with proper grammar and punctuation
result = transcribe_audio(
    debug=False,
    prompt="Filipino spoken. Proper grammar and punctuation. Skip fillers.",
    timed_recording=False,
    record_seconds=0,
    is_english=False,
)

print("\nTranscription/Translation:", result)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combining transcribing and translating here is a bit weird in this function, and also drops the prompt param for translations. (The prompt should be in english for translation and language of choice in a transcription). I'd split this out into two clear helper functions for translate and transcribe.

def process_audio(file_name, is_english=True, prompt=""):
    with open(file_name, "rb") as audio_file:
        if is_english:
            response = client.audio.transcriptions.create(
                model="whisper-1", file=audio_file, prompt=prompt
            )
        else:
            response = client.audio.translations.create(
                model="whisper-1", file=audio_file
            )

        return response.text.strip()

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't this this is how we intend for the prompt parameter to be used – looking at our docs, it is more of an example(s) than an instruction.
Screenshot 2024-11-26 at 4 03 42 PM

Large diffs are not rendered by default.

16 changes: 13 additions & 3 deletions registry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1251,8 +1251,8 @@
- teomusatoiu
tags:
- moderation


- title: Summarizing Long Documents
path: examples/Summarizing_long_documents.ipynb
date: 2024-04-19
Expand Down Expand Up @@ -1323,4 +1323,14 @@
- maxreid-openai
tags:
- completions
- chatgpt
- chatgpt

- title: Using Whisper API to transcribe text using your Device Microphone
path: examples/Whisper_transcribe_device_microphone.ipynb
date: 2024-07-06
authors:
- carlkho-minerva
tags:
- whisper
- audio
- transcribe