|
| 1 | +> **Note** |
| 2 | +> We have rebuilt the Entire architecture during a 24 hour Hackathon and brought down the latency to just 3-5 seconds. Checkout the [new repo](https://github.com/nagarajpandith/hackverse) for the latest version of the project. |
| 3 | +
|
| 4 | + |
| 5 | +[Live Demo](https://jabwemeet.vercel.app/) | [Pitch Deck](https://www.canva.com/design/DAFdhOPv1eA/qvB2ivAdB--1m9PxY-buWw/view?utm_content=DAFdhOPv1eA&utm_campaign=designshare&utm_medium=link&utm_source=publishsharelink) | [Hackoverflow 1.0](https://hack-overflow.tech/) |
| 6 | + |
| 7 | +# Jab We Meet |
| 8 | +Jab We Meet is a web application that allows users who speak different languages to converse with ease by translating the audio on the fly and speaking the translated audio in the native language of the user as well as provide them with translated captions. It also offers HD video and screen share, and can accommodate up to 100 concurrent users. The application also generates automatic meeting summaries and transcripts, making it easy for participants to review important details from the meeting. |
| 9 | + |
| 10 | +> This is the Winning solution developed at 36-hour National Level hackathon called 'Hackoverflow 1.0' by PHCET, Navi Mumbai. |
| 11 | +
|
| 12 | + |
| 13 | + |
| 14 | +## Use cases |
| 15 | +- Startups and MNCs with international work culture. |
| 16 | +- Freelancers with daily foreign communication. |
| 17 | +- Content creators & live streamers to reach global range audience. |
| 18 | +- Educational organizations to teach courses to multilingual students. |
| 19 | + |
| 20 | +## Built with |
| 21 | + |
| 22 | +<p align="left"> |
| 23 | +<img src="https://ui-lib.com/blog/wp-content/uploads/2021/12/nextjs-boilerplate-logo.png" height="50px"> |
| 24 | +<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/4c/Typescript_logo_2020.svg/1024px-Typescript_logo_2020.svg.png?20221110153201" height="50px"> |
| 25 | +<img src="https://pbs.twimg.com/profile_images/1526251266862505985/KlWqqTp1_400x400.png" height="50px"> |
| 26 | +<img src="https://pbs.twimg.com/profile_images/1504919223168077836/RSsCSpKf_400x400.jpg" height="50px"> |
| 27 | +<img src="https://trpc.io/img/logo.svg" height="50px"> |
| 28 | +<img src="https://www.svgrepo.com/show/374002/prisma.svg" height="50px"> |
| 29 | +</p> |
| 30 | + |
| 31 | +- [**Next JS**](https://nextjs.org/): React-based framework for building server-side rendered and statically exported web apps. |
| 32 | +- [**Typescript**](https://www.typescriptlang.org/): Statically typed superset of JavaScript, adds type annotations to enhance code reliability & readability |
| 33 | +- [**Livekit**](https://livekit.io/): End-to-end WebRTC infrastructure to build live video and audio applications. |
| 34 | +- [**Pusher**](https://pusher.com/): WebSockets solution for Realtime updates and bidirectional communication. |
| 35 | +- [**TailwindCSS**](https://tailwindcss.com/): Utility-first CSS framework |
| 36 | +- [**Planetscale**](https://planetscale.com/): Highly scalable, globally distributed database |
| 37 | +- [**tRPC**](https://trpc.io/): Provides a simple, type-safe way to build APIs for TS & JS |
| 38 | +- [**Prisma ORM**](https://www.prisma.io/): Modern, type-safe ORM for Node.js and TS |
| 39 | +- **APIs used**: Browser's [Webspeech](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API) API for transcribing and tts/speech synthesis. [Google-translate-browser](https://www.npmjs.com/package/google-translate-api-browser) API for translating texts, OneAI's [Summarizer](https://www.oneai.com/summarize) for summarizations of meeting transcripts. |
| 40 | + |
| 41 | +> **Note** |
| 42 | +> This project was bootstrapped with [create-t3-app](https://create.t3.gg/). Check [package.json](https://github.com/swasthikshetty10/hackoverflow/blob/main/package.json) to know all the dependencies and tech stack we used. |
| 43 | +
|
| 44 | +## Features |
| 45 | +- Multilingual Meeting Support |
| 46 | +- Real-time Translation and Transcriptions |
| 47 | +- Automatic Meeting Minutes generation |
| 48 | +- Support to upto 100 concurrent users |
| 49 | +- Supports HQ Video streaming |
| 50 | + |
| 51 | +## Architecture |
| 52 | +<img width="1073" alt="Screenshot 2023-03-18 at 11 09 44 AM" src="https://user-images.githubusercontent.com/83623339/226187854-03ec9559-1122-42a3-93c7-80614fdae396.png"> |
| 53 | + |
| 54 | +## Installation steps |
| 55 | + |
| 56 | +1. - Fork the [repo](https://github.com/swasthikshetty10/hackoverflow) |
| 57 | + - Clone the repo to your local system `git clone https://github.com/swasthikshetty10/hackoverflow.git` |
| 58 | + - Change current directory `cd hackoverflow` |
| 59 | +2. Install latest version of [Nodejs](https://nodejs.org/en/) and install all the dependencies: |
| 60 | + |
| 61 | +```bash |
| 62 | +npm install |
| 63 | +``` |
| 64 | + |
| 65 | +3. Generate prisma client |
| 66 | + |
| 67 | +```bash |
| 68 | +npx prisma generate |
| 69 | +``` |
| 70 | + |
| 71 | +4. Copy and Rename the .env.example to .env, place it in the root directory and fill the essential vars. |
| 72 | + |
| 73 | +```bash |
| 74 | +cp .env.example .env |
| 75 | +``` |
| 76 | + |
| 77 | +> **Warning** |
| 78 | +> Do not rename the original .env.example, it is used to keep track for env vars list as .env with values is gitignored |
| 79 | +
|
| 80 | +5. Run the development server: |
| 81 | + |
| 82 | +```bash |
| 83 | +npm run dev |
| 84 | +``` |
| 85 | + |
| 86 | +## Known Issues |
| 87 | +- There is latency of around 5-6 seconds between the original time and the time when translated audio is spoken out. The latency issue can be tackled by following ways: |
| 88 | + - Use native models for faster transcriptions |
| 89 | + - Use paid APIs for quicker translations like Whisper AI, Google translate etc. |
| 90 | + - Develop a learning model to predict the next moves of the speaker, which is a replacement for WebSpeech API's final transcription boolean. |
| 91 | +- As the app relies on browser's native Web Speech API for transcribing and tts, few browsers like Brave with known issues for this API and microphone functions tend to fail at running the app successfully. |
| 92 | + |
| 93 | +## Future scope of development |
| 94 | +- Use deepAI models to sync video/lip movements & translated audio. |
| 95 | +- Resemble the spoken audio(translated) to source audio(speaker) pitch. |
| 96 | + |
| 97 | +## Team Members |
| 98 | + |
| 99 | +| <img src = "https://avatars.githubusercontent.com/u/83623339?v=4" width="50px"> | <img src = "https://avatars.githubusercontent.com/u/62538932?v=4" width="50px"> | <img src = "https://avatars.githubusercontent.com/u/74966490?v=4" width="50px"> | |
| 100 | +| :-----------------------------------------------------------------------------: | :-----------------------------------------------------------------------------: | :------------------------------------------------------------------------------: | |
| 101 | +| [Nagaraj Pandith](https://github.com/nagarajpandith/) | [Swasthik Shetty](https://github.com/swasthikshetty10/](https://github.com/rudra246)>) | [Tanishka Rao](https://github.com/tanishkarao16) | |
| 102 | + |
| 103 | +## License |
| 104 | +[](https://opensource.org/licenses/Apache-2.0) |
| 105 | + |
| 106 | +This repository is licensed under [Apache License 2.0](https://github.com/swasthikshetty10/hackoverflow/blob/main/LICENSE) |
| 107 | + |
| 108 | +## Attributions |
| 109 | + |
| 110 | +- [Icon from - Flaticon](https://www.flaticon.com/free-icons/) |
0 commit comments