Skip to content

🎨 An app to detect color palettes in the real world - powered by VisionCamera

License

Notifications You must be signed in to change notification settings

mrousavy/Colorwaver

Folders and files

NameName
Last commit message
Last commit date

Latest commit

f97faa0 Β· Oct 14, 2021

History

93 Commits
Oct 13, 2021
Sep 21, 2021
Aug 28, 2021
Oct 14, 2021
Sep 21, 2021
Oct 14, 2021
Oct 13, 2021
Oct 14, 2021
Oct 13, 2021
Aug 28, 2021
Aug 28, 2021
Aug 28, 2021
Aug 28, 2021
Aug 28, 2021
Oct 13, 2021
Aug 28, 2021
Aug 28, 2021
Sep 21, 2021
Aug 28, 2021
Sep 27, 2021
Aug 28, 2021
Aug 28, 2021
Aug 28, 2021
Sep 21, 2021
Aug 28, 2021
Oct 14, 2021
Sep 21, 2021
Oct 14, 2021

Repository files navigation

Colorwaver

An app to detect colorwaves (swatches/palettes) in the real world - powered by VisionCamera and Reanimated.

I wrote this app in less than a day, a speed simply not matched by native app development. Because it's written in React Native (TypeScript/JavaScript), it runs both on iOS and Android, but performance critical parts (e.g. the image processing algorithm or the animations) are backed by native Objective-C/Swift/Java code.

See this Tweet for more information.

Buy Me a Coffee at ko-fi.com

Try it!

Download the repository and run the following commands to try Colorwaver for yourself:

iOS

yarn try-ios

Android

yarn try-android

Project structure

This is a bare React Native project, created with create-react-native-app.


  • πŸ“ src: Contains the actual TypeScript + React (-Native) front-end for the Colorwaver App.
    • πŸ“„ src/getColorPalette.ts: Exposes the native iOS/Android frame processor plugin as a JS function with TypeScript types. This function has to be called from a frame processor. ('worklet')

    • πŸ“„ src/Router.tsx: The main component that gets registered by index.js. This acts as a main navigator, navigating either to the Splash Page (Permissions) or the App Page (Main App) depending on whether the user has granted permissions or not.

    • πŸ“„ src/Splash.tsx: The first Splash page to ask the user for Camera permission.

    • πŸ“„ src/App.tsx: Contains the actual source code for the App's front-end.

      • VisionCamera is used to get a Camera device and display a Camera component. Also, a frame processor (a function that gets called for every frame the camera "sees") gets attached, which calls the native iOS/Android frame processor plugin.
      • Reanimated is used to smoothly animate between color changes.

      Because VisionCamera also uses the Worklet API, the entire process between receiving a camera frame and actually displaying the palette's colors does not use the React-JS Thread at all. The frame processing runs on a separate Thread from VisionCamera, which then dispatches the animations on the Reanimated UI Thread. This is why the App runs as smooth as a native iOS or Android app.

    • πŸ“„ src/useAnimatedColor.ts: A helper function to animate color changes with SharedValues.


  • πŸ“ ios: Contains the basic skeleton for a React Native iOS app, plus the native getColorPalette(...) Frame Processor Plugin.
    • πŸ“„ ios/PaletteFrameProcessorPlugin.m: Declares the Swift frame processor plugin "getColorPalette(...)".

    • πŸ“„ ios/PaletteFrameProcessorPlugin.swift: Contains the actual Swift code for the native iOS frame processor plugin "getColorPalette(...)".

      This uses the CoreImage API to convert the CMSampleBuffer to a UIImage, and then uses the UIImageColors library to build the color palette.

      VisionCamera's frame processor API is used to expose this function as a frame processor plugin.

    • πŸ“„ ios/Colorwaver-Bridging-Header.h: A Bridging Header to import Objective-C headers into Swift.

    • πŸ“„ ios/Podfile: Adds the UIImageColors library.

    • πŸ“„ ios/UIColor+hexString.swift: An extension for UIColor to convert UIColor instances to strings. This is required because React Native handles colors as strings.


  • πŸ“ android: Contains the basic skeleton for a React Native Android app, plus the native getColorPalette(...) Frame Processor Plugin.
    • πŸ“„ android/app/build.gradle: The gradle build file for the Android project. The following third-party dependencies are installed:

      • androidx.camera: camera-core
      • androidx.palette: palette
    • πŸ“ android/app/src/main/java/com/colorwaver/utils: Contains two files copied from android/camera-samples to convert ImageProxy instances to Bitmaps. (YuvToRgbConverter.kt)

    • πŸ“ android/app/src/main/java/com/colorwaver: Contains the actual Java source code of the Project.

    • πŸ“„ android/app/src/main/java/com/colorwaver/MainApplication.java: Sets up react-native-reanimated.

    • πŸ“„ android/app/src/main/java/com/colorwaver/MainActivity.java: Installs the PaletteFrameProcessorPlugin frame processor plugin inside of the onCreate method.

    • πŸ“„ android/app/src/main/java/com/colorwaver/PaletteFrameProcessorPlugin.java: Contains the actual Java code for the native Android frame processor plugin "getColorPalette(...)".

      This uses the YuvToRgbConverter to convert the ImageProxy to a Bitmap, and then passes that to the Palette API from AndroidX to build the color palette.

      VisionCamera's frame processor API is used to expose this function as a frame processor plugin.


  • πŸ“„ babel.config.js: Adds the native frame processor plugin getColorPalette (called __getColorPalette) to Reanimated's global list.

Credits