How to Use OpenCV in React Native for Image Processing - Blog Brainhub.eu

April 05, 2018 0 Comments

How to Use OpenCV in React Native for Image Processing - Blog Brainhub.eu

 

 

If you have ever wondered how to process your images using OpenCV with React Native, then you’re in the right place.

OpenCV together with React Native enables you to process images on mobile devices (most likely you’d like to process images taken by your device’s camera).

The top advantages of those are:

  • Easy to implement.
  • Easy to use.
  • Lots of tutorials all over the Internet and solid official documentation of OpenCV.
  • The size of your mobile app will be only a dozen or so megabytes bigger.

Let me show you how I used the OpenCV and React Native step by step to process my images, but first a few words of introduction.

What is OpenCV?

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

The library has more than 2500 optimized algorithms, including a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms.

These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc.

OpenCV has more than 47k people community and an estimated number of downloads exceeding 14 million.

The library is used extensively in companies, research groups and by governmental bodies.

OpenCV is written natively in C++.

Since 2010 OpenCV was ported to the Android environment, it allows using the full power of the library in mobile applications development.

In 2012 OpenCV development team actively worked on adding extended support for iOS. Full integration is available since version 2.4.2 (2012).

React Native was first released in 2015 by Facebook. React Native lets you build mobile apps using only JavaScript. It uses the same design as React, letting you compose a rich mobile UI from declarative components.

React Native and OpenCV are good friends!

If we google for “react native opencv” we stumble upon the following links:

What is this tutorial about?

In this tutorial, we will be building an example project which uses the device’s camera to take a photo, processes it with native code and returns the information if the taken image is blurred or clear.

Doing it in plain JavaScript would be highly ineffective. JavaScript is not sufficient for very heavy calculations.

Take a note that we are by no means Java or Objective-C developers hence our Java/Objective-C code may be far from good. If you want to contribute to this subject, please contact us.

OpenCV basic preparation

1. react-native init reactNativeOpenCvTutorial

2. Run “downloadAndInsertOpenCV.sh” script in your project’s catalog (can be found here) which downloads and inserts openCV files for both Android and iOS. The paths in the file might not match your preferences so you may need to change them.

Tutorial for Android

1. Open your project in Android Studio.

2. Follow the tips in Android Studio for synchronizing your project.

3. Download the newest version of OpenCV for Android. In my case, it’s 3.4.1.

4. Import OpenCV to Android Studio:

From File -> New -> Import Module, choose sdk/java folder in the unzipped OpenCV archive.

5. Update build.gradle under imported OpenCV module to update 4 fields to match your project’s build.gradle:

a) compileSdkVersion

b) buildToolsVersion

c) minSdkVersion

d) targetSdkVersion.

6. Add module dependency:

Application -> Module Settings, and select the Dependencies tab. Click “+” icon at the bottom, choose Module Dependency and select the imported OpenCV module.

For Android Studio v1.2.2, to access Module Settings in the project view, right-click the dependent module -> Open Module Settings.

Open up Module Settings.

Adding Module Dependency through Module Settings - Android.

Click on the “+”, select “Module dependency” and select the OpenCV library from the list.

Importing OpenCV Module - Android.

7. Inside android/app/src/java create a package called i.e. “com.reactlibrary”.

8. Update your Manifest with proper permissions:

<uses-permission android:name="android.permission.CAMERA" /> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

See the whole file.

9. Create a file RNOpenCvLibraryModule.java inside your newly created package and fill it as shown here.

10. Create a file called RNOpenCvLibraryPackage.java inside your newly created package and fill it as shown here.

11. Add proper imports in your MainApplication.java file, add your OpenCV package to the list and proper code to the MainApplication class like this:

Imports: import com.reactlibrary.RNOpenCvLibraryPackage; import org.opencv.android.BaseLoaderCallback; import org.opencv.android.LoaderCallbackInterface; import org.opencv.android.OpenCVLoader; import android.util.Log;

Add “new RNOpenCvLibraryPackage()” to your list of packages.

Adding OpenCV library to the list of packages - Android.

Add BaseLoaderCallback in your MainApplication class:

Adding BaseLoaderCallback - Android.

Also, add following callbacks to your MainApplication class:

Adding callbacks to the MainApplication - Android.

See the whole file here.

Tutorial for iOS

1. Open the iOS project in XCode.

2. Add opencv2.framework to your Linked Frameworks and Libraries.

Adding opencv2.framework to Linked Frameworks and Libraries - iOS.

3. Create a new group in the iOS catalog. I named it “OpenCV”.

4. Add a .pch file and insert it into the OpenCV catalog.

5. Add proper content to your .pch file – shown here.

6. Create a file named RNOpenCvLibrary.h and fill it as shown here.

7. Create a file named RNOpenCvLibrary.mm and fill it as shown here.

8. Set Precompile Prefix Header to Yes and set Prefix Header path like this:

Setting Precompile Prefix Header path - iOS.

9. Add the following to your Info.plist file:

<key>NSCameraUsageDescription</key> <string>Your message to user when the camera is accessed for the first time</string> <key>NSPhotoLibraryAddUsageDescription</key> <string>Your message to user when the photo library is accessed for the first time</string>

See the whole file.

Final part – JavaScript

1. Inside your src folder create a folder named e.g. “NativeModules” and a file named OpenCV.js and fill it with:

import { NativeModules } from 'react-native'; export default NativeModules.RNOpenCvLibrary;

See the whole file.

2. We will be using some third-party libraries for quick setup. Open up the terminal and type:

$ npm i –save react-native-svg
$ npm i –save react-native-camera

$ npm i –save react-native-easy-toast


Tag cloud