At Pillow, we release an update to our Pillow Pro app (iOS & Android) every 2 weeks or so. Most releases are over-the-air (OTA) updates that are done using CodePush, and are fully automated with Bitrise. If you would like to know more about our CodePush + Bitrise CI setup, read my previous blog post here.
In this post, I’ll be talking briefly about why we need to test our app and the kinds of tests we can use. Then, I’ll jump straight into how we use Jest (unit tests) and Detox (end-to-end tests) for our Pillow Pro app. And finally, I’ll talk about running these tests on a CI such as Bitrise.
The most important reason is to ensure high product quality. Testing can help identify bugs introduced during the development phase of the app and addressing these bugs before release means that your users will always get to use a reliable product.
Unit tests are used to test small modular pieces of code (aka units) independently.
Integration tests combine related “units” together and test them as a group.
End-to-end tests are comprehensive, user-level tests that ensures that the flow of the app works as intended from start to finish.
Regression tests help to verify that the software that was previously developed and tested still performs the same way after a few changes.
One of my favorite things about Jest is a feature called Snapshot testing.
Snapshot tests are a very useful tool whenever you want to make sure the UI for a component or the object from an redux action or reducer does not change unexpectedly.
Here’s a good explanation of what a snapshot test is. A snapshot test generates a snapshot of a component and compares it against itself every time the tests are run. This helps us easily identify unintended side-effect changes and update the tests as needed.
Check out the official Jest docs if you are curious to learn more about Snapshot testing: https://facebook.github.io/jest/docs/snapshot-testing.html
The official docs for Jest are great. Just follow this guide and you should be up and running in no time.
We have tests for redux actions, redux reducers and all of our UI components.
In the same directory that a new component, action or reducer is added, we create a
tests folder and create a file within it named
newcomponent.spec.js. Then refer to the below templates for an action, reducer or component respectively and modify the test to fit the specific use case.
1. Redux Actions
2. Redux Reducers
npm run test. This will create a new
snapshots folder in the same directory and add the generated snapshots in there.
Additionally, you can also add code coverage to figure out how well your codebase is tested as a whole. Here’s a simple example on how to set this up.
An end-to-end test emulates a user by finding and interacting with pieces of UI in your app in a production/release environment.
Detox is a gray box E2E Tests and Automation Library for Mobile Apps built by Wix
We initially wrote our end-to-end tests using Appium which is an open source test automation framework for mobile apps. But Appium did not provide a pleasant developer experience at all for two main reasons:
- The setup was pretty complicated & had too many moving parts.
- The tests were really flaky, meaning a lot of tests would fail for no apparent reason and had to be re-run to pass.
We then looked into Detox as an alternative and were pretty impressed by how simple the setup was. So we gave Detox a shot and have not looked back since. The only caveat is that Detox currently works only on iOS and Android support is still WIP.
Recently Detox has been getting a lot of attention from the React Native community as Microsoft, CallStack.io and Wix are all dedicating resources to help improve the framework. Hopefully, it will soon mature into the go-to E2E testing framework for React Native apps.
Setting up Detox is surprisingly simple. Just follow the getting started docs and you should have your first test passing within minutes.
A really useful feature in Detox is the ability to set iOS permissions as part of the tests. In our case, we request Push Notification permissions as well as Location permission when the app is in use. Here’s how we handle them in Detox:
Here’s a simple example of what a detox test for logging into the app might look like:
And finally, here’s a part of our E2E test suite in action. Couple things to note:
- We use
await device.reloadReactNative();to make sure the app is reloading after every test. Hence you’ll notice a screen flash after the user logs in for instance.
- To give you some context on what you are seeing below: The Pillow Pro app is used by Pillow Pros to complete cleaning activities for Pillow Residential Units.
Once you have all the tests working locally, you can configure them to run on a CI service such as Bitrise. We have a
tests workflow that runs the unit tests and end-to-end tests on every commit and posts the build status back to our Slack channel.
You can view the entire Bitrise .yml file for the
tests workflow here.
Overall, adding unit & end-to-end tests has given us the freedom to make substantial changes to our app without having to worry about breaking existing functionality and ship new features with confidence. Also, automating the tests on Bitrise CI has helped us streamline our release pipeline by preceding every release with the
Hopefully, this post details how easy it is to test a React Native app and automate the testing process as well. Feel free to ask us any questions you might have regarding our setup and we would be more than happy to answer them.
Lastly, we’re hiring! Check us out: pillow.com/careers 🙌🏽