Offline Storage for Progressive Web Apps
2016 will hopefully be the year we build for network resilience.
Internet connections can be flakey or non-existent on the go, which is why offline support and reliable performance are common features in Progressive Web Apps. In this post, I'll summarize some ideas around offline data storagefor PWAs - think the JSON payloads, images and general static data required to provide a meaningful experience offline.
A recommendation for storing data offline:
Some quick answers to common questions on why:
- Both APIs are asynchronous (IDB is event based and the Cache API is Promise based). They also work in Web Workers, Window and Service Workers.
- IDB is available everywhere. Service Workers (and the Cache API) are available in Chrome, Firefox, Opera and are in development for Edge.
- While IDB doesn't support Promises, several strong libraries giving us Promise wrappers exist. See below for recommendations. The API has mandatory complexity (transactions, schema versioning) that these libraries also try to smooth over where possible.
- Native support for IDB Promises has been proposed as have observers.
- How much can you store? In Chrome and Opera: Your storage is per origin (rather than per API). Both storage mechanisms will store data until the browser quota is reached. Apps can check how much quota they're using with the Quota Management API. Firefox: no limits, but will prompt after 50MB data stored. Mobile Safari: 50MB max, Desktop Safari: unlimited (prompts after 5MB), IE10+ maxes at 250MB and prompts at 10MB. PouchDB track IDB storage behavior. Future facing: For apps requiring more persistent storage, see the on-going work on Durable Storage.
- Safari has fixed many long-standing IDB bugs in their latest Tech Previews. That said, some folks have run into stability issues with Safari 10's IDB. Until more research has been done here, YMMV. Please do test and file browser bugs so the folks @webkit and related OSS library authors can take a looksie. LocalForage, PouchDB, YDN and Lovefield use WebSQL in Safari by default due to UA sniffing (there wasn't an efficient way to feature-test for broken IDB at the time). This means these libraries will work in Safari 10 without extra effort (just not using IDB directly).
- URL addressable resources are typically static resources that surprisingly..live at a URL. For PWAs, you could cache the static files composing your application shell (JS/CSS/HTML files) in the Cache API and fill in the offline page data from IndexedDB. There are no hard and fast rules around this however and a PWA could get by just using Cache API.
- Debugging support for IndexedDB is available in Chrome (Application tab), Opera, Firefox (Storage Inspector) and Safari (see the Storage tab).
IndexedDB Libraries worth checking out
- localForage(~8KB, Promises, good legacy browser support)
- idb-keyval (<500 bytes, Promises, use if only need key-value support)
- idb (~1.7KB, Promises, also does iteration, indexing)
- Dexie (~16KB, Promises, complex queries, secondary indices)
- PouchDB (~45KB (supports custom builds), synchronization)
- LokiJS (in-memory)
- ydn-db(dexie-like, works with WebSQL)
Service Worker Libraries worth checking out
- sw-toolbox (offline caching for dynamic/runtime requests)
- sw-precache (offline precaching for static assets/application shells)
- Webpack users can directly use the above or offline-plugin
What about other storage mechanisms?
- Web Storage (e.g LocalStorage) is synchronous, has no Web Worker support and is size-limited (only strings). Although ideas for async LS have been kicked around in the past, current focus is on getting IndexedDB 2.0 in a good state. I would personally use IDB + a library.
- Cookies have their uses but are synchronous, lack Web Worker support and are size-limited. In previous projects I've used js-cookie for handling cookies via JS (~800bytes gzipped). Support for an Async Cookies API is being sketched out right now with a polyfill in the works.
- WebSQL is asynchronous (callback-based), however it also has no Web Worker support and was rejected by Firefox and Edge but is in Chrome and Safari.
- File System API is asynchronous too (callback-based) and does work in Web Workers and Windows (albeit with a synchronous API). Unfortunately it doesn't have much interest outside of Chrome and is sandboxed (meaning you don't get native file access).
- File API is being improved over in the File and Directory Entries API and File API specs. A File API library exists and for file-saving, I've been using FileSaver.js as a stop-gap. The writable-files proposal may eventually give us a better standards-track solution for seamless local file interaction.
Current and future offline storage work
If offline storage interests you, the below efforts are worth keeping an eye on. I'm particularly excited about Promises support in IndexedDB being possible without the need for a library.
- Durable Storage: protect storage from the user agent's clearing policies
- Indexed Database API 2.0: advanced key-value data management
- Promisified IndexedDB: native support for a Promise-friendly version of IDB
- IndexedDB Observers: native IDB observation without needing wrapper around the database
- Quota Management API: check how much quota an app/origin is using
- writable-files: allow sites to interact with local files more seamlessly
- Directory downloads: allow sites to download directories without .zip files
- File and Directory Entries API: support for file and directory upload by drag-and-drop
Offline storage isn't quite magical and an understanding of the underlying APIs will go far in helping you make the most out of what we now have available. Whether you prefer to directly use these APIs or work with an abstraction library, take some time to get familiar with your options.
With thanks to Nolan Lawson, Joshua Bell, Jake Archibald, Dru Knox and others for their previous work in the web storage space.
Hopefully this will help craft an offline experience that makes your PWA ✨