Follow-up on Sharing Data with WatchKit Extensions
Recently I posted about a potential problem when sharing a Core Data SQLite-backed persistent store between a host app and one or more iOS extensions. The short version of the problem is that there doesn’t seem to be a way to prevent the creation of duplicate entries when syncing model objects from a remote API resource.
On Stack Overflow and elsewhere I got a lot of helpful replies with potential solutions or workarounds:
Accept Duplicates, Clean Up Later – Tom Harrington suggests not trying to prevent duplicates at all, but instead use the same approach that one applies to syncing Core Data via iCloud. Apple’s engineers explicitly caution you to anticipate merge conflicts, like duplicate entries, and to design cleanup strategies from the beginning. He’s written about this approach at length here.
Redesign Extension to be Read-Only – Dave Delong suggests a simple and obvious solution: don’t allow your extension to have write privileges for your shared Core Data store. If your extension design can handle this, it might be the cleanest and easiest approach.
File Coordinators – Apple’s WatchKit Programming Guide recommends using
NSFileCoordinatorto synchronize read/write access to shared file resources. However, in a bizarre technical note they also claim that usingNSFileCoordinatorin an extension can lead to unpreventable deadlocks and should not be used. If Apple ever resolves this issue, usingNSFileCoordinatormight be easier than cleaning up duplicates. Then again, it might be unwieldy, depending upon your persistence architecture.Build Custom Delta-Driven Syncing Architecture – If your remote API resource is not capable of running its own code (as is the case with iCloud and Core Data, for example), Milen Dzhumerov (an engineer behind Clear.app) describes an alternative in which the shared remote resource is not the whole object graph but instead a history of changes. This is essentially how iCloud+CoreData is supposed to work, with the exception that a domain-specific implementation is better suited to resolve conflicts in nondestructive ways. This is indeed how Clear.app was designed to work:
A peculiar property of the system is that the actual object graph is never stored on iCloud, only instructions on how to build it. When new data arrives from other devices, the whole history needs to be reconstructed to be able to obtain the latest object graph. […] When a device joins iCloud, it pushes its current object graph as a set of changes (so it would push create changes for each entity). When a device replays creation changes, it can detect duplicate requests for the creation of the same object (by looking at the object UID).
Milen’s entire article is fascinating and worth a read.
A painfully obvious solution occurred to me after writing my previous post. As long as the remote API resource is a server with its own conflict resolution management, why not just treat the extension as yet another client app? In other words, let each extension have its own complete stack, not sharing anything (except perhaps Keychain credentials and simple user default settings). The worst case scenario is that the extension and the host app are out of sync until both have had an opportunity to re-sync with the remote API, which may be a short-lived problem depending upon the design of your extension.
WatchKit Extension Problem: Sharing a Core Data Store Can Lead to Duplicate Entries
The WatchKit SDK was released yesterday, emphasizing an interesting problem that affects all iOS 8 extensions that need to share a Core Data store with their host app. Consider this scenario:
General Setup
A host iOS 8 app.
One or more iOS 8 extensions (WatchKit, Share, etc.) bundled with the host app.
The host app and all extensions share the same Core Data SQLite store in the shared app group container.
Each app/extension has its own NSPersistentStoreCoordinator and NSManagedObjectContext.
Each persistent store coordinator uses a persistent store that shares the same SQLite resources in the group container as all the other persistent stores.
The app and all extensions use a common codebase for syncing content from a remote API resource on the Internet.
Sequence of Events Leading to the Problem
The user launches the host app. It begins fetching data from the remote API resource. Core data model objects are created based on the API response and “upserted” into the host app’s managed object context. Each API entity has a uniqueID that identifies it in the remote API backend. By “upsert,” I mean that for each API entity, the host app only creates a new entry in Core Data if an existing entry for a given uniqueID cannot be found.
Meanwhile, the user also launches one of the host app’s extensions. It, too, performs some kind of fetch from the same remote API. It also attempts to perform an “upsert” when parsing the API responses.
The Problem: What happens if both the host app and an extension try to upsert a Core Data entry for the same API entity at the same time? To see how this could come about, let’s look at the sequence of events for an upsert:
Core Data Upsert Sequence
The API parsing code parses the uniqueID for a given API entity.
The parser performs a Core Data fetch for any entry that matches a predicate where
uniqueIDis equal to the parsed uniqueID.If an existing entry is not found, the parser inserts a new Core Data entry for this API entity, set’s its
uniqueIDattribute to the parsed uniqueID.The parser saves the managed object context, which pushes the new entry data down to the SQLite backing store.
Problem in Detail
Let’s assume the host app and the extension are independently parsing an API response for the same API entity at the same time. If both the host app and an extension reach Step 3 before either of them has finished Step 4, then they will both be trying to insert a new Core Data entry for the same uniqueID. When they reach Step 4 and call save: on their respective managed object contexts, Core Data will happily create duplicate entries.
As far as I’m aware, Core Data doesn’t have any way to mark an attribute as unique. I need a Core Data equivalent to a SQLite INSERT OR IGNORE + UPDATE combo.. Or else I need a way to “lock” the persistent store’s SQLite backing store, which sounds like a recipe for trouble.
Is there a known approach to this rather novel problem introduced by iOS 8 extensions? If you have an answer, please consider posting it on Stack Overflow.
How I Learned that Video Games are For Old Ladies, Too
One weekend in the late ‘80s an old friend of my parents came to visit. I’d never heard my mom or dad mention her before, nor have I heard much about her since. She was one of those old friends who tunes a kid into the fact that his parents used to be younger, groovier people.
Our house guest stepped in the door with a suitcase in each hand. She had short silver hair. She looked about two decades older than my parents. Too old to be their friend, I thought. I was in elementary school and had never had a friend more than two years apart from me. She travelled to see us alone. I got the impression she had never married. That was weird, too. Our family was churchy. Every family we knew was nuclear. I don’t think I’d ever met a woman so old who wasn’t married before.
I showed her where she could set down her things. Being the older brother I had the bigger of the two kids bedrooms. Mine was used as a guest room when we needed it. It had an old square dining table in the corner. There was a faux walnut Zenith on it. Next to the TV was an NES, less than a year old.
“You have a Nintendo!” she said. “I love these.”
“Pshyeahright,” I thought to myself. Some old lady with silver hair? She’s just trying to smooth things over with the kid who’s losing his bedroom for the weekend.
After dinner she sat down in front of the old Zenith and started playing Super Mario Bros.
“Do you know the infinite lives trick?” she asked me.
“The what?” I asked.
“Watch this,” she replied.
Then I marveled as she played a perfect game, straight through to World 3, Level 1, one of the night levels with a jet black sky. Near the end of the level there’s a half-pyramid of gold bricks. Two Koopa turtles come marching down it as soon as the pyramid comes into view.
“Here’s the trick,” she said.
What she did next would take me weeks to master. I doubt I could do it today, if I still had an NES and an old Zenith. She did it on the first try, flawlessly. She popped the second Koopa, bouncing on its shell with quick short hops. With each hop, the shell ricocheted back and forth one tiny bricks’ worth of distance on the edge of the pyramid, perpetuated by this improbable old woman’s mastery of the controls. With each pop, more and greater points bubbled up from the collisions. The points turned into one-ups. Boop-beep-boop. Boop-beep-boop. Mario’s life count climbed up higher and higher. 98, 99, 100. It got so high that the lives counter used strange symbols instead of numbers. I have no idea how many lives Mario had by the time she got bored and let the second shell go.
Thank you, silver-haired lady, whoever you are, for teaching me that software can be hacked and integers can overflow. Thanks most of all for teaching a rude little boy that video games, like all passions, are meant for anyone who loves them.
Unread by the Numbers
We’re in the midst of an enormous refactor at Bloglovin. We’ve left thousands of lines of code in our wake over the last three weeks, and it got me thinking about the amount of code that went into Unread over the ~12 months of its development.
Here’s Unread by the numbers, counting lines of code just in .m files. These line counts do not include header files, documentation, or third party libraries except for libraries like JTSImageViewController or OvershareKit which were written by me1 in the course of developing Unread:
26,993 UNRKit (API & persistence code) 20,138 Unread iPhone Project (misc. & not-yet-reusable UI) 17,514 Overshare Kit (sharing library) 8,637 UNRUIKit (reusable UI code) 2,430 JTSImageviewController 381 JTSReachability 239 JTSScrollIndicator 139 JTSSemanticReload ------ ------------------ 76,471 TOTAL
I worked on Unread for about a year before handing over the reigns to Supertop. Most of this code was written in the first-six months of development, though.
People might be most surprised by that Overshare Kit count. It represents 23 percent of the overall amount of code. I wanted Unread to have lots of great sharing options from 1.0, and with a library that could be reused in any future app. Overshare Kit is now mostly obviated by iOS 8 Extensions, but at the time it was unclear if Apple would ever bother trying to make sharing easier on iOS.
These numbers also don’t include numerous lines of code from open source libraries without which Unread would have been impossible: FMDB, FastImageCache, jQuery, Bigfoot.js, and others.
-
In the case of OvershareKit I am not the only author, but I contributed the bulk of the initial library and API design. ↩
Bloglovin 3.0.3 Available Now
A new version of Bloglovin is available on the App Store now. This is the first new version since I joined the iOS team there this summer. Version 3.0.3 is a minor update, mostly squashing a bunch of bugs. It also adds provisional support for iPhone 6 and iPhone 6 Plus. Efficiently delivering appropriate image thumbnails for iPhone 6 Plus is a huge challenge, but a fun one. I’m looking forward to many more releases.