One Undeployed Record Kept an iPhone Silent for 18 Days

Hi, I’m WindMori.

While testing MononoMori 4.0 before release, I found that one of my test iPhones hadn’t uploaded a single change to iCloud in 18 days. And the in-app sync log didn’t show a single error.

The cause turned out to be a data model added in 4.0 that had never made it into the schema of CloudKit’s production environment. When the server rejected that one record, Core Data stopped the whole upload batch, and with it all syncing for the rest of that session.

The fix was a single schema deploy. What actually took time was finding it. This post walks through how I tracked it down, and the safeguards I put in place afterward.

First, how MononoMori’s sync is set up

MononoMori stores its data on-device with SwiftData. When iCloud sync is on, NSPersistentCloudKitContainer, which sits behind SwiftData, mirrors it to the user’s own CloudKit private database.

CloudKit has two environments, and this whole problem lived in the gap between them:

• Development: Debug builds run straight from Xcode connect here. The schema here is created on demand: the first time a record type is uploaded, CloudKit builds the record type and its fields from whatever that record carries. Fields without a value don’t get created.
• Production: TestFlight and App Store builds connect here. Nothing is ever created automatically. It only accepts structure deployed from Development with “Deploy Schema Changes” in the console.

Each SwiftData model maps to a CloudKit record type prefixed with CD_. SDItem becomes CD_SDItem, for example. And once a field is in Production, it can’t be changed or deleted.

4.0 adds quite a few new models. One of them is SDBasketSharedItem, created whenever you put an item from a Shared Grove into your basket. It’s rarely used, and that turns out to matter.

Two devices out of sync, and a perfectly clean log

I noticed the problem while testing 4.0’s new original-file sync for the Treasure Box, adding things back and forth between an iPhone and an iPad. Nothing I added on the iPhone ever showed up on the iPad, and nothing changed in the CloudKit console either. My first thought: the new feature is broken.

The strange part was that the sync log in the app’s hidden diagnostics menu showed no errors at all. Both devices were on the same build, the same Apple ID, the same subscription, and both were on Production. Every usual suspect checked out.

Tracking it down: five steps, two wrong turns

Looking back, the real clue showed up very early. I just didn’t take it seriously at first.

1. Compare the diagnostics reports from both devices. The difference was in an easy-to-miss field. The iPad’s “last upload” was a few minutes ago; the iPhone’s was October 13. So the iPhone hadn’t uploaded successfully since that day. It downloaded once at launch, and after that it never received any of the iPad’s later uploads.
2. Fully quit and reopen the app. An upload failure finally showed up in the log: CKErrorDomain 2 (partialFailure, meaning some records were rejected). The log had been clean because the failure only happens at the moment the app relaunches and retries the upload. But the error had only a domain and a code. It didn’t say which record was rejected or why. By the time Core Data hands the event to the app, the per-record details are gone.
3. Check Logs in the CloudKit console. Looking up the request by UTC time, it showed as RecordDelete, BAD_REQUEST. I assumed some delete had been rejected, and even talked myself out of my first hunch that a schema was missing. Wrong turn number one: that batch actually held 25 records, and the console only gave one broad category for the whole thing.
4. Connect to a Mac and check device logs in the Console app. Core Data’s errors were all there, but the important parts all read <private>. iOS redacts private arguments from non-debugged processes when it writes the log, so having the app read its own logs wouldn’t work either. I almost shipped an extra build to try it. Wrong turn number two.
5. Run from Xcode, but against Production. Xcode’s console shows the full error, but a normal Debug build connects to Development, which would switch this device’s data to a different environment. The trick is to temporarily add one line to the entitlements so the Debug build connects to Production, then delete it as soon as you’re done:

<key>com.apple.developer.icloud-container-environment</key>
<string>Production</string>

This time, I finally saw the server’s own words:

"Invalid Arguments" (12/2006); server message = "Cannot create new type CD_SDBasketSharedItem in production schema"
... 24 "Batch Request Failed" CKErrors omitted ...

Root cause: Production only knows what’s been deployed

Around October 13, I’d tested putting a Shared Grove item into the basket on that iPhone, which created the first-ever SDBasketSharedItem. But no Debug build had ever uploaded that model, so Development had no CD_SDBasketSharedItem, and none of my earlier deploys included it. So Production rejected it.

So why can one record freeze an entire device?

• Uploads are atomic per batch. When one record is rejected, the other 24 in the same batch fail along with it (Batch Request Failed).
• Core Data treats this error as unrecoverable. It resets its internal state, then refuses every request for the rest of that session. Uploads and downloads both stop.
• On the next launch, the same batch is rejected again. And so on, with the app only ever getting a bare CKErrorDomain 2.

That’s why the iPad was fine: it had never created one of these records. Meanwhile, every change on the iPhone since October 13 had stayed on the device.

My old habit was “after adding a model, save one record in the development environment, then deploy.” That’s exactly what misses a rarely used model like this one, or a field that’s usually empty.

The fix: build out Development in one go, then deploy

The fastest-looking option was to create the type by hand in the console. I didn’t. Production fields can’t be changed or deleted once deployed, so one mistake would be permanent. When I looked at CD_SDBasketSharedItem in Production afterward, Core Data had also added fields of its own like CD_moveReceipt (BYTES), CD_moveReceipt_ckAsset, and CD_spaceKey_ckAsset. Getting those right by hand would have been close to impossible.

Apple’s answer for SwiftData is initializeCloudKitSchema(). It generates a Core Data model from your SwiftData models, then uploads one sample record per model with every field filled in, so Development gets built out in one go. I turned it into a small Debug-only tool that runs against a throwaway empty store and never touches real data:

let model = NSManagedObjectModel.makeManagedObjectModel(for: AppSchema.models)!
let description = NSPersistentStoreDescription(url: throwawayStoreURL)
description.cloudKitContainerOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: containerID)
description.shouldAddStoreAsynchronously = false
let container = NSPersistentCloudKitContainer(name: "Store", managedObjectModel: model)
container.persistentStoreDescriptions = [description]
container.loadPersistentStores { _, error in /* handle error */ }
try container.initializeCloudKitSchema()

I’d planned to run it in the Simulator, but on my macOS beta, Xcode 27’s Simulator window (DeviceHub) crashed the moment it opened, so I couldn’t sign in to iCloud there. So I took a different route: run the app on the Mac as an iPad app. The Mac is already signed in to iCloud. One run with a launch argument built everything in about 20 seconds, then quit right away without ever opening the forest database.

Next I deployed to Production from the CloudKit console, then used Apple’s cktool to export both schemas and compare them: 39 record types, 466 fields, a perfect match.

Once I reinstalled the TestFlight build on that iPhone, 18 days of backed-up changes reached the iPad all at once. That was a real weight off my shoulders.

Making sure it doesn’t happen again

You only find out what Production is missing when some user actually uses that field, and the way you find out is that uploads quietly stop. So the safeguards have to sit before release, and they can’t depend on me remembering.

1. Build out Development with one command before release. ./scripts/cloudkit_schema.sh create builds a Mac Debug version of the app and runs initializeCloudKitSchema() once with a launch argument, using the Mac’s iCloud account. No Simulator or device needed, and it takes about a minute with a warm build cache.
2. Deploy in the console. Apple doesn’t offer a command line for deploying to Production, so this step stays manual.
3. Check automatically before uploading. check uses cktool to export the Development and Production schemas, compares them record type by record type and field by field, and lists anything Production is missing. Before archiving, the upload script runs gate automatically: if model files changed, it runs create first, then check, and stops the upload if anything is missing. (cktool needs a CloudKit management token, set up once.)
4. See the problem right in the app’s diagnostics. The hidden diagnostics menu now has Uploads, Downloads, and Setup rows that persistently record the last success and when failures started, like Failing since 10月13日 · CKErrorDomain 2. If uploads keep failing, a warning appears below. There’s also a new “Check the CloudKit schema” action that asks Production directly from a TestFlight build whether every record type the app uses is there.
5. Write the rule into the dev docs. The old rule was “after adding a model, save one record in the development environment, then deploy,” which is exactly how this slipped through. Now it’s: run the script before every release → deploy → verify.

On top of all that, I still test iCloud sync between two devices by hand before every release. The automated checks cover “is the structure complete?” The manual test covers “does it actually work in use?” I need both.

Things that’ll make next time much faster

Looking back, most of the time went into looking in the wrong places, not into the fix. A few notes for my future self:

• Check “last successful upload” first, not whether the log has errors. Logs get cleared and have a size cap, and a stuck device doesn’t log anything at all between launches.
• Fix the order of investigation. Uploads in diagnostics → Check the CloudKit schema → if needed, “temporary Production entitlement + run from Xcode” to see the server’s actual message. The first two take a few minutes on the phone. Only the third needs a computer.
• Know the dead ends up front. Errors in Core Data’s sync events only have a domain and code. Details in device logs are <private>, and the same goes when the app reads them itself. Logs in the CloudKit console only give a broad, per-request category. Knowing this saves round after round of “let’s ship one more build and see.”
• Schema work doesn’t need a device. An iPad app runs directly on an Apple silicon Mac, with the Mac’s iCloud account built in. For one-off dev tools, that’s far easier than the Simulator (sign-in, a window) or a real device (Developer Mode, unlocking).
• Automate what you can in scripts, and have scripts stop and tell you about what you can’t. Everything checkable in the release flow goes into the upload script. Deploying in the console can’t be automated, so the script stops there and spells out what’s missing.

Final thoughts

Luckily, no users were affected this time. The problem was on my test device, and 4.0 is still in TestFlight. But if it had shipped with the public release, all users would have seen was “sync seems a bit off.” No message in the app, and no feedback reaching me.

CloudKit and Core Data hide most of the complexity, which is usually a relief. The trade-off is that when something goes wrong, they’re just as quiet.

For an app made by one person, what users trust MononoMori with is the records and memories behind each of their things. What I can do is automate the pre-release checks as much as possible, and leave a window in the app that shows what’s really going on. That way, even if fog rolls into the forest, I’ll be the first to see it.