Commit graph

2 commits

Author SHA1 Message Date
Dan Stillman
a1ce85decb Overhaul object downloading/processing during data syncs
Previously, objects were first downloaded and saved to the sync cache,
which was then processed separately to create/update local objects. This
meant that a server bug could result in invalid data in the sync cache
that would never be processed. Now, objects are saved as they're
downloaded and only added to the sync cache after being successfully
saved. The keys of objects that fail are added to a queue, and those
objects are refetched and retried on a backoff schedule or when a new
client version is installed (in case of a client bug or a client with
outdated data model support).

An alternative would be to save to the sync cache first and evict
objects that fail and add them to the queue, but that requires more
complicated logic, and it probably makes more sense just to buffer a few
downloads ahead so that processing is never waiting for downloads to
finish.
2016-03-23 04:29:04 -04:00
Dan Stillman
eb89b4ed12 Test for DB integrity check 2016-01-08 03:31:43 -05:00