Previously, objects were first downloaded and saved to the sync cache,
which was then processed separately to create/update local objects. This
meant that a server bug could result in invalid data in the sync cache
that would never be processed. Now, objects are saved as they're
downloaded and only added to the sync cache after being successfully
saved. The keys of objects that fail are added to a queue, and those
objects are refetched and retried on a backoff schedule or when a new
client version is installed (in case of a client bug or a client with
outdated data model support).
An alternative would be to save to the sync cache first and evict
objects that fail and add them to the queue, but that requires more
complicated logic, and it probably makes more sense just to buffer a few
downloads ahead so that processing is never waiting for downloads to
finish.
- Hide notes, tags and related for feed items in itembox
- Add feed support for <enclosure> elements
- Add feed syncing methods for synced settings (additional work is
needed on the sync architecture to download synced settings from the
server)
- Change feed item clear policy to be less aggressive
- Adjust for deasyncification
- Disable translate-on-select
- Closeadomasven/zotero#7, Remove context menu items from feeds
Also:
- Remove last-sync-time mechanism for both WebDAV and ZFS, since it can
be determined by storage properties (mtime/md5) in data sync
- Add option to include synced storage properties in item toJSON()
instead of local file properties
- Set "Fake-Server-Match" header in setHTTPResponse() test support
function, which can be used for request count assertions -- see
resetRequestCount() and assertRequestCount() in webdavTest.js
- Allow string (e.g., 'to_download') instead of constant in
Zotero.Sync.Data.Local.setSyncState()
- Misc storage tweaks
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
This will appear much less frequently, since non-conflicting field changes on
both sides can be resolved automatically, but genuine field conflicts still
require manual conflict resolution.
The merge pane is no longer editable, since the itembox code to do that is
async and can't run in a modal window, but it's not really necessary,
particularly with conflicts happening less frequently.
TODO:
- Remote item deletions
- File conflicts
- Maybe handle some edge cases where the conflicted items fail to save
Since modal windows (e.g., the Create Bib window and the Quick Copy site
editor window) can't use yield, style retrieval
(Zotero.Styles.getVisible()/getAll()) is now synchronous, depending on a
previous async Zotero.Styles.init(). The translator list is generated in
the prefs window and passed into the Quick Copy site editor, but it's
possible the translators API should be changed to make getTranslators()
synchronous with a prior init() as well.
- Simplified schema
- Tags are now added without reloading entire tag selector
- On my system, adding 400 tags to an item (separately, with the tag
selector updating each time) went from 59 seconds to 42. (Given that
it takes only 13 seconds with the tag selector closed, though,
there's clearly more work to be done.)
- Tag selector now uses HTML flexbox (in identical fashion, for now, but
with the possibility of fancier changes later, and with streamlined
logic thanks to the flexbox 'order' property)
- Various async fixes
- Tests
Restore prepopulated charset table, but this time with just the
encodings from the WHATWG Encoding Standard. Assigning a charset to
Zotero.Item::attachmentCharset runs the value through
Zotero.CharacterSets.toCanonical() automatically.
This migrates attachment charsets to the new canonical values, clearing any
that are unsupported.
Other legacy mappings could still be added back, as disussed in #760.
And add group.fromJSON(json, userID), which sets editable and
filesEditable properties based on the group JSON (libraryReading, role
lists, etc.) and the given user
Relations are now properties of collections and items rather than
first-class objects, stored in separate collectionRelations and
itemRelations tables with ids for subjects, with foreign keys to the
associated data objects.
Related items now use dc:relation relations rather than a separate table
(among other reasons, because API syncing won't necessarily sync both
items at the same time, so they can't be stored by id).
The UI assigns related-item relations bidirectionally, and checks for
related-item and linked-object relations are done unidirectionally by
default.
dc:isReplacedBy is now dc:replaces, so that the subject is an existing
object, and the predicate is now named
Zotero.Attachments.replacedItemPredicate.
Some additional work is still needed, notably around following
replaced-item relations, and migration needs to be tested more fully,
but this seems to mostly work.
The sync cache will have pristine copies of the existing versions of
local objects for better conflict resolution, but downloads will get
saved to the sync cache first before processing, so the cache needs to
be able to hold more than one version.
And use 'version' instead of 'itemVersion' for object version for items
Also add deferred foreign key checking to system.sql so that DROP TABLE
commands don't fail mid-transaction
- Use async DB and OS.File for bundled file updates
- Remove support for translator/style ZIP files -- the two options are
now non-unpacked XPIs with subfolders or unpacked source installations
- Now that we have async file access, don't store translator code in
database cache -- just store metadata so that it's available without
reading each translator file
- Change the (previously partially asyncified) Zotero.Styles/Translators
APIs a bit -- while the getAll/getVisible methods are asynchronous and
will wait for loading, the get() methods are synchronous and require
styles/translators to be initialized before they're called. Most
places that end up calling get() probably call getAll/getVisible first
and should therefore be async, but if there's any way to trigger a
get() first, that will need to be adjusted.
- Asyncify various other style/translator-related code
XPI support is untested, as is style/translator usage, so there are
almost certainly bugs. The latter depends on updated export format
support (#659), since toArray() no longer exists on this branch.
Addresses #529 and #520
Note that this loses conflicting changes to translate_item.js from 849803473a,
so those will need to be reapplied if applicable. /cc @aurimasv, @mtd91429
Instead of limiting charsets to a fixed list, dynamically populate it
with any charset name of less than 50 ASCII characters. Previously,
unknown charsets were discarded.
Zotero.Item.prototype.attachmentCharset now always returns a charset
name. It can be set with either a name or a charsetID.
Also:
- Remove the unused 'originalPath' column in itemAttachments
