I'm not totally sure why we didn't do this years ago, but this fixes
performances problems in the new React tags box, and should improve
autocomplete performance generally in large libraries.
Since modal windows (e.g., the Create Bib window and the Quick Copy site
editor window) can't use yield, style retrieval
(Zotero.Styles.getVisible()/getAll()) is now synchronous, depending on a
previous async Zotero.Styles.init(). The translator list is generated in
the prefs window and passed into the Quick Copy site editor, but it's
possible the translators API should be changed to make getTranslators()
synchronous with a prior init() as well.
- Simplified schema
- Tags are now added without reloading entire tag selector
- On my system, adding 400 tags to an item (separately, with the tag
selector updating each time) went from 59 seconds to 42. (Given that
it takes only 13 seconds with the tag selector closed, though,
there's clearly more work to be done.)
- Tag selector now uses HTML flexbox (in identical fashion, for now, but
with the possibility of fancier changes later, and with streamlined
logic thanks to the flexbox 'order' property)
- Various async fixes
- Tests
Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
- Fix search for January dates
- Support 'yesterday'/'today'/'tomorrow' and localized equivalents (case-insensitive) in date searches (e.g., [Date Added] [is] ['yesterday'])
Also fixed to not display tags twice if both manual and automatic and to not display automatic tags if manual versions of the same tags are already linked to the current item
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
