Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
This mostly works, but there is still much to be done:
- getTranslators() calls in preferences_export.js and quickCopy.js need
to be made asynchronous
- getTranslators() calls in mimeTypeHandler.js need to be made
asynchronous. This looks hard, since there is a synchronous Mozilla
API involved
- I need to test lookup, the connector server, and export more to make
sure that they actually work
- This API needs to be implemented for the connector and bookmarklet
- Translators should be read off the disk using async IO
This change should improve Firefox startup time. If the Zotero pane is
opened before Zotero has initialized, the pane will be blocked with a
progress bar until it's done. Standalone currently does the same, but it
should be changed to just delay opening the window if possible.
The upgrade wizard has been disabled for schema upgrades, in the hope
that upgrades can be done in a way that won't break compatibility with
earlier versions (or that can at least be done in a way such that we can
put out point releases of the last major version that provide
compatibility during beta/post-upgrade periods).
This patch likely breaks many things, and definitely breaks connector
mode.
This change also removes upgrade steps for databases from Zotero 2.1b2
and earlier. Users with such databases will need to upgrade via Zotero
4.0.x first or delete their data directories and start anew. This should
only affect users who haven't opened Zotero since Nov. 2010.
Currently generates multi-source citations for multiple items for note
styles, but a list of note citations might be better.
Also, context menu option is unchanged, since it becomes very long if
you add "Citation/Bibliography".
Adds three new preferences controlling whether a new collection is created upon import:
- extensions.zotero.import.createNewCollection.fromFile
Controls whether a new collection is created by import from cog menu
- extensions.zotero.import.createNewCollection.fromClipboard
Controls whether a new collection is created by import from clipboard
- extensions.zotero.import.createNewCollection.fromFileOpenHandler
Controls whether a new collection is created when import is initiated by
double-clicking a file or by dragging it over Zotero. This preference applies only to
Zotero Standalone, and is configurable in the dialog that appears asking the user
to confirm the import.
Closes#19, [papercuts] (26) Clipboard import into an open collection
- fix attachment export/import
- make Translator return a copy of displayOptions or configOptions objects
- adjust file export functions for above change
this currently leaks a hidden browser object, since deleting it seems to cause the print command to fail. this may not be a big issue, since this feature is used so rarely anyways...
closes#1650: suppress author does not work for multiple sources
closes#1505: Edit Biblography Button Strips Year Disambiguation
closes#1503: Editing a bibliography resets all reference numbers to 1 (new)
closes#1262: Broken pluralization with et al. + other issues
closes#1238: Localize quotation marks
closes#1191: Harmonize 'plural/pluralize' label attribute with CSL schema
closes#1154: Only one works page numbers are added to the citation are when citing multiple works by the same author
closes#1097: Disambiguation issues
closes#1083: Defect in IEEE CSL with Multiple Citations
closes#993: more sophisticated subsequent-author-substitute
closes#833: text-transform doesn't work with name
- File import now uses a translucent overlay over Zotero pane with a progress meter that doesn't hang
- New method Zotero.wait(timeout) to allow synchronous code to wait for events on main thread to be processed until timeout is reached
- Wait status can be tested with Zotero.waiting property
- Zotero.showZoteroPaneProgressBar(msg, determinate) locks Zotero and creates overlay with progress meter
- Code that might trigger via timers or external UI should check Zotero.lock
Lock checks so far:
- Translator save icon and RIS/Refer import display error if triggered while locked
- Browser content context menu options are now in a Zotero submenu and are disabled when Zotero is locked
- Sync, repository, and DB backup timers check for lock and bail
- If a new window is opened, Zotero pane can't be opened and display an error message until lock is released
Probably need to check lock in word processor integration code and advanced search window
Also:
- New method Zotero.sleep(ms) (currently unused) to allow synchronous code to sleep and allow events on main thread to be processed
- Support for group libraries
- General support for multiple libraries of different types
- Streamlined sync support
- Using solely libraryID and key rather than itemID, and removed all itemID-changing code
- Combined two requests for increased performance and decreased server load
- Added warning on user account change
- Provide explicit error message on SSL failure
- Removed snapshot and link toolbar buttons and changed browser context menu options and drags to create parent items + snapshots
- Closes#786, Add numPages field
- Fixes#1063, Duplicate item with tags broken in Sync Preview
- Added better purging of deleted tags
- Added local user key before first sync
- Add clientDateModified to all objects for more flexibility in syncing
- Added new triples-based Relation object type, currently used to store links between items copied between local and group libraries
- Updated zotero.org translator for groups
- Additional trigger-based consistency checks
- Fixed broken URL drag in Firefox 3.5
- Disabled zeroconf menu option (no longer functional)
Developer-specific changes:
- Overhauled data layer
- Data object constructors no longer take arguments (return to 1.0-like API)
- Existing objects can be retrieved by setting id or library/key properties
- id/library/key must be set for new objects before other fields
- New methods:
- ZoteroPane.getSelectedLibraryID()
- ZoteroPane.getSelectedGroup(asID)
- ZoteroPane.addItemFromDocument(doc, itemType, saveSnapshot)
- ZoteroPane.addItemFromURL(url, itemType)
- ZoteroPane.canEdit()
- Zotero.CollectionTreeView.selectLibrary(libraryID)
- New Zotero.URI methods
- Changed methods
- Many data object methods now take a libraryID
- ZoteroPane.addAttachmentFromPage(link, itemID)
- Removed saveItem and saveAttachments parameters from Zotero.Translate constructor
- translate() now takes a libraryID, null for local library, or false to not save items (previously on constructor)
- saveAttachments is now a translate() parameter
- Zotero.flattenArguments() better handles passed objects
- Zotero.File.getFileHash() (not currently used)
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
closes#165, verify import/export can carry all data for all fields and item types
closes#168, make sure MODS import works with files from external sources
