And fix deprecation warning from passing an nsIFile
(Also updates the Zotero.Styles.install() documentation to note that the
first parameter's `file` property can be a string path.)
Can be an absolute path or 'profile' to use 'zotero' subdirectory of
profile directory as in earlier versions (but which won't be treated as
a legacy location eligible for migration)
Closes#1305
We were manually looking up the current locale in various places and
passing it to nsIStringBundleService::createBundle(), but that hasn't
even been a supported argument for years. (I assume it was a long time
ago?)
Previously, if a translator or style was fixed, people didn't get the
fix until their client checked the repository for updates, which could
take up to 24 hours. Now, in addition to checking once a day, we check
every time Zotero is started and also when we receive a notification
from the streaming server, which happens immediately after a translators
or style is updated on GitHub. To avoid DDoSing ourselves, the
notification includes a random delay (within a given period) before the
update is triggered by the client.
The streaming server connection is now made when either "Automatically
check for updated translators and styles" or "Sync automatically" is
enabled. It can be disabled via the extensions.zotero.streaming.enabled
pref.
* Use mocha, chai & sinon from the npm. As of sinon 2.0
sinon-as-promised is no longer required so it is removed
* Tweak code to re-use the same loader with the same environment
throghout the code
* Introduce browserify step for testing tools that only provide
node-compatible libraries (sinon, chai-as-promised)
* Introduce copy step for test data to resolve multiple issues with
tests depending on files not being symlinks
* Re-introduce custom implementation of setTimeout to resolve issues
with few tests
* Re-introduce custom Bluebird Promises config & monkey patch
Follow-up to https://github.com/zotero/zotero-standalone-build/commit/22cfe2404
The devtools must be included in the build by passing the -t flag to
build.sh in zotero-standalone-build.
After starting Zotero with the flag, connect from a Firefox instance
with remote debugging enabled by choosing "Connect…" from the Web
Developer menu and connecting to port 6100.
Since 1) debug output logging via the prefs isn't necessarily possible
for startup errors in Standalone, 2) real-time output is prohibitively
slow and has a miniscule scrollback buffer on Windows unless you use a
Cygwin or Git terminal, and 3) copying/pasting/emailing was annoying
anyway, make -ZoteroDebug open a popup window that shows errors and
debug output and allows submitting straight to the server with a Debug
ID.
This should replace the existing debug output viewer as well, but that's
less of a priority.
-ZoteroDebugText or the debug.log pref can still be used to dump to the
terminal.
Immediate sync triggering on remote library change using WebSocket API.
Currently kicks off a normal sync process for the modified library --
actual object data isn't pushed.
(This might not stay enabled for 5.0 Final.)
This is a bit iffy, since at least on macOS the passed file seems to
come in as either -file or -url at different times, but this seems to
work on macOS and Windows. Needs further testing with a packaged version
on Windows.
Improves proxy support
- Automatically detect and dehyphenise https proxies which use EZProxy
HttpsHyphens
- Web translators now pass around Zotero.Proxy instances which can
proxify/deproxify urls passed to `translate.setLocation()` before calling
`translate.getTranslators()`/ translate.detect()`. The proxy passing is
done within connector background/injected processes and between
standalone and connectors.
- Proxy protocol unified with connectors. Connectors can now pass
proxies to `/connector/save_items`. The proxies will be used to resolve
true item and attachment urls when saving.
Closeszotero/zotero#578, zotero/zotero#721
Relevant zotero/zotero#34, zotero/zotero#556
Look for other profiles, from both apps (Firefox and Standalone), that
point to the data directory being migrated and update prefs.js in those
profiles to point to the new location.
Also reorganize code into Zotero.Profile and Zotero.DataDirectory
namespaces
And don't skip alerts in Zotero.alert() during automated tests. (That
was intended to avoid long timeouts after unexpected failures, but,
e.g., PDF metadata lookups (which are currently disabled in automated
tests) should just be mocked so they don't intermittently fail.)
Don't start the connector pipe (Mac/Linux), don't listen for IPC on the
command-line (Windows), and don't try to release Standalone's lock if DB is
busy when Firefox starts.
- Use custom exception for user-initiated sync cancellations, which can bubble
up to the sync runner -- this should help with a sync stop button (#915)
- Separate out deletions-downloading code
- Refactor delay generator handling on library version mismatch
- Clearer variable names
This mostly gets ZFS file syncing and file conflict resolution working
with the API sync process. WebDAV will need to be updated separately.
Known issues:
- File sync progress is temporarily gone
- File uploads can result in an unnecessary 412 loop on the next data
sync
- This causes Firefox to crash on one of my computers during tests,
which would be easier to debug if it produced a crash log.
Also:
- Adds httpd.js for use in tests when FakeXMLHttpRequest can't be used
(e.g., saveURI()).
- Adds some additional test data files for attachment tests
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
There's a lot more to do, and this isn't ready for actual usage, but the
basic functionality is mostly in place and has decent test coverage. It
can successfully upgrade a library last used with classic syncing and
pull down changes via the API. Uploading mostly works but is currently
disabled for safety until it has better test coverage.
Downloaded JSON is first saved to a cache table, which is then used to
populate other tables and later for generating PATCH requests and
automatically resolving conflicts (since it shows what was changed
locally and what was changed remotely). Objects with unmet dependencies
or unknown fields are skipped for now but don't block the rest of the
sync.
Some of the bigger remaining to-dos:
- Tests for uploading
- Re-do the preferences to get an API key
- File sync integration
- Full-text syncing integration
- Manual conflict resolution (though this already includes much smarter
conflict handling that automatically resolves many conflicts)
And use it in resetDB() test support function, mainly to allow
skipBundledFiles for resetDB calls. Translator installation and
initialization can take a long time, but tests that need a clean DB
don't necessarily rely on translators. Without this, running resetDB()
in beforeEach() for many tests is prohibitively slow.
Get rid of data_access.js, at long last. Existing calls to
Zotero.getCollections() will need to be replaced with
Zotero.Collections.getByLibrary() or .getByParent().
Also removes Zotero.Collection::getCollections(), which is redundant
with Zotero.Collections.getByLibrary(), and Zotero.Collections.add().
The latter didn't didn't include a libraryID anyway, so code might as
well just use 'new Zotero.Collection' instead.
'b' for *b*undled files
Translators and styles take a long time to install and initialize in
source installations, and they're unnecessary for many tests.
This shaves about 10 seconds off each test run for me on one system (and
that's with some help from filesystem caching).
- Protocol handler extensions can now handle promises and can also make
data available as it's ready instead of all at once (e.g., reports now
output one entry at a time)
- zotero:// URL syntaxes are now more consistent and closer to the web
API (old URLs should work, but some may currently be broken)
Also:
- Code to generate server API, currently available for testing via
zotero://data URLs but eventually moving to HTTP -- zotero://data URLs match
web API URLs, with a different prefix for the personal library (/library vs.
/users/12345)
- Miscellaneous fixes to data objects
Under the hood:
- Extensions now return an AsyncChannel, which is an nsIChannel implementation
that takes a promise-yielding generator that returns a string,
nsIAsyncInputStream, or file that will be used for the channel's data
- New function Zotero.Utilities.Internal.getAsyncInputStream() takes a
generator that yields either promises or strings and returns an async input
stream filled with the yielded strings
- Zotero.Router parsers URLs and extract parameters
- Zotero.Item.toResponseJSON()
This required doing additional caching at startup (e.g., item types and fields)
so that various methods can remain synchronous.
This lets us switch back to using the current Sqlite.jsm. Previously we were
bundling the Fx24 version, which avoided freezes with locking_mode=EXCLUSIVE
with both sync and async queries.
Known broken things:
- Autocomplete
- Database backup
- UDFs (e.g., REGEXP function used in Zotero.DB.getNextName())
Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
Cmd on OS X, Shift on Windows/Linux
How do I not get to close a ticket for this?
Unfortunately on Windows it doesn't seem possible to set the cursor
effect to arbitrary states (see note in libraryTreeView.js::
_setDropEffect() for the gory details), so this just uses the default
cursor there. On OS X and Linux the cursor reflects the requested
action.
This should make it much easier to debug startup errors, particularly in
Standalone.
This also adds a general mechanism to set Zotero initialization options via
command-line flags.
- Close Zotero pane before database is closed prior to reload, instead
of waiting until reload is complete
- Show an error message if Zotero Standalone is not accessible when it
should be
Cmd on OS X, Shift on Windows/Linux
How do I not get to close a ticket for this?
Unfortunately on Windows it doesn't seem possible to set the cursor
effect to arbitrary states (see note in libraryTreeView.js::
_setDropEffect() for the gory details), so this just uses the default
cursor there. On OS X and Linux the cursor reflects the requested
action.
Other changes:
- Factored out Zotero.Translators from Zotero.Translator. The latter
should be usable in the bookmarklet and connectors without changes.
- configOptions, displayOptions, and hiddenPrefs no longer copy on
read. I don't think this actually affects any existing code.
- Zotero.Translate._loadTranslator() now returns a promise
This change should improve Firefox startup time. If the Zotero pane is
opened before Zotero has initialized, the pane will be blocked with a
progress bar until it's done. Standalone currently does the same, but it
should be changed to just delay opening the window if possible.
The upgrade wizard has been disabled for schema upgrades, in the hope
that upgrades can be done in a way that won't break compatibility with
earlier versions (or that can at least be done in a way such that we can
put out point releases of the last major version that provide
compatibility during beta/post-upgrade periods).
This patch likely breaks many things, and definitely breaks connector
mode.
This change also removes upgrade steps for databases from Zotero 2.1b2
and earlier. Users with such databases will need to upgrade via Zotero
4.0.x first or delete their data directories and start anew. This should
only affect users who haven't opened Zotero since Nov. 2010.
- New tag colors support, with the ability to assign colors to up to 6
tags per library. Tags with colors assigned will show up at the top of
the tag selector and can be added to (and removed from) selected items
by pressing the 1-6 keys on the keyboard. The tags will show up as
color swatches before an item's title in the items list.
- Synced settings, with Notifier triggers when they change and
accessible via the API (currently restricted on the server to
'tagColors', but available for other things upon request)
- Silent DB upgrades for backwards-compatible changes. We'll do
something fancier with async DB queries in 4.0, but this will work for
changes that can be made without breaking compatibility with older
clients, like the creation of new tables. The 'userdata' value is
capped at 76, while further increments go to 'userdata2'.
TODO:
- Try to avoid jitter when redrawing swatches
- Optimize tag color images for retina displays
- Redo attachment dots in flat style?
- Clear all colors from an item with 0 (as in Thunderbird), but I don't
think we can do this without undo
- New promise-based architecture
- Library-specific file sync queues, allowing other libraries to
continue if there's an error in one library
- Library-specific sync errors, with error icons next to each library
- Changed file uploading in on-demand download mode, which had been missing
- On-demand download progress indicator in middle pane
- More accurate progress indicator
- Various tweaks and bug fixes
- Various future tweaks and bug fixes
Adds three new preferences controlling whether a new collection is created upon import:
- extensions.zotero.import.createNewCollection.fromFile
Controls whether a new collection is created by import from cog menu
- extensions.zotero.import.createNewCollection.fromClipboard
Controls whether a new collection is created by import from clipboard
- extensions.zotero.import.createNewCollection.fromFileOpenHandler
Controls whether a new collection is created when import is initiated by
double-clicking a file or by dragging it over Zotero. This preference applies only to
Zotero Standalone, and is configurable in the dialog that appears asking the user
to confirm the import.
Closes#19, [papercuts] (26) Clipboard import into an open collection
