- The Mozilla CommonJS loader is no longer available, so bundle the
Fx52 version of it
- Strict mode is enforced
- `this` is only defined as a global object in .jsm files, not .js files
- `this` can't be converted to a string for BackstagePass test, so check
for presence of Components.utils.import instead
- The return value from import() is no longer available
- Check for retracted items using data from Retraction Watch
- Show an X next to retracted items in the items list, and show a
scary message at the top of the item pane with more info and links.
- Lookup is done in a privacy-preserving manner using k-anonymity --
the server is unable to determine the specific items that exist in
the client, so people who don't sync don't need to share any library
data (though the server doesn't log the lookups anyway).
TODO:
- Pop up an alert when new items are found
- Show a confirmation prompt when citing a retracted item
- Support items without DOIs or PMIDs
- Add a proper PMID field and expand DOI to more item types so these
values don't need to be parsed out of Extra
- Clear the banner immediately when all possible fields are cleared
instead of waiting a few seconds
- Support items within collections and searches:
zotero://select/library/collections/:collectionKey/items/:itemKey
zotero://select/groups/:groupID/collections/:collectionKey/items/:itemKey
- Fix the 'itemKey' parameter:
zotero://select/library/collections/:collectionKey/items?itemKey=:itemKey1,:itemKey2
- Select library root if collection/search not specified
This adds selectItems() to ZoteroPane and collectionTreeView and removes
the ancient, unused 'expand' argument to selectItem(), which didn't
really make sense there. It also includes a new
itemTreeView::ensureRowsAreVisible() that tries to scroll to an
appropriate place (or, better yet, not scroll at all) given the
specified rows and page size.
Currently only .status and .getResponseHeader() (for getting 'Location')
are available in the returned object, but we could make the body
available if necessary.
Move `Zotero.getString()` and intl init code to `Zotero.intl` to make
it easier to re-use.
Link `Zotero.getString()` to `Zotero.intl.getString()`.
Do not expose `getStringFromBundle`, `pluralFormGet`, and
`pluralFormNumForms` because they are not used.
This is loosely based on the same functionality in ZotFile, but it tries
to do the right thing based on existing Zotero settings: either the new
PDF handler setting in the prefs or the system-default app. The latter
can only reliably be determined on Windows (and this uses ZotFile's
function to read that from the registry), but this tries to figure it
out on macOS and Linux too using the Mozilla handler service. (The
handler service only gets you an app name, not a path, so on Linux we
can try reading mimetypes.list and the like in case someone is using a
system-default okular or evince not in /usr/bin, but that's not yet
implemented.)
This uses the new 5.0 URL format, and a 'page' query parameter instead
of a path component:
zotero://open-pdf/library/items/[itemKey]?page=[page]
zotero://open-pdf/groups/[groupID]/items/[itemKey]?page=[page]
It also accepts ZotFile-style URLs, though, so if you uninstall ZotFile
you should still be able to open those links. ZotFile will need to
accept the new format for new links to work when ZotFile is installed,
since it will override this handler.
This functionality will be necessary for annotation extraction (#1018)
and for imported annotations from Mendeley (#1451).
And fix deprecation warning from passing an nsIFile
(Also updates the Zotero.Styles.install() documentation to note that the
first parameter's `file` property can be a string path.)
Can be an absolute path or 'profile' to use 'zotero' subdirectory of
profile directory as in earlier versions (but which won't be treated as
a legacy location eligible for migration)
Closes#1305
We were manually looking up the current locale in various places and
passing it to nsIStringBundleService::createBundle(), but that hasn't
even been a supported argument for years. (I assume it was a long time
ago?)
Previously, if a translator or style was fixed, people didn't get the
fix until their client checked the repository for updates, which could
take up to 24 hours. Now, in addition to checking once a day, we check
every time Zotero is started and also when we receive a notification
from the streaming server, which happens immediately after a translators
or style is updated on GitHub. To avoid DDoSing ourselves, the
notification includes a random delay (within a given period) before the
update is triggered by the client.
The streaming server connection is now made when either "Automatically
check for updated translators and styles" or "Sync automatically" is
enabled. It can be disabled via the extensions.zotero.streaming.enabled
pref.
* Use mocha, chai & sinon from the npm. As of sinon 2.0
sinon-as-promised is no longer required so it is removed
* Tweak code to re-use the same loader with the same environment
throghout the code
* Introduce browserify step for testing tools that only provide
node-compatible libraries (sinon, chai-as-promised)
* Introduce copy step for test data to resolve multiple issues with
tests depending on files not being symlinks
* Re-introduce custom implementation of setTimeout to resolve issues
with few tests
* Re-introduce custom Bluebird Promises config & monkey patch
Follow-up to https://github.com/zotero/zotero-standalone-build/commit/22cfe2404
The devtools must be included in the build by passing the -t flag to
build.sh in zotero-standalone-build.
After starting Zotero with the flag, connect from a Firefox instance
with remote debugging enabled by choosing "Connect…" from the Web
Developer menu and connecting to port 6100.
Since 1) debug output logging via the prefs isn't necessarily possible
for startup errors in Standalone, 2) real-time output is prohibitively
slow and has a miniscule scrollback buffer on Windows unless you use a
Cygwin or Git terminal, and 3) copying/pasting/emailing was annoying
anyway, make -ZoteroDebug open a popup window that shows errors and
debug output and allows submitting straight to the server with a Debug
ID.
This should replace the existing debug output viewer as well, but that's
less of a priority.
-ZoteroDebugText or the debug.log pref can still be used to dump to the
terminal.
Immediate sync triggering on remote library change using WebSocket API.
Currently kicks off a normal sync process for the modified library --
actual object data isn't pushed.
(This might not stay enabled for 5.0 Final.)
This is a bit iffy, since at least on macOS the passed file seems to
come in as either -file or -url at different times, but this seems to
work on macOS and Windows. Needs further testing with a packaged version
on Windows.
Improves proxy support
- Automatically detect and dehyphenise https proxies which use EZProxy
HttpsHyphens
- Web translators now pass around Zotero.Proxy instances which can
proxify/deproxify urls passed to `translate.setLocation()` before calling
`translate.getTranslators()`/ translate.detect()`. The proxy passing is
done within connector background/injected processes and between
standalone and connectors.
- Proxy protocol unified with connectors. Connectors can now pass
proxies to `/connector/save_items`. The proxies will be used to resolve
true item and attachment urls when saving.
Closeszotero/zotero#578, zotero/zotero#721
Relevant zotero/zotero#34, zotero/zotero#556
Look for other profiles, from both apps (Firefox and Standalone), that
point to the data directory being migrated and update prefs.js in those
profiles to point to the new location.
Also reorganize code into Zotero.Profile and Zotero.DataDirectory
namespaces
And don't skip alerts in Zotero.alert() during automated tests. (That
was intended to avoid long timeouts after unexpected failures, but,
e.g., PDF metadata lookups (which are currently disabled in automated
tests) should just be mocked so they don't intermittently fail.)
Don't start the connector pipe (Mac/Linux), don't listen for IPC on the
command-line (Windows), and don't try to release Standalone's lock if DB is
busy when Firefox starts.
- Use custom exception for user-initiated sync cancellations, which can bubble
up to the sync runner -- this should help with a sync stop button (#915)
- Separate out deletions-downloading code
- Refactor delay generator handling on library version mismatch
- Clearer variable names
.contains() was removed in Firefox 48, but .includes() wasn't available until
40, so use indexOf() for now. We can start using .contains() once we no longer
need to support 38 ESR.
While trying to get translation and citing working with asynchronously
generated data, we realized that drag-and-drop support was going to
be...problematic. Firefox only supports synchronous methods for
providing drag data (unlike, it seems, the DataTransferItem interface
supported by Chrome), which means that we'd need to preload all relevant
data on item selection (bounded by export.quickCopy.dragLimit) and keep
the translate/cite methods synchronous (or maintain two separate
versions).
What we're trying instead is doing what I said in #518 we weren't going
to do: loading most object data on startup and leaving many more
functions synchronous. Essentially, this takes the various load*()
methods described in #518, moves them to startup, and makes them operate
on entire libraries rather than individual objects.
The obvious downside here (other than undoing much of the work of the
last many months) is that it increases startup time, potentially quite a
lot for larger libraries. On my laptop, with a 3,000-item library, this
adds about 3 seconds to startup time. I haven't yet tested with larger
libraries. But I'm hoping that we can optimize this further to reduce
that delay. Among other things, this is loading data for all libraries,
when it should be able to load data only for the library being viewed.
But this is also fundamentally just doing some SELECT queries and
storing the results, so it really shouldn't need to be that slow (though
performance may be bounded a bit here by XPCOM overhead).
If we can make this fast enough, it means that third-party plugins
should be able to remain much closer to their current designs. (Some
things, including saving, will still need to be made asynchronous.)
This mostly gets ZFS file syncing and file conflict resolution working
with the API sync process. WebDAV will need to be updated separately.
Known issues:
- File sync progress is temporarily gone
- File uploads can result in an unnecessary 412 loop on the next data
sync
- This causes Firefox to crash on one of my computers during tests,
which would be easier to debug if it produced a crash log.
Also:
- Adds httpd.js for use in tests when FakeXMLHttpRequest can't be used
(e.g., saveURI()).
- Adds some additional test data files for attachment tests
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
There's a lot more to do, and this isn't ready for actual usage, but the
basic functionality is mostly in place and has decent test coverage. It
can successfully upgrade a library last used with classic syncing and
pull down changes via the API. Uploading mostly works but is currently
disabled for safety until it has better test coverage.
Downloaded JSON is first saved to a cache table, which is then used to
populate other tables and later for generating PATCH requests and
automatically resolving conflicts (since it shows what was changed
locally and what was changed remotely). Objects with unmet dependencies
or unknown fields are skipped for now but don't block the rest of the
sync.
Some of the bigger remaining to-dos:
- Tests for uploading
- Re-do the preferences to get an API key
- File sync integration
- Full-text syncing integration
- Manual conflict resolution (though this already includes much smarter
conflict handling that automatically resolves many conflicts)
And use it in resetDB() test support function, mainly to allow
skipBundledFiles for resetDB calls. Translator installation and
initialization can take a long time, but tests that need a clean DB
don't necessarily rely on translators. Without this, running resetDB()
in beforeEach() for many tests is prohibitively slow.
Since modal windows (e.g., the Create Bib window and the Quick Copy site
editor window) can't use yield, style retrieval
(Zotero.Styles.getVisible()/getAll()) is now synchronous, depending on a
previous async Zotero.Styles.init(). The translator list is generated in
the prefs window and passed into the Quick Copy site editor, but it's
possible the translators API should be changed to make getTranslators()
synchronous with a prior init() as well.
- Simplified schema
- Tags are now added without reloading entire tag selector
- On my system, adding 400 tags to an item (separately, with the tag
selector updating each time) went from 59 seconds to 42. (Given that
it takes only 13 seconds with the tag selector closed, though,
there's clearly more work to be done.)
- Tag selector now uses HTML flexbox (in identical fashion, for now, but
with the possibility of fancier changes later, and with streamlined
logic thanks to the flexbox 'order' property)
- Various async fixes
- Tests
Get rid of data_access.js, at long last. Existing calls to
Zotero.getCollections() will need to be replaced with
Zotero.Collections.getByLibrary() or .getByParent().
Also removes Zotero.Collection::getCollections(), which is redundant
with Zotero.Collections.getByLibrary(), and Zotero.Collections.add().
The latter didn't didn't include a libraryID anyway, so code might as
well just use 'new Zotero.Collection' instead.
Groups were already being loaded for the collections list, so we might
as well just store them initially and let Zotero.Libraries.getName() be
a synchronous call.
'b' for *b*undled files
Translators and styles take a long time to install and initialize in
source installations, and they're unnecessary for many tests.
This shaves about 10 seconds off each test run for me on one system (and
that's with some help from filesystem caching).
