Previously, if a translator or style was fixed, people didn't get the
fix until their client checked the repository for updates, which could
take up to 24 hours. Now, in addition to checking once a day, we check
every time Zotero is started and also when we receive a notification
from the streaming server, which happens immediately after a translators
or style is updated on GitHub. To avoid DDoSing ourselves, the
notification includes a random delay (within a given period) before the
update is triggered by the client.
The streaming server connection is now made when either "Automatically
check for updated translators and styles" or "Sync automatically" is
enabled. It can be disabled via the extensions.zotero.streaming.enabled
pref.
Only reset delay generator after regular message, not on connect,
because a subscription failure happens after a connect. Also cancel
reconnect if auto-sync is disabled during the failure delay.
This required checking the file size of every file (including auxiliary
snapshot files) in every attachment to be uploaded, and it was an
estimate anyway, because snapshots are compressed, so we kept track of
the compression ratio, but there was no particular reason to think later
files would have the same ratio... In any case, we're not currently
displaying file sync progress in 5.0 anyway, and while we should, we can
probably just show a count of attachments remaining.
And use them in new importTextAttachment() and importHTMLAttachment()
test support functions. These can be used to avoid needing a hidden
browser for determining the character set of the imported text
documents.
Previously only Host: 127.0.0.1 was allowed. (The check prevents a DNS
rebinding attack.)
This allows requests to translation-server using localhost
(zotero/translation-server#48).
Fixes#125
Previously they were retried only on a backoff schedule and after a
client upgrade, but that would make it difficult to report errors
(because you'd see the error but it would then go away if you clicked
Sync again).
- Don't start checking for idle until UI is ready + 30 seconds
- Increase idle delay from 5 seconds to 30 seconds
- Increase interval from 100 ms to 200 ms
* Without this, preference change observers stack, triggering multiple
times if Zotero is re-initialized multiple times within the same
browser instance (like in tests)
* Use mocha, chai & sinon from the npm. As of sinon 2.0
sinon-as-promised is no longer required so it is removed
* Tweak code to re-use the same loader with the same environment
throghout the code
* Introduce browserify step for testing tools that only provide
node-compatible libraries (sinon, chai-as-promised)
* Introduce copy step for test data to resolve multiple issues with
tests depending on files not being symlinks
* Re-introduce custom implementation of setTimeout to resolve issues
with few tests
* Re-introduce custom Bluebird Promises config & monkey patch
* Add a multi-process, gulp-based build system to support es6 features,
async/await, jsx and scss
* Add a package.json to support dependency management and allow starting
the build process via npm
* Replace embedded Bluebird library with npm-installed one
* Add react, react-dom and web-library
* Introduce a custom require() loader in include.js as well as a minimal
local require() implementation in various other places
Previously, if an id was psased as a string and the id existed in the
cache, an error wouldn't be thrown, but if there id wasn't in the cache
(e.g., because it was in an unloaded library) it would. This requires an
integer in all cases.
Note that, among other things, any code that gets ids from object keys
will need to convert them to integers before passing to getAsync().
Services.locale.getApplicationLocale() -> getAppLocale()
(nsILocaleService -> mozILocaleService)
It's possible this will fix some of the locale resolution problems in
Standalone:
"This is actually a meaningful change because the old one was actually
using OS locale for platforms other than Windows. The new one is using
the app locale for all platforms." [1]
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1346674#c0
When refreshing the note editor, store the associated itemID on the
textbox, and check it when saving to make sure it's for the expected
note. This might prevent weird situations (which I can't reproduce, but
there was a report in the forums [1]) where something goes wrong and the
note doesn't refresh when selecting a new item, and then one note's
content is saved over another's.
[1] https://forums.zotero.org/discussion/65708/
While objects in the sync queue that fail to save should remain in the
queue, objects that just don't exist remotely need to be removed, or
else they'll be retried forever.
This makes it a little easier to use the proper plural form of a word.
Currently this only supports English forms and is used only in debug
output. For proper plural form support, use PluralForm.jsm.
There was a report [1] of slow file syncing that showed all file
timestamps being reported as ending with 020 (e.g., 1436492361020). The
previous code assumed that systems without millisecond precision would
report as whole seconds, so the timestamp checks weren't matching and it
was moving on to hash-based checks (which seemed to be taking a very
long time, but that's another matter). This changes the comparison so
that, as long as both timestamps floor to the same whole second, they'll
be considered equal.
[1] https://forums.zotero.org/discussion/65515/5-0-beta-sync-problem
Item data may not have been loaded for a library when requesting an item
from a document (e.g., for Refresh), so we need to load all data for
requested items to avoid unloaded-data errors. (Data isn't loaded if
it's already been loaded, so hopefully this doesn't slow things down too
much.)
d5cf33a798 adds a `yield` to bibliography.js, which runs in modal
windows (e.g., Create Bib), but there's a weird interaction between
Bluebird and modal dialogs that can result in hangs -- presumably
something to do with things being queued on the event loop but the modal
dialog preventing other code from running? This was breaking
bibliographyTest, but it seemed to work fine for me in normal usage,
waiting properly for a running styles initialization to finish. It's
possible this problem is limited to tests, but in the past, at least, I
apparently decided that this was a general problem with `yield` in modal
dialogs [1]. (See also: [2].) In any case, calling `yield
Zotero.Styles.init()` from the Create Bib window was hanging the test,
so for now do a synchronous check for style initialization to avoid it,
and we should make sure that `yield` actually works in other contexts.
[1] https://github.com/zotero/zotero/commit/99dd1c069776
[2] https://github.com/zotero/zotero/commit/c2dd531cec4
Instead of My Publications being a separate library, have it be a
special collection inside My Library. Top-level items can be dragged
into it as before, and child items can be toggled off and on with a
button in the item pane. Newly added child items won't be shown by
default.
For upgraders, items in the My Publications library will be moved into
My Library, which might result in their being duplicated if the items
weren't removed from My Library. The client will then upload those new
items into My Library.
The API endpoint will continue to show items in the separate My
Publications library until My Publications items are added to My
Library, so the profile page will continue to show them.
Contains a dummy doc plugin, which is useful for:
- Testing integration.js functionality
- Serving as succint documentation for development of new integration
plugins
I don't know why why, but wbp.saveDocument() destroys the
document object which is passed into it. This means that further access
of the document is impossible and raises errors, which we were not
handling properly and not returning on attachment saving.
I've added error handling and changed removed code which tries to access
the document after the Zotero.Utilities.Internal.saveDocument() call.
Addresses https://forums.zotero.org/discussion/64745/5-0-beta-apparently-stuck-at-saving-to
The web library will probably still display the old tag in addition to
the new one, at least until browser restart. We'll have to deal with
that separately.
Closes#1205
When refreshing, if fewer than 100 tags to show, just create them from
scratch instead of updating the full set. Otherwise, remove the full set
from DOM and add it back in after updates to avoid reflows (from #1204).
There are various things that could be done to optimize this further
(avoiding unnecessary sorting during full refreshes, calculating a hash
of the full set and not updating it every time), but we should probably
just replace it with @tnajdek's React version first.
Closes#1204
Changes `libraryTreeView::addEventListener('load')` and similar to
`libraryTreeView::onLoad.addListener(listener, once)`, etc. `once` is an
optional boolean that, when true, causes the listener to fire once and
then be removed. This is implicit for 'load'.
'load' maintains its special behavior of running immediately if the
treeview has already been loaded.
Also adds `waitForLoad()` and `waitForSelect()` functions that return
promises on event completion, since most uses of those events were just
resolving deferreds.
TODO: Add text fields for controlling number of items and chunk size
(Chunk size should be small for testing UI interactions and performance
but large for generating lots of data.)
For building up a big library and testing UI interaction during writes
To use, load chrome://zotero/content/tools/data_generator.html in
Firefox (for now).
Could be improved a bit to add other kinds of data (collections,
child items, relations)
The items list is generated from the database (via search), but new
items may have been added to the database but not yet been registered,
causing unloaded-data errors during sorting. Avoid that by not showing
unregistered items when generating the items list.
Additional protections are necessary -- it's still possible to get
errors, and maybe a crash, if an item has been registered but not yet
fully loaded -- but this addresses the most common one.
We could monkey-patch a finally function in connectors as described in
this gist https://gist.github.com/jish/e9bcd75e391a2b21206b , but it
appears to silence propogation of unhandled rejections
and it could later come back to bite us in the butt.
Native and proper support for finally is, however, planned in ES7.
When dragging an item to another library, we have to check if there's a
linked item in the target library, but items might not yet be laoded in
the other library, so item.getLinkedItem() can fail with "Item [n] not
yet loaded].
Fixing required asyncifying the follow functions:
- Zotero.Item::getLinkedItem()
- Zotero.Collection::getLinkedCollection()
- Zotero.URI.getURIItem()
- Zotero.URI.getURICollection()
- Various integration functions
As noted in 27cb099c82, import translators should be rewritten to return
a promise from doImport() and wait for promises from successive
item.complete() calls. They should then be marked as minVersion: "5.0"
to be handled properly by this new code.
(But this tries to account, albeit with somewhat worse behavior, for
translators that haven't been rewritten and sandboxes without Promise
(which is currently the case with child sandboxes in the client).)
(Oh, and I haven't tested this at all in the connectors.)
Function that automatically follows Link headers and makes additional
requests as necessary, feeding the XMLHTTPRequest object from each
request to a reducer function.
- Archive remotely missing that user chooses to keep
- Ignore archived groups that don't existing remotely
- Unarchive groups that become available again
This can happen because objects are uploaded in batches, but the version
number from the last batch is used for all objects (since
APIClient.uploadObjects() handles batching internally).
Previously on Windows, where we don't have /bin/mv, we were recursing
into the data directory and copying files individually, which is very
slow, so automatic migration was disabled. Instead, try moving
directories with OS.File.move() with the `noCopy` flag. Moving
directories is technically unsupported by OS.File, but probably only
because of the possibility of a cross-volume copy (which is only
implemented for some platforms), and using `noCopy` hopefully prevents
that. If someone does have their data directory or storage directory on
a different volume, the migration might be quite slow, but leaving a
data directory behind in the Firefox profile directory (where it can be
easily misplaced with a seemingly unrelated Firefox reset) is worse.
E.g., moving 3,600 items to the trash now takes 4 seconds instead of 62
Instead of saving each item, update internal state and database directly
(which is more brittle but worth it). Also avoid unnecessary sorting
after removing an item from the items tree.
This allows for imported items to be saved individually instead of being
saved in a batch after processing the entire imported file (which for
large imports would hang the UI, even if the actual saving was
asynchronous). This also fixes the progress meter during asynchronous
saves.
To take advantage of this, import translators will need to return a
promise when available (using the native Promise object) from doImport()
and wait for optional promises from item.complete().
The logic here can probably be streamlined further. (E.g., we might be
able to say that item.complete() always returns a promise.) It's
complicated by the fact that, at the moment, Promise isn't available in
child sandboxes, though this can probably be fixed.
Tests forthcoming, but they require a translator that supports this,
which needs to be committed separately.
View with -w for a cleaner diff.
There seems to be a bug here where, if the item is off-screen when
modified, it doesn't become visible, but it now does stay visible when
it's on-screen to begin with.
At least until we have a better interface for setting library-specific
sync settings
Previously, if My Library file syncing was disabled or set to WebDAV, My
Publications files wouldn't be uploaded.
When adding many search conditions (e.g., when matching many items with the
`key` condition), the query can fail due to either the bound parameter limit or
the expression tree size limit.
To avoid this, add support for an 'inlineFilter' property on search conditions
when using the 'is' or 'isNot' operator. 'inlineFilter' is a function that
returns a quoted value suitable for direct embedding in the SQL statement, or
false if not valid. Multiple consecutive conditions for the same 'inlineFilter'
field are combined into an `IN (x, y, z)` condition.
- Add Cmd/Ctrl-Shift-I for "Import…"
- Cmd/Ctrl-Shift-V apparently pastes without formatting, so don't override it.
Instead, add a shortcut key to the menu option, defaulting to
Cmd-Option-Shift-I/Ctrl-Alt-Shift-I
- The I in both is now localizable
