extensions.zotero.sync.debugUploadPolicy = {1, 2}
1 to prompt, 2 to throw
This can be used to test sync functionality without altering the server
state unexpectedly. When prompting, the request body, if any, is logged
to debug output.
Previously, item-dependent menu options in the context menu of
collectionsTreeRow types other than 'collection' weren't updated after
the asynchronous loading of the items. The menu is now updated for all
types, so, e.g., "Empty Trash" is now updated properly after items in
the trash have been loaded.
- Fix upgrading of Mozilla-style attachments/storage file paths on upgrade
(requires re-upgrade)
- Save relative paths using forward slashes for consistency, and convert
to platform-appropriate slashes on use
Don't start the connector pipe (Mac/Linux), don't listen for IPC on the
command-line (Windows), and don't try to release Standalone's lock if DB is
busy when Firefox starts.
- Fixes#994, 5.0: "+" doesn't expand all collections within a library
- If a container (library, collection) is closed directly, the open state of
all containers below it are now restored when it's reopened. Previously all
collections would be closed on a manual reopen (though they might have been
restored on the next Zotero restart).
- If "-" is pressed, all containers are closed, and reopening the library will
show only top-level collections.
- Use custom exception for user-initiated sync cancellations, which can bubble
up to the sync runner -- this should help with a sync stop button (#915)
- Separate out deletions-downloading code
- Refactor delay generator handling on library version mismatch
- Clearer variable names
Return false for single ids or skip for multiple ids. This is the original
behavior, but at some point it started throwing an UnloadedDataException. IDs
are always loaded at initialization, though, so we know whether the objects
actually exist.
- Use an increasing notifier batch size, so objects initially appear one by one
but then start showing up in batches, up to 50. (UI updates are expensive, so
for larger syncs we don't want to update after each object.)
- Avoid separate save to update attachment file sync state, which was also
happening outside of notifier batches (causing individual updates regardless
of the batch size)
- Add a 10ms delay after processing each object, which keeps the UI responsive
during downloads. #989 could reduce this to 1 during idle, to save a few
minutes when downloading very large libraries.
.contains() was removed in Firefox 48, but .includes() wasn't available until
40, so use indexOf() for now. We can start using .contains() once we no longer
need to support 38 ESR.
This is necessary to get a library version after the write instead of an
item version. Otherwise after a full-text write, the main library
version is behind, so the next sync checks all object types for that
library instead of getting a 304.
Full text is batched up to 500K characters or 10 items, whichever is
less.
This also switches to using ?format=versions for /fulltext requests,
which isn't currently necessary but reflects what it's actually doing.
If a version is returned for an item's full-text content but a 404 is returned
for the content itself (because it's missing in Elasticsearch for some reason),
don't throw an error.
Also remove legacy array comprehensions in fulltext and syncFullTextEngine test
files, which apparently weren't being run.
Addresses #980. There are a couple other uses of the old API, but I'm not sure
how they're triggered, and we might be gone from Firefox before this is
relevant.
This is necessary because you can copy a database synced with a
different account into the data directory without affecting the stored
pref.
Also tweak the text to use proper quotes and remove quaint references to
"the server".
A queue can be created and passed as an option to data layer methods, which
will then queue events on that queue instead of the main internal queue. A
queue or an array of queues can then be passed to Zotero.Notifier.commit() to
commit those events.
Some auxiliary functions don't yet take a queue, so those events will still get
run on DB transaction commit.
Sync data processing now processes notifier events in batches to reduce
repaints, even though individual objects are processed within their own
transactions (so that failures don't roll back other objects' data).
Also remove some unused notifier code
I'm not sure this matters -- since it was broken, itemTreeView notify()
code probably accounts for situations where, say, a 'modify' comes in
before an 'add' -- but it might avoid some edge cases in the future.
It's generally not necessary to call this, but it can be useful if
specific data needs to be loaded in an unloaded library (e.g., an item's
creators for RTF Scan)
`nArcs` is undefined in `this.ItemProgress.prototype.setProgress()`. This has been the case for a long time but never came up because the progress indicator isn't used in Zotero, I believe. I think it's a nice UI element though and could be used for non-disruptive progress indicators.
- Only show first pane (without Sharing pane) if no files or files
aren't included
- Update authorship checkbox to reflect file include setting to avoid
confusion
- Clarify in intro text that license applies to files, not notes --
notes here are no different than notes in any public library, so it's
sort of up to the user to clarify those if they're substantial enough
for it to matter
- Adjust alignment of authorship checkbox if more than one line
Otherwise, when right-clicking on a collection that's not currently
selected, some of the menu items appear gray at first, and a second
right-click is necessary after the items have loaded. This way the menu
items turn black once the items have loaded.
Items in a library are now loaded only when a library is clicked on and
at sync time. There might be some other areas where they need to be
loaded or where this causes problems (e.g., drag and drop, word
processor integration).
I think this only happens if the Zotero pane hasn't yet been opened, which also
means an initial refresh() hasn't been done, which means that updates aren't
necessary.
Otherwise the test could run scrapeThisPage() before translators were ready. It
would be good to make scrapeThisPage wait for detection to complete so that an
early press still uses a translator for saving, but this way tests can also
test for the proper icon (though they don't now).
Instead of requiring a click on the menuitem at the top of the submenu, allow
a click on the menu itself.
This is a hack that, among other things, replicates the flash effect on
menuitems on OS X. Unfortunately, <menu> elements can't have checkboxes, so
only the menuitem in the submenu will be checked. (Otherwise I'd remove the
redundant menuitem in the submenu.)
- Allow clicking on non-editable abstract field to expand/collapse it
- Change cursor when hovering over abstract to show it can be toggled
- Default abstracts to expanded
- Fix persistence of last translation target
- Add checkbox to menuitem of selected target
- Remove unnecessary flex attributes
Also:
- Move collectionTreeView row id (e.g., "L1", "C123") and image
generation to Zotero.Library and Zotero.Collection properaties,
.collectionTreeViewID and .collectionTreeViewImage -- currently used
only for the feed add-to button, but could be expanded for use in
collectionTreeView
And without extra init() call
This fixes the button being initialized before Zotero.hiDPI is set, resulting
in a low-res icon until the target is changed.
The on-change auto-sync now syncs only the modified library, and does so
quite efficiently (and should be able to be made more efficient), so we
might be able to reduce the timeout below 15 seconds.
Instead of getting batches of unused primary key ids, even if they're lower
than other ids, which for some reason seemed like a good idea in 2008, just do
a `MAX()` on the table at startup and return the next available id on each call
to `Zotero.ID.get()`. This is much simpler, and not reusing ids allows them to
be used as a chronological sort field.
While SQLite's `SELECT last_insert_rowid()` could return auto-increment values,
it's unsafe with async DB access, since a second `INSERT` can come in before
the first `last_insert_rowid()` is called. This is true even in a transaction
unless a function that calls it is never called in parallel (e.g., with
`Zotero.Promise.all()`, which can be faster than sequential `yield`s).
Note that the next id is always initialized as MAX() + 1, so if an object is
added and then deleted, after a restart the same id will be given. (This is
equivalent to (though unrelated to) SQLite's `INTEGER PRIMARY KEY` behavior,
as opposed to its `INTEGER PRIMARY KEY AUTOINCREMENT` behavior.)
Closes#993, Feed items out of order
Previously, objects were first downloaded and saved to the sync cache,
which was then processed separately to create/update local objects. This
meant that a server bug could result in invalid data in the sync cache
that would never be processed. Now, objects are saved as they're
downloaded and only added to the sync cache after being successfully
saved. The keys of objects that fail are added to a queue, and those
objects are refetched and retried on a backoff schedule or when a new
client version is installed (in case of a client bug or a client with
outdated data model support).
An alternative would be to save to the sync cache first and evict
objects that fail and add them to the queue, but that requires more
complicated logic, and it probably makes more sense just to buffer a few
downloads ahead so that processing is never waiting for downloads to
finish.
- Hide notes, tags and related for feed items in itembox
- Add feed support for <enclosure> elements
- Add feed syncing methods for synced settings (additional work is
needed on the sync architecture to download synced settings from the
server)
- Change feed item clear policy to be less aggressive
- Adjust for deasyncification
- Disable translate-on-select
- Closeadomasven/zotero#7, Remove context menu items from feeds
This reverts Zotero.Translate.ItemGetter.prototype.nextItem() to being
synchronous post-deasyncification. This will need to be made to work
asynchronously in the future if _attachmentToArray(), which is called by
nextItem, is changed to use async file access (which might be required
at some point).
Addresses #734, [Async DB] Import/export fails
notificationCallbacks no longer QIs to webNavigation in Firefox 46, so
this gets the relevant browser object by getting the content window id
from channel.loadInfo and finding the window via nsIWindowMediator.
Unfortunately the window id provided under e10s is invalid (or something
-- it's a very high number that can't be found via nsIWindowMediator),
so something else will need to be done for that.
This allows Zotero.Relations.getByPredicateAndObject()/getByObject() and
Zotero.Item::getLinkedItem()/Zotero.Collection::getLinkedCollection() to
be synchronous, which is necessary for word processor integration.
Instead of creating a duplicate copy of the item with the same primary
data and saving that, it's safer just to use clone() (which doesn't
preserve ids) and apply changes to the main object.
While trying to get translation and citing working with asynchronously
generated data, we realized that drag-and-drop support was going to
be...problematic. Firefox only supports synchronous methods for
providing drag data (unlike, it seems, the DataTransferItem interface
supported by Chrome), which means that we'd need to preload all relevant
data on item selection (bounded by export.quickCopy.dragLimit) and keep
the translate/cite methods synchronous (or maintain two separate
versions).
What we're trying instead is doing what I said in #518 we weren't going
to do: loading most object data on startup and leaving many more
functions synchronous. Essentially, this takes the various load*()
methods described in #518, moves them to startup, and makes them operate
on entire libraries rather than individual objects.
The obvious downside here (other than undoing much of the work of the
last many months) is that it increases startup time, potentially quite a
lot for larger libraries. On my laptop, with a 3,000-item library, this
adds about 3 seconds to startup time. I haven't yet tested with larger
libraries. But I'm hoping that we can optimize this further to reduce
that delay. Among other things, this is loading data for all libraries,
when it should be able to load data only for the library being viewed.
But this is also fundamentally just doing some SELECT queries and
storing the results, so it really shouldn't need to be that slow (though
performance may be bounded a bit here by XPCOM overhead).
If we can make this fast enough, it means that third-party plugins
should be able to remain much closer to their current designs. (Some
things, including saving, will still need to be made asynchronous.)
If 'pdf' flag is included in object POSTed to saveSnapshot, import the
PDF directly and save as top-level item. Currently the PDF is
redownloaded -- there might be a better way to get the PDF data over
without redownloading. (It uses passed cookies, though, so gated PDFs
should still work.)
I guess the idea that switching to PNGs obviated the need for this was wishful
thinking (though it doesn't seem to be necessary for the single buttons anymore
on Linux, and it's no longer necessary on OS X, which has generally saner
styling in Firefox).
- Fix spacing on Windows and Linux in latest Firefox versions
- Tweak icon colors on Windows and OS X
- Adjust Z SVG to take up full height, so Z is a full 16px instead of
14px with slight anti-aliasing
- Use generated PNGs instead of SVG for Z toolbar icons, to remove the
need for complicated size rules
- Add separate platform-specific .svg files that are used by a
zotero-build script, make-z-icons, to generate the Z PNGs; the main
SVG is still used directly in the menu panel and customization
palette, with platform media queries to determine the coloring
Unfortunately this will need to be partly redone, since retrieveItem(), and
therefore itemToCSLJSON(), and therefore itemToExportFormat(), need to be
synchronous. The item data load statements in itemToExportFormat() will
probably need to be performed earlier, when they can be async, and made
available to the session for retrieval by retrieveItem(), but I'll let someone
more familiar with the citation infrastructure do that.
This restores some code in retrieveItem() that may have been accidentally
removed in a merge, though it probably won't be useful anymore anyway.
Addresses #529
Now that 500 errors are retried in file downloads (ec28c5a3), we have to
override the default backoff schedule in order to get expected failures.
This also fixes an error that occurred on a retried download.
In Fx44, SQL queries must use '?' with LIKE and cannot concatenate a
placeholder string (e.g., 'foo%'). This is for Sqlite.jsm only, so it
doesn't affect 4.0.
For now, delete it and clear it from the cache so it's updated properly going
forward, but really we want to reinstall the correct file automatically
(#903).
This apparently has been happening for a year, since #617.
Might be better to show open/save on cancel, as implemented for the
RIS/BibTeX prompt in 4ecdd55 (which caused this), but that would have to
wait for the better promise handling in 5.0. Doing "Save Link As..." for
a CSL file also seems a bit more straightforward than for RIS/BibTeX,
which can be served in complicated ways.
Instead, pass ids directly to SQLite. This seems to take about the same
amount of time or a little less (by avoiding the time it takes to start
a transaction) and avoids blocking other transactions when switching
views.
Most checks are now just foreign key checks and can be checked with
"PRAGMA foreign_key_checK".
The DB Repair Tool will need to be updated to handle the new schema (but
still accept the old one).
Fixes the incompatible-version dialog and adds info on the version used
to upgrade the database, which may be helpful for troubleshooting.
Also fixes showing of the Zotero toolbar icons even in case of a startup
error.
Also:
- Remove last-sync-time mechanism for both WebDAV and ZFS, since it can
be determined by storage properties (mtime/md5) in data sync
- Add option to include synced storage properties in item toJSON()
instead of local file properties
- Set "Fake-Server-Match" header in setHTTPResponse() test support
function, which can be used for request count assertions -- see
resetRequestCount() and assertRequestCount() in webdavTest.js
- Allow string (e.g., 'to_download') instead of constant in
Zotero.Sync.Data.Local.setSyncState()
- Misc storage tweaks
Improves UX of sync authentication.
The account is now linked and unlinked and an API key related to
the client is generated transparently in the background.
The API key is deleted on unlinking.
No sync options are allowed before linking an account.
Closes#864
This adds a 'channel' property to Zotero.HTTP.UnexpectedStatusException,
because the 'channel' property of the XHR can be garbage-collected
before handling, and the channel's 'securityInfo' property is necessary
to detect certificate errors.
This should be refactored further, but this is a start at swapping in
Bluebird. Unfortunately the process doesn't work correctly because of
issues in the translation framework, which will need to be addressed
separately -- right now a parent item is successfully created but the UI
still shows an error and the PDF isn't placed under the parent.
Closes#852
Currently shows a *+* icon for adding a tag and
a *link* icon for removing it.
There does not appear to be any way to have a custom mouse cursor,
which makes this interaction extremely unintuitive. It might be
possible to hack this in, but it does not really warrant the effort.
E.g. `window.setCursor` does not respond whilst dragging.
This changes Zotero.Translate.Base.translate() to take an options object (in
order to take a 'collections' parameter, which is passed to the
Zotero.Translate.ItemSaver constructor). The old parameters are still supported
with a deprecation warning, and there may be other places that still need to be
updated.
Display a warning when choosing data directory
Or on opening Zotero Pane for existing users with data dir in dropbox
Also:
Fix a bug where it won't use custom path if "Choose..." button is
pressed instead of radio button.
Change filepicker to show current data directory on display
Disallow everything other than adding a child note in My Publications,
since the wizard only shows on drag (currently, at least). Need to
disable translation separately.
Also show "Delete Item[s]..." in the trash context menu, and hide rather
than disable various menu options.
Addresses #703 -- still need to disable saved search saving
This mostly gets ZFS file syncing and file conflict resolution working
with the API sync process. WebDAV will need to be updated separately.
Known issues:
- File sync progress is temporarily gone
- File uploads can result in an unnecessary 412 loop on the next data
sync
- This causes Firefox to crash on one of my computers during tests,
which would be easier to debug if it produced a crash log.
Also:
- Adds httpd.js for use in tests when FakeXMLHttpRequest can't be used
(e.g., saveURI()).
- Adds some additional test data files for attachment tests
When called on an identified object (i.e., one with an id or
library/key), loadAllData() must be called first. When called on a new
object (which is more common anyway), fromJSON() can be called
immediately.
If Standalone is closed in full-screen mode in 10.11, 'sizemode' on the main
window is persisted as 'fullscreen'. On reopen, the window doesn't go into
full-screen mode, but it still lacks menubar or titlebar. This patch forces the
window into 'normal' mode if it was left in 'fullscreen'.
(Firefox appears to ignore 'fullscreen' without actually changing the value,
but I didn't find the code that handles that.)
Absolute paths have been stored as strings on all platforms for a while,
but old Mac persistent descriptors (Base64-encoded opaque alias records)
could still exist in the DB. Additionally, relative paths for stored
files were stored as Mozilla-specific opaque strings rather than UTF-8
strings.
This adds a schema step to convert those to strings paths in the DB.
Since Mac persistent descriptors aren't converted if the file isn't
found, we still handle and (convert) old-style persistent descriptors if
necessary when reading paths from the DB.
This also moves path string handling -- converting a path to a prefixed
string for stored or base-dir-relative files -- to the
Zotero.Item#attachmentPath setter instead of save() so that reading it
back immediately returns the correct value. One consequence is that the
attachment link mode must now be set before setting the path.
Zotero.Item#getFile() is now deprecated in favor of getFilePath() and
getFilePathAsync() (which checks file existence).
Zotero.File.directoryContains() now takes string paths instead of files.
