Commit graph

6306 commits

Author SHA1 Message Date
Dan Stillman
4c46ebcf96 loadAllData() no longer necessary before fromJSON()
Post-deasyncification
2016-05-21 16:33:46 -04:00
Dan Stillman
f8716fbe88 Don't change Date Modified when updating local item after upload
If the API returns a modified item after an upload (e.g., to strip invalid
characters), don't update the Date Modified field when saving those changes to
the local version (though it would still be good to avoid API-side changes as
much as possible).
2016-05-21 16:33:46 -04:00
Dan Stillman
8b4a5936a3 Fix checking for file sync downloads at sync time 2016-05-21 16:33:45 -04:00
Dan Stillman
99eb39e288 Always include 'contentType'/'charset'/'filename' in attachment JSON
And omit in ZFS file sync requests

The API previously didn't allow these properties to be set for group items,
because they were set atomically during the file upload process, but 1) that's
not really necessary (makes a little sense for 'filename', but not really a big
deal if an old file is renamed on another computer before the new file is
synced down) and 2) skipping them results in the properties getting erased
after items are uploaded and the empty values returned by the server overwrite
the local values.
2016-05-21 16:33:35 -04:00
Dan Stillman
f5812852cb Default to My Library if pane not yet open saving snapshot from connector 2016-05-20 16:22:28 -04:00
Dan Stillman
0be2796500 Fix webpage/snapshot saving from connector 2016-05-20 15:51:54 -04:00
Dan Stillman
157b8deda9 Allow processor passed to Zotero.HTTP.processDocuments to return a promise
If a promise is returned, it's waited for before continuing with the next
document.
2016-05-20 15:51:01 -04:00
Dan Stillman
d8abfa4f67 Save all item data values as string, and convert old integers to strings
Before 5.0 we performed a regexp on new item data values to determine if
they were integers and saved them natively in SQLite if so. We no longer
do that, but setField() used strict equality when checking for changes,
so an item could be marked as changed when comparing to a new string
value (e.g., from a write response from the API, which always returns
strings). To avoid that, this converts all old values in the DB to
strings and saves all incoming values as strings automatically. (This
should also help with searching and some other things.)
2016-05-18 17:25:50 -04:00
Dan Stillman
b9bbf007f0 Don't bother opening browser window for syncLocal::checkUser() tests
The window is passed to nsIPromptService, which handles finding a parent window
(hidden or otherwise) automatically if necessary.
2016-05-18 02:31:41 -04:00
Dan Stillman
2c18968c3a Fix Quick Format dialog for search results in unloaded libraries
Since the search might return items in libraries that haven't yet been loaded,
we have to get those items asynchronously and load their data explicitly. Need
to test this with a large document to make sure it still properly populates the
list with search results and then adds matching cited items.
2016-05-17 13:43:11 -04:00
Dan Stillman
7192929ac3 Add Zotero.DataObjects.prototype.loadDataTypes(objects, dataTypes)
Bulk-loads data for objects that are potentially in different libraries. This
would generally be used to load necessary data for cross-library search
results, since those results might include objects in libraries that haven't
yet been loaded.
2016-05-17 13:43:11 -04:00
Dan Stillman
dd9cc40c16 Cancel pending feed update check on Zotero shutdown
This avoids errors for a missing 'feeds' table during tests, when the callback
started in Zotero.Feeds.init() runs but the Zotero object has been torn down
and the database hasn't yet been reinitialized.
2016-05-17 13:43:11 -04:00
Dan Stillman
6907af86fd Add XPCOM clearTimeout() and remove Zotero.wait()
Properly emulate setTimeout() with support for cancelling timers.
2016-05-17 13:43:11 -04:00
Dan Stillman
a285e97815 Remove pumpGenerator/promiseGenerator
And update integration code to use ES6 generators and Bluebird's .each()
instead.
2016-05-17 13:43:11 -04:00
Dan Stillman
d8025be676 Ignore sanitization changes when comparing notes in conflict
Until we have a consistent way of sanitizing HTML on client and server, account
for differences manually. More differences between HTMLPurifier and TinyMCE
should be added as necessary.
2016-05-17 02:33:53 -04:00
Dan Stillman
6f04b34d59 Close #945, Optimize bulk reload on multi-item changes 2016-05-16 18:25:35 -04:00
Dan Stillman
56dcd6da26 Add file sync info/link to File Not Found dialog 2016-05-15 04:24:00 -04:00
Dan Stillman
427234917d Properly test whether file syncing is enabled for a library 2016-05-15 04:22:24 -04:00
Dan Stillman
a8ea8656d2 Fix sync error after changing child item to top-level 2016-05-15 03:34:06 -04:00
Dan Stillman
a7ea92fd52 Mark items with missing full-text cache files as unsynced for full text
Fixes #954, Full-text content processor tries to process missing cache
files on every idle
2016-05-15 01:30:35 -04:00
Dan Stillman
82d00a4e7a Change some instances of attachmentMIMEType to attachmentContentType 2016-05-13 15:56:35 -04:00
Dan Stillman
eb400587e8 Fix various bugs saving from connector and add test 2016-05-13 15:00:54 -04:00
Dan Stillman
55c60a69ac Delay feed fetching 5 seconds at startup
Would be better not to start until after the initial library's items had
been loaded, but that's a little awkward to do from non-window code.
Might make more sense when there's only Standalone.
2016-05-13 02:26:46 -04:00
Dan Stillman
ed312b2c89 Tweak text in empty-library sync warning dialog 2016-05-12 17:00:28 -04:00
Dan Stillman
59c0c7fcad Don't include file properties for ZFS-synced libraries
They're disallowed by the API in group libraries and not necessary in personal
libraries, because they're set atomically with the file upload.
2016-05-12 17:00:28 -04:00
Dan Stillman
0f45569ccd Default to NC for My Publications CC license, and reorder SA option 2016-05-12 14:31:57 -04:00
Dan Stillman
62ef910631 Better debug output for full sync 2016-05-12 03:02:50 -04:00
Dan Stillman
92ec1324dc Add a debug flag to prompt or throw on an attempted sync upload
extensions.zotero.sync.debugUploadPolicy = {1, 2}

1 to prompt, 2 to throw

This can be used to test sync functionality without altering the server
state unexpectedly. When prompting, the request body, if any, is logged
to debug output.
2016-05-12 03:02:50 -04:00
Dan Stillman
98cd96eb01 Make DELETE a write method in Zotero.HTTP.isWriteMethod() 2016-05-12 03:02:26 -04:00
Dan Stillman
8bbde63284 Update menu for non-collections when right-clicking on unselected row
Previously, item-dependent menu options in the context menu of
collectionsTreeRow types other than 'collection' weren't updated after
the asynchronous loading of the items. The menu is now updated for all
types, so, e.g., "Empty Trash" is now updated properly after items in
the trash have been loaded.
2016-05-12 02:12:22 -04:00
Dan Stillman
16a58c03be Save scroll position across non-selected changes for collections view 2016-05-09 13:58:33 -04:00
Dan Stillman
9345452601 Don't restore items tree scroll position if selecting new items 2016-05-09 13:58:23 -04:00
Dan Stillman
0bab038925 Fix #984, 5.0: Avoid scrolling items list when adding items via sync 2016-05-09 13:26:33 -04:00
Dan Stillman
53e1e1a9b7 .collectionTreeViewID -> .treeViewID, and add it to Zotero.Item too
Also .collectionTreeViewImage -> .treeViewImage
2016-05-09 12:34:03 -04:00
Dan Stillman
cb8b2bda1b Windows file path fixes
- Fix upgrading of Mozilla-style attachments/storage file paths on upgrade
  (requires re-upgrade)
- Save relative paths using forward slashes for consistency, and convert
  to platform-appropriate slashes on use
2016-05-09 02:30:00 -04:00
Dan Stillman
317b1dfa0f Disable connector switching for 5.0 beta
Don't start the connector pipe (Mac/Linux), don't listen for IPC on the
command-line (Windows), and don't try to release Standalone's lock if DB is
busy when Firefox starts.
2016-05-08 22:38:58 -04:00
Dan Stillman
216900094b Merge branch '4.0' 2016-05-07 13:27:56 -04:00
Dan Stillman
e5edd12371 RTF Scan fixes
- Firefox 46 compatibility
- Stretched progress meters

Partial backport of c5716a395f
2016-05-07 13:18:22 -04:00
Dan Stillman
c7c271f2c4 Fix CSL editor in Firefox 46 2016-05-07 13:17:57 -04:00
Dan Stillman
e1706e15e2 Expand/collapse library fixes
- Fixes #994, 5.0: "+" doesn't expand all collections within a library
- If a container (library, collection) is closed directly, the open state of
  all containers below it are now restored when it's reopened. Previously all
  collections would be closed on a manual reopen (though they might have been
  restored on the next Zotero restart).
- If "-" is pressed, all containers are closed, and reopening the library will
  show only top-level collections.
2016-05-07 04:02:42 -04:00
Dan Stillman
458d110269 Add file for custom sync exceptions 2016-05-06 05:18:09 -04:00
Dan Stillman
950b583e3b Remove obsolete code 2016-05-06 05:00:36 -04:00
Dan Stillman
52ca6d4713 Handle browser-offline error from anywhere in sync code
And don't throw from ConcurrentCaller, because it results in an
unhandled rejection
2016-05-06 05:00:17 -04:00
Dan Stillman
2894e4f462 Fix search by collection 2016-05-06 04:31:49 -04:00
Dan Stillman
8ba9fe7b80 Remove redundant failure delay handling in sync engine
The API client now automatically retries 5xx errors with a backoff schedule, so
the engine doesn't need to do it.
2016-05-06 04:31:49 -04:00
Dan Stillman
a78f923a72 Sync engine cleanup
- Use custom exception for user-initiated sync cancellations, which can bubble
  up to the sync runner -- this should help with a sync stop button (#915)
- Separate out deletions-downloading code
- Refactor delay generator handling on library version mismatch
- Clearer variable names
2016-05-06 04:31:49 -04:00
Dan Stillman
ab4138cf26 Don't throw in Zotero.DataObjects::get() for nonexistent objects
Return false for single ids or skip for multiple ids. This is the original
behavior, but at some point it started throwing an UnloadedDataException. IDs
are always loaded at initialization, though, so we know whether the objects
actually exist.
2016-05-05 06:36:24 -04:00
Dan Stillman
bf3ad6a2a4 Fix test breakage from 3ab335a0 and 09c3a95a 2016-05-05 06:36:24 -04:00
Dan Stillman
b7ad582d99 Merge pull request #991 from adomasven/fix/feed-import-test
Fixes feed-import (and possibly other feed) test unhandled rejections
2016-05-05 06:08:33 -04:00
Adomas Venčkauskas
d4a799b5cb Closes #981. Fixes feed-import (and possibly other feed) test unhandled rejections 2016-05-05 11:07:27 +01:00
Dan Stillman
09c3a95a7e Improve downloaded object processing
- Use an increasing notifier batch size, so objects initially appear one by one
  but then start showing up in batches, up to 50. (UI updates are expensive, so
  for larger syncs we don't want to update after each object.)
- Avoid separate save to update attachment file sync state, which was also
  happening outside of notifier batches (causing individual updates regardless
  of the batch size)
- Add a 10ms delay after processing each object, which keeps the UI responsive
  during downloads. #989 could reduce this to 1 during idle, to save a few
  minutes when downloading very large libraries.
2016-05-05 04:47:01 -04:00
Dan Stillman
4ac2da42d1 Zotero.Items.getAsync() -> get() in itemTreeView::notify() 2016-05-05 02:16:09 -04:00
Dan Stillman
3ab335a078 Don't clear search field when updating items list during sync
Fixes #985
2016-05-05 02:16:08 -04:00
Dan Stillman
8c98abe9dc Fix #988, 5.0: Tab skips Abstract and Extra fields 2016-05-04 21:10:08 -04:00
Dan Stillman
cd5e805b9e Sync logic improvements
- Cancel sync when cancelling conflict resolution window
- Don't try to upload unsynced objects if present in sync queue
2016-05-04 01:39:24 -04:00
Dan Stillman
391f525a75 Close #975, Process conflicts for all batches together 2016-05-03 23:18:42 -04:00
Dan Stillman
f8a0f9ad1d Use synchronous object getters during sync
Since objects should always already be loaded
2016-05-03 18:27:08 -04:00
Dan Stillman
f751495ce8 Remove debug output hover over sync icon 2016-05-03 18:27:08 -04:00
Philipp Zumstein
aef8453924 Fix some for each ... in cases (#978) 2016-05-03 17:51:12 -04:00
Dan Stillman
096ebb188b Try to log a better error for some instances of Zotero.debug(e); 2016-05-03 13:47:37 -04:00
Dan Stillman
72c8711cd3 String::contains() -> indexOf() for Firefox 48 (and 38) compatibility
.contains() was removed in Firefox 48, but .includes() wasn't available until
40, so use indexOf() for now. We can start using .contains() once we no longer
need to support 38 ESR.
2016-05-03 12:08:38 -04:00
Dan Stillman
315100e5ef Remove obsolete style code 2016-05-03 12:06:29 -04:00
Dan Stillman
e8aec31715 Fix various cases of sync errors with read-only libraries
Addresses #983
2016-05-03 02:55:26 -04:00
Dan Stillman
f844d9e46d Skip full-text content download if main library version hasn't changed
Since a data sync always happens first, the main library version will
always be higher if there's any full-text content to download.
2016-05-03 01:17:53 -04:00
Dan Stillman
e0e744f9b1 Use multi-item requests for full-text writes
This is necessary to get a library version after the write instead of an
item version. Otherwise after a full-text write, the main library
version is behind, so the next sync checks all object types for that
library instead of getting a 304.

Full text is batched up to 500K characters or 10 items, whichever is
less.

This also switches to using ?format=versions for /fulltext requests,
which isn't currently necessary but reflects what it's actually doing.
2016-05-03 01:17:52 -04:00
Dan Stillman
a0c7cf9bee Remove redundant error handling in Z.Sync.Data.Engine::_uploadObjects()
makeRequest() now retries 5xx errors automatically, so it's not
necessary higher up.
2016-05-02 05:53:07 -04:00
Dan Stillman
4cc6408105 Use object destructuring in Zotero.Sync.Data.Engine::_uploadObjects() 2016-05-02 05:52:42 -04:00
Dan Stillman
862b371b49 Refactor some API response handling code 2016-05-02 01:28:37 -04:00
Dan Stillman
14ce455483 Fix #982, 5.0: File dots not showing up for child attachments 2016-04-30 19:18:59 -04:00
Dan Stillman
6afce0cf22 Fix "treeRow is undefined" error dragging to bottom of collections list
If you drag past the collections list the row is -1.

Fixes #958
2016-04-30 17:15:13 -04:00
Dan Stillman
10181f7f56 Fix sync error on missing full-text
If a version is returned for an item's full-text content but a 404 is returned
for the content itself (because it's missing in Elasticsearch for some reason),
don't throw an error.

Also remove legacy array comprehensions in fulltext and syncFullTextEngine test
files, which apparently weren't being run.
2016-04-30 01:24:33 -04:00
Dan Stillman
750d691712 Throw errors instead of strings in FeedReader rejections 2016-04-29 23:32:49 -04:00
Dan Stillman
13c4bbebfa Save master item when merging 2016-04-29 20:29:44 -04:00
Dan Stillman
877063e73d Merge branch '4.0' 2016-04-29 20:29:31 -04:00
Dan Stillman
b4ac43fd77 Fix startup error on Firefox 48
Addresses #980. There are a couple other uses of the old API, but I'm not sure
how they're triggered, and we might be gone from Firefox before this is
relevant.
2016-04-29 17:28:24 -04:00
Dan Stillman
ef71d3f729 Revert "Try disabling gzip compression of uploads, in case it fixes sync error"
This reverts commit adec007df0.

Doesn't seem to make a difference.
2016-04-29 11:57:11 -04:00
Dan Stillman
adec007df0 Try disabling gzip compression of uploads, in case it fixes sync error
Just to see if it fixes the "Error connecting to server" errors some people are
seeing after upgrading to Firefox 46.
2016-04-29 00:38:19 -04:00
Adomas Venčkauskas
477a1fb5bc Fixes sync error when transfering credentials from legacy to API key 2016-04-28 15:18:09 +01:00
Dan Stillman
186561f320 Merge pull request #950 from adomasven/feature/import-feeds-from-opml
Close adomasven/zotero#11. Add support for feed imports from OPML files
2016-04-28 01:09:47 -04:00
Adomas Venčkauskas
3b758e562b Close adomasven/zotero#11. Add support for feed imports from OPML files 2016-04-27 20:32:20 +01:00
Dan Stillman
bb522868ad Merge pull request #977 from adomasven/fix/feed-remove-after-setting
Closes #961. Fixes feed dialogue remove read articles value not saving
2016-04-27 13:29:36 -04:00
Adomas Venčkauskas
98a3c4aeaa Closes #961. Fixes feed dialogue remove read articles value not saving 2016-04-27 16:38:28 +01:00
Adomas Venčkauskas
1892f7b5b0 Closes #951. Bypasses cache when retrieving feeds 2016-04-27 15:27:09 +01:00
Dan Stillman
295e9f3ecf Don't retry cancelled conflicts immediately
If other items were saved in the same batch, the conflict resolution
window could reappear immediately after cancelling it.
2016-04-27 05:45:35 -04:00
Dan Stillman
c146adce38 Hide sync error panel after custom button is clicked 2016-04-27 03:15:29 -04:00
Dan Stillman
e5bf7eff21 Open account settings in new tab from quota message 2016-04-27 03:15:10 -04:00
Dan Stillman
6d6afdd706 Show correct quota message for personal library 2016-04-27 03:14:51 -04:00
Dan Stillman
0ebf49849a Log counts of storage requests that succeeded and failed 2016-04-27 02:35:24 -04:00
Dan Stillman
1c90a77298 Fix handling of 413 for over-quota errors
And fix handling of custom error dialog button text/callbacks in
general.
2016-04-26 18:59:23 -04:00
Dan Stillman
098655d913 Prompt about username change at sync time, not just in prefs
This is necessary because you can copy a database synced with a
different account into the data directory without affecting the stored
pref.

Also tweak the text to use proper quotes and remove quaint references to
"the server".
2016-04-26 18:12:20 -04:00
Dan Stillman
758af2be9e Align "Set Up Syncing" button in prefs 2016-04-26 16:27:26 -04:00
Dan Stillman
a62e7e46bf Remove some unreachable code
Addresses #974
2016-04-26 02:00:51 -04:00
Dan Stillman
95ba68a757 Don't skip uploads of more than 25 items…
Fixes #965
2016-04-26 01:35:01 -04:00
Dan Stillman
55688a8104 Load item data in feed libraries before refreshing feeds 2016-04-26 00:00:12 -04:00
Dan Stillman
a8e94a8cba Extra debug logging for #965 2016-04-25 23:41:49 -04:00
Dan Stillman
f82fb89e1c Sort parent collections and items first in uploads
Closes #972
2016-04-25 20:16:31 -04:00
Dan Stillman
12e369b2b6 Show the invalid login prompt for an invalid API key
Addresses #953, but the message should still be updated to be more
precise now that passwords aren't used directly.
2016-04-25 03:12:11 -04:00
Dan Stillman
4eee369b66 Use /keys/current for key requests
Possible after https://github.com/zotero/dataserver/commit/beb38579ca1
2016-04-25 02:51:35 -04:00
Dan Stillman
ebb2f1667d Log feed errors to console 2016-04-25 00:50:28 -04:00
Dan Stillman
a05134e903 Fix search by file type
Fixes #966
2016-04-25 00:50:27 -04:00