Commit graph

1029 commits

Author SHA1 Message Date
Dan Stillman
cbfa4be437 Add 'Type:' to extraToCSL() test
Follow-up to #1744
2019-10-26 17:05:45 -04:00
Dan Stillman
c25fbe7c1c Remove all colored tags on selected items if 0 is pressed
Like Thunderbird
2019-10-26 16:45:04 -04:00
Dan Stillman
52fd91950d Fix deleted item fields reappearing on sync
This was a regression from 4b60c6ca27. The original plan for introducing
new fields was to have them save to the sync cache even if they weren't
supported by the current Zotero version and process them on upgrade, and
so I changed `DataObjectUtilities.patch()` to omit fields that didn't
exist locally when patching the sync cache JSON so they wouldn't be
wiped on the server. That caused this bug where locally deleted fields
were restored on every sync. It's also no longer necessary now that
we decided to just reject unknown fields from saving, so we can just
revert to the previous behavior of blanking out locally missing fields
(with the tweak that fields that are already false or empty in the base
version can be omitted).
2019-10-24 01:22:00 -04:00
Dan Stillman
513f7d6555 Fx60: Fix localization
Also fixes #1690 (at least on Catalina), and possibly other things

general.useragent.locale and intl.locale.matchOS are no longer used.
2019-10-23 19:08:13 -04:00
Dan Stillman
d4f682aa88 Merge branch 'fx60' 2019-10-21 21:44:01 -04:00
Dan Stillman
956813ac1f Fix "err is undefined" on 200 response for nonexistent WebDAV file
Fixes #1741
2019-10-20 15:03:18 -04:00
Dan Stillman
1710eb1c4b Don't store unknown/invalid fields in Extra in non-strict mode
And fix a couple things for if we turn it back on

This code came along with the type/field handling overhaul, but I think
it was originally intended for handling unknown fields during sync
before we decided on strict mode, so it wasn't finished and causes
various problems [1]. It could still be useful for preserving fields
from translators before they're available on items, but the better fix
there is just to add the missing fields, so I'm not sure if we'll end up
needing it.

[1] https://groups.google.com/d/msg/zotero-dev/a1IPUJ2m_3s/hfmdK2P3BwAJ
2019-10-18 03:37:24 -04:00
Dan Stillman
89672ed0a4 Fix "c1 is undefined" sync CR error 2019-10-09 18:29:04 -04:00
Dan Stillman
44d4586242 Fix test after 8aeb6f7fe4 2019-09-28 07:40:01 -04:00
Dan Stillman
5723683b3b Fix "attachmentSyncedModificationTime must be a number" sync error
https://forums.zotero.org/discussion/79011/zotero-error-report
2019-09-21 01:59:09 -04:00
Dan Stillman
4b7cdddb4a Change apply-to-all checkbox text for some conflicts
Say "Use the [local|remote] version for all remaining conflicts" for
everything instead of saying "Use [local|remote] fields for all
remaining conflicts" for some conflicts.

This also fixes a test failure after 54343c49fb.
2019-09-21 01:35:36 -04:00
Dan Stillman
4b60c6ca27 Type/field handling overhaul
This changes the way item types, item fields, creator types, and CSL
mappings are defined and handled, in preparation for updated types and
fields.

Instead of being predefined in SQL files or code, type/field info is
read from a bundled JSON file shared with other parts of the Zotero
ecosystem [1], referred to as the "global schema". Updates to the
bundled schema file are automatically applied to the database at first
run, allowing changes to be made consistently across apps.

When syncing, invalid JSON properties are now rejected instead of being
ignored and processed later, which will allow for schema changes to be
made without causing problems in existing clients. We considered many
alternative approaches, but this approach is by far the simplest,
safest, and most transparent to the user.

For now, there are no actual changes to types and fields, since we'll
first need to do a sync cut-off for earlier versions that don't reject
invalid properties.

For third-party code, the main change is that type and field IDs should
no longer be hard-coded, since they may not be consistent in new
installs. For example, code should use `Zotero.ItemTypes.getID('note')`
instead of hard-coding `1`.

[1] https://github.com/zotero/zotero-schema
2019-09-16 02:27:22 -04:00
Dan Stillman
20d7ef6c0b Add separate .eslintrc for tests 2019-09-16 01:28:52 -04:00
Dan Stillman
e8dd1f7824 Test updates for newer Sinon 2019-09-16 01:27:30 -04:00
Dan Stillman
25e34b7b99 Update WebDAV request timeouts for HTTP.request() auto 5xx retry 2019-09-16 01:27:18 -04:00
Dan Stillman
dc60e5f840 HTTP.request() improvements
- Move 5xx retries and connection checking out of the sync API client
  and into HTTP.request() so that they apply to all requests. 429 handling
  remains in the API client, since not all callers necessarily want to
  handle that the same way. Callers can still handle 5xx themselves by
  including the relevant 5xx status codes in `successCodes` or by passing
  `errorDelayMax: 0`.
- Add `cancellerReceiver` option, which is a callback that receives a
  function that will cancel the request, whether it's an active request
  or an automatic delay before a 5xx retry.

This also updates Sinon to 7.3.2.
2019-09-16 01:26:34 -04:00
Dan Stillman
25f29d92f2 Fix error reporting within fake XHR server handlers in tests
And fix removal of Bluebird lines from stack traces
2019-09-16 00:52:53 -04:00
Dan Stillman
f04f186396 Add missing yield in sync code 2019-09-16 00:47:45 -04:00
Dan Stillman
7ed0d8a408 Fix recognizePDF test after DOI translator change 2019-09-16 00:46:49 -04:00
Dan Stillman
54343c49fb Fix "getNote() can only be called on notes and attachments" CR error
This bug may be as old as the sync system itself. It could occur if
there were conflicts for both a note and a regular item in the same
batch.
2019-09-16 00:39:06 -04:00
Dan Stillman
b08bd6849e Fx60: Update DB query onRow() behavior
onRow() handlers now get passed a cancellation function as a second
argument
2019-08-27 06:00:35 -04:00
Dan Stillman
564e72196f Update httpd.js to Fx60 version 2019-08-27 05:30:11 -04:00
Dan Stillman
9a49718638 Remove some old Zotero for Firefox code
In particular, remove code related to opening/closing the Zotero pane,
which affects tests. The pane is now opened by default in Firefox, which
brings its behavior closer to the main version.
2019-08-27 00:47:39 -04:00
Dan Stillman
433794916a Fx60: Remove instances of nsILocalFile
nsILocalFile was merged into nsIFile, so either replace it with that or
switch to Zotero.File.pathToFile().
2019-08-27 00:47:39 -04:00
Dan Stillman
6f965251ed Add FilePicker module to replace nsIFilePicker
`nsIFilePicker::show()` is removed in Firefox 60 in favor of `open()`,
which takes a callback (and apparently has been preferred for a long
time).

There's no point switching to that, so this module is a version of
nsIFilePicker with an async `show()` that returns a promise and some
XPCOM-isms replaced (e.g., string paths instead of nsIFile).
2019-08-27 00:47:39 -04:00
Dan Stillman
7504086a5e Use modified Firefox executable from zotero-standalone-build in tests 2019-08-27 00:47:39 -04:00
Dan Stillman
d89fe8e80a Fx60: Add extensions.legacy.enabled for tests 2019-08-27 00:47:39 -04:00
Dan Stillman
75daaa3b7c Fix incorrect 'return' in test 2019-08-27 00:47:39 -04:00
Dan Stillman
cde5db463a Fx60: Restore default prefs in tests
Default prefs from legacy extensions don't seem to be used anymore
2019-08-27 00:47:39 -04:00
Dan Stillman
ac456ab9bf Fx60: Additional prefs changes for tests 2019-08-27 00:47:39 -04:00
Dan Stillman
8c59df435f Fx60: Fix snapshot filenames
nsIURL doesn't seem to work anymore, so add Zotero.Utilities.parseURL(),
which uses the `url` package from NPM and adds fileName, fileExtension,
and fileBaseName.
2019-08-27 00:47:39 -04:00
Dan Stillman
f7854bfcae Fx60: Remove versioned JavaScript 2019-08-27 00:47:39 -04:00
Dan Stillman
bb59429664 Add "Convert Linked Files to Stored Files…" menu option
In new File → Manage Attachments submenu

Closes #1637
2019-08-19 05:00:32 -04:00
Dan Stillman
c2e79c0717 Close window after protocol handler tests
Regression from fda002ec34
2019-08-03 02:48:01 -04:00
Dan Stillman
fda002ec34 Beginning of tests for zotero://select 2019-08-02 02:29:08 -04:00
Dan Stillman
9becefc3fd Fix test for AMA subtitle capitalization 2019-08-01 17:32:56 -04:00
Dan Stillman
9b82373f70 Add ability for Scaffold to provide alternative translators
Zotero.Translate::setTranslatorProviderMethods(methods) can be used to
provide custom 'get' and 'getAllForType' methods that override the
default Zotero.Translators methods.
2019-07-24 04:31:11 -04:00
Dan Stillman
2603373b86 Additional protections for HTTP endpoints
Reject browser-based requests that don't require a CORS preflight
request [1] if they don't come from the connector or include
Zotero-Allowed-Request: 1

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#Simple_requests
2019-07-15 07:30:12 -04:00
Dan Stillman
708b8798c2 Check for X-Zotero-Connector-API-Version header in /connector/import 2019-07-12 02:08:55 -04:00
Adomas Venčkauskas
c4f40c38e9 Unsuppress accidental test suppression from fe01e17 2019-07-04 15:46:24 +03:00
Adomas Venčkauskas
fe01e17e93 Fix integration test breakage (regression cd14a53) 2019-07-04 15:38:46 +03:00
Adomas Venčkauskas
cd14a536dc Add retracted citation warning tests. Fix a bug with embedded prompts 2019-07-04 15:16:26 +03:00
Dan Stillman
82e7f67df8 Fix retraction test 2019-07-04 07:58:56 -04:00
Dan Stillman
0beddb9680 Add flags to disable retraction warnings
Separate flags for hiding the retraction altogether and for hiding
citation warnings for it

New functions:

Zotero.Retractions.hideRetraction(item)
Zotero.Retractions.shouldShowCitationWarning(item)
Zotero.Retractions.disableCitationWarningsForItem(item)

Addresses #1710
2019-07-03 01:23:02 -04:00
Dan Stillman
f49d5805cd Apply subtitle capitalization to additional styles
This also fixes a style initialization error when the style id is
invalid, which was a regression in a0be2da42c.
2019-07-01 22:39:12 -04:00
Dan Stillman
ead93b6ccc Stop uploading files on quota error until next manual sync or restart 2019-06-22 05:29:47 -04:00
Dan Stillman
50a4308514 Remove retraction flag from items that no longer match 2019-06-19 06:41:33 -04:00
Dan Stillman
b4b19c4811 Add Zotero.Retractions.getRetractionsFromJSON()
Addresses #1703
2019-06-12 01:05:49 -04:00
Dan Stillman
d353439980 Add retractions.enabled hidden pref to disable retraction checking 2019-06-11 21:24:17 -04:00
Dan Stillman
5c03813d81 Add Retracted Items virtual collection
Shown automatically when retracted items are detected
2019-06-10 02:38:42 -04:00
Dan Stillman
502f5fe491 Run notifier observers in tests after all others
This ensures that main functionality will have already run if a test
waits on the same event.
2019-06-10 02:20:11 -04:00
Dan Stillman
79a8917bdc Fix test failure after 3727d0f559 2019-06-07 03:17:26 -04:00
Dan Stillman
7f4f2770ba Add notification banner when retracted items are found
And other retraction tweaks
2019-06-07 01:15:17 -04:00
Dan Stillman
985f10d5ed Fix WebDAV test after 1f1b2e9b2 2019-06-03 03:35:32 -04:00
Dan Stillman
c0e2aa03d3 Fix test for delay after second library version conflict 2019-05-31 02:44:41 -04:00
Dan Stillman
e33a2d730d Fix tag type handling when merging items
Most importantly, don't change all tags to manual on the merged item.
2019-05-29 05:43:33 -04:00
Adomas Ven
48778f2847 Document export-import UI and integration code (#1501) 2019-05-14 21:06:18 -04:00
Dan Stillman
3fbb17a2e6 Properly upload local changes after automatic conflict resolution
If an object changed on both sides and the changes were either
non-conflicting or identical but there were other local changes, the
local object was incorrectly being marked as synced, causing it not to
be uploaded until it was next modified locally.
2019-05-08 04:34:21 -04:00
Dan Stillman
ec0b993a17 Fix erroneous sync conflict if creators are changed in the same way 2019-05-03 01:15:27 -04:00
Adomas Venčkauskas
78c3d5808b Fix citeproc errors due to improper citeproc state updates
The error is triggered upon initial interaction with a doc after Zotero
restart or if new external citations (copied into the document) are
peresnt and `session.updateSession()` is called without a subsequent
`session.updateDocument()` call. `session.updateSession()` is called
without a subsequent `session.updateDocument()` call every time the
user cancels a citation insert.

More specifically, `session.updateSession()` is called every time a
citation dialog is invoked. It retrieves all citations and writes them
into a local `session.citationsByIndex` object. Moreover, it marks
each citation that hasn't seen before in a `session.newIndices` object.
`session.newIndices` is there to ensure that we load every new citation
into citeproc upon document update. This object is built by marking any
citation that does not appear in the previous invocation's list of
citations as new. However, if the document is never updated (because the
user cancels the insertion) then the new indices are not loaded
into citeproc. This commit fixes that, by excluding citeproc unloaded
items from the previous invocation's citation list.
2019-04-08 17:43:52 +03:00
Frank Bennett
e618410eb2 Export CSL JSON with title-short rather than shortTitle 2019-04-04 00:45:32 +09:00
Dan Stillman
9ed48f3837 Keep earliest Date Added when merging items
Closes #1669
2019-04-02 14:01:25 -04:00
Dan Stillman
d9cee322cd Tag selector performance overhaul
- Use react-virtualized to render tags on demand, reducing the number
  of DOM elements from potentially tens of thousands to <100. This
  requires tags to be absolutely positioned, so sizing and
  positioning need to be precomputed rather than relying on CSS.
- Avoid unnecessary refreshes, speed up tag retrieval, and optimize
  sorting
- Debounce reflowing when resizing tag selector

Also:

- Scroll to top when changing collections
- Allow tags to take up full width of tag selector without truncation

Closes #1649
Closes #281
2019-03-28 06:28:34 -04:00
Dan Stillman
71f9420cff Fix linked-file rename pref when retrieving metadata
Due to a typo in d0f7fd6df7, linked files were still being renamed even
with the pref off if metadata was found for the file. The test I added
was only for adding a file to an existing item, which didn't trigger
metadata retrieval.

This also adds a hook for stubbing the actual PDF recognition process so
we can test certain behaviors without making HTTP requests.
2019-03-27 06:43:07 -04:00
Dan Stillman
49226edd07 Temporarily ignore collection-title for podcast in itemFromCSLJSON test
See #1667
2019-03-21 02:22:07 -04:00
Dan Stillman
8f1f1f1fba Update itemFromCSLJSON test for podcast broadcast mapping
Follow up to bf4deeff8f
2019-03-21 02:20:38 -04:00
Dan Stillman
03941dafe0 Fix tests after 42667e7090
It seems like the Zotero.Utilities.debounce() on handleSearch() in
tagSelector.jsx was somehow causing the function to be run without being
triggered from the onSearch events, resulting in an extra render. I'm
not sure why that was happening, but it's fixed now that there's no
longer a debounce() there.
2019-03-20 08:54:48 -04:00
Dan Stillman
a92b29cebf Fix race conditions in tag selector tests
And take an optional second parameter in waitForTagSelector() to
indicate how many updates to wait for, since certain operations trigger
two updates, one from notify() and the other from onItemViewChanged().
2019-03-18 04:58:50 -04:00
Dan Stillman
b30d31c4ee Fix tag selector test 2019-03-15 21:57:38 -04:00
Dan Stillman
7717df637e Fix hang in server_connector test after 94ccba45b
The test incorrectly contained an item wait for an attachment that
doesn't get saved to the group library, but the test was passing because
the main item was being moved to the group twice. 94ccba45b somehow
fixed that, and since the behavior during the test is now correct, I'm
not going to worry about this unless we notice a problem.
2019-03-15 21:01:12 -04:00
Dan Stillman
94ccba45b9 Avoid unnecessary tag queries (regression from React tag selector) 2019-03-15 12:46:27 -04:00
Adomas Venčkauskas
72fb67d15b Restore "Delete All Automatic Tags" menu option for tag selector. Closes #1660 2019-03-06 16:22:05 +02:00
Dan Stillman
b20bf345e6 Address race condition in tag selector tests (#1659) 2019-03-06 07:27:11 -05:00
Adomas Venčkauskas
e8f186479f Address tag selector tests race conditions. Addresses #1659 2019-03-06 13:13:40 +02:00
Dan Stillman
d7dc5670d5 When clearing automatic tags, don't delete manual tags with same name 2019-03-05 07:55:00 -05:00
Dan Stillman
fc43514ff0 Deselect tags when deleted (regression from Reactification) 2019-03-05 07:55:00 -05:00
Dan Stillman
04779d8d1c Add import option for storing or linking files
This allows files in Mendeley imports to be stored and files in
RIS/BibTeX/etc. to be linked.

Closes #329
2019-03-02 06:24:33 -05:00
Dan Stillman
d0f7fd6df7 Add separate "Rename linked files" pref for auto-renaming
Defaulting to false

People who've been relying on auto-renaming of linked files for the last
year will have to turn this back on.
2019-02-26 22:23:43 -05:00
Dan Stillman
945c413c42 Closes #1644, Allow selecting multiple items with zotero://select
This adds selectItems() to ZoteroPane and collectionTreeView and removes
the ancient, unused 'expand' argument to selectItem(), which didn't
really make sense there. It also includes a new
itemTreeView::ensureRowsAreVisible() that tries to scroll to an
appropriate place (or, better yet, not scroll at all) given the
specified rows and page size.
2019-02-23 17:35:09 -05:00
Dan Stillman
4d3625f101 Fix tests after removing RECOGNIZE_URL 2019-02-19 04:37:22 -05:00
Dan Stillman
1061893998 "Attachment Content" search improvements
- Fix incorrect results for ANY search with multiple "Attachment
  Content" conditions and no other conditions
- Dramatically speed up single-word searches by avoiding unnecessary
  text scans (which probably addresses #1595)
- Clean up code
2019-02-19 04:10:25 -05:00
Frank Bennett
2c136ffbb0 Add test of multiple new insertions with implicit update on the first 2019-02-13 10:18:05 +02:00
Frank Bennett
bffbe00399 Test of implicit citation update 2019-02-13 10:17:58 +02:00
Frank Bennett
0e18c5f1de Test for bibliography sort update on change to item 2019-02-10 11:07:10 +09:00
Dan Stillman
3942b0e0d5 Fix a tag selector test after b1fad505d7 2019-01-30 04:44:16 -05:00
Dan Stillman
b1fad505d7 Fix sorting of colored tags after Reactification
Regressed to alphabetic sorting instead of number-key sorting
2019-01-29 22:18:49 -05:00
Dan Stillman
2d71b13ce0 Fix some spurious failures in PDF retrieval test 2019-01-29 07:35:39 -05:00
Dan Stillman
c2a637472e Fix foreign key constraint error for collections with all-numeric keys
This was like a lottery where instead of winning something your library
stopped syncing.
2019-01-29 05:09:00 -05:00
Dan Stillman
8b4bb62efc Don't load linked URLs during import
Regression from 5a6a772ca2, I think
2019-01-26 00:20:26 -05:00
Adomas Venčkauskas
f59f3c5f7e Fix a failing /connector/updateSession test (regr 90a70f7) 2019-01-22 12:25:15 +02:00
Dan Stillman
eb5d28b626
Merge pull request #1606 from adomasven/feature/reactify
Reactified Tag Selector
2019-01-21 04:07:28 -05:00
Adomas Venčkauskas
a24cada451 React Tag Selector polish, i18n and tests
- Added icon-button UI code for the menubutton
- Upgrade to React 16 to allow non-standard attrs, such as `tooltiptext`
to support XUL tooltips
- Add i18n support for React UI elements
- Update tests for reactified tag selector
2019-01-21 11:01:27 +02:00
Philipp Zumstein
97e53c5230 Fix some typos found by codespell (#1570)
$ codespell --skip="./.git,./styles,./translators,./chrome/locale,./resource,./chrome/content/zotero/xpcom/xregexp,./chrome/content/zotero/xpcom/rdf,./chrome/content/zotero/xpcom/citeproc.js,./test/resource,./node_modules" -q 2 -i 3 -w
2019-01-20 20:39:27 -05:00
Dan Stillman
9c2d0d7272 Add skipped test for importing related items from Zotero RDF
This is hard to do currently because the natural place to do it (and
where the previous seeAlso stuff was done) is translate_item.js, but
with async import translators that now only gets one item at a time,
whereas saving item relations requires all items to be saved. So this
would probably need to be done in the import code in translate.js.

It might also require undoing
https://github.com/zotero/zotero/pull/453 so that getResourceURI() works
on notes and figuring out another solution for the problem that was
trying to solve.
2019-01-14 02:36:59 -05:00
Dan Stillman
c8fb2ecd1f Add test for exporting related items in Zotero RDF
These aren't currently imported, but better to at least include them.
2019-01-14 02:29:09 -05:00
Dan Stillman
0d5abb018a Ignore invalid paths during export
Invalid paths, including Windows UNC paths on other OSes, caused exports
to fail. Now they're ignored, which is what we do for other missing
attachment files.

Fixes #1622
2019-01-12 02:34:31 -05:00
Dan Stillman
9c0f5998a3 Use correct time-zone offset for access dates in bibliographies
Fixes #1069
2019-01-10 03:14:28 -05:00
Dan Stillman
5a6a772ca2 Fix linked-URL attachments not being saved to groups without files
It looks like this may have been broken for years.
2019-01-08 03:45:26 -05:00
Dan Stillman
506ed313da Fix moving item via connector to library with files from one without
Broken in ce5be0bc75 for non-webpage items
2019-01-07 05:16:17 -05:00
Dan Stillman
7575cd8b29 Don't accept keyboard input before new-collection prompt appears
Fixes #1613
2018-12-27 07:11:15 -05:00