Commit graph

886 commits

Author SHA1 Message Date
Dan Stillman
bb59429664 Add "Convert Linked Files to Stored Files…" menu option
In new File → Manage Attachments submenu

Closes #1637
2019-08-19 05:00:32 -04:00
Dan Stillman
c2e79c0717 Close window after protocol handler tests
Regression from fda002ec34
2019-08-03 02:48:01 -04:00
Dan Stillman
fda002ec34 Beginning of tests for zotero://select 2019-08-02 02:29:08 -04:00
Dan Stillman
9becefc3fd Fix test for AMA subtitle capitalization 2019-08-01 17:32:56 -04:00
Dan Stillman
9b82373f70 Add ability for Scaffold to provide alternative translators
Zotero.Translate::setTranslatorProviderMethods(methods) can be used to
provide custom 'get' and 'getAllForType' methods that override the
default Zotero.Translators methods.
2019-07-24 04:31:11 -04:00
Dan Stillman
2603373b86 Additional protections for HTTP endpoints
Reject browser-based requests that don't require a CORS preflight
request [1] if they don't come from the connector or include
Zotero-Allowed-Request: 1

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#Simple_requests
2019-07-15 07:30:12 -04:00
Dan Stillman
708b8798c2 Check for X-Zotero-Connector-API-Version header in /connector/import 2019-07-12 02:08:55 -04:00
Adomas Venčkauskas
c4f40c38e9 Unsuppress accidental test suppression from fe01e17 2019-07-04 15:46:24 +03:00
Adomas Venčkauskas
fe01e17e93 Fix integration test breakage (regression cd14a53) 2019-07-04 15:38:46 +03:00
Adomas Venčkauskas
cd14a536dc Add retracted citation warning tests. Fix a bug with embedded prompts 2019-07-04 15:16:26 +03:00
Dan Stillman
82e7f67df8 Fix retraction test 2019-07-04 07:58:56 -04:00
Dan Stillman
0beddb9680 Add flags to disable retraction warnings
Separate flags for hiding the retraction altogether and for hiding
citation warnings for it

New functions:

Zotero.Retractions.hideRetraction(item)
Zotero.Retractions.shouldShowCitationWarning(item)
Zotero.Retractions.disableCitationWarningsForItem(item)

Addresses #1710
2019-07-03 01:23:02 -04:00
Dan Stillman
f49d5805cd Apply subtitle capitalization to additional styles
This also fixes a style initialization error when the style id is
invalid, which was a regression in a0be2da42c.
2019-07-01 22:39:12 -04:00
Dan Stillman
ead93b6ccc Stop uploading files on quota error until next manual sync or restart 2019-06-22 05:29:47 -04:00
Dan Stillman
50a4308514 Remove retraction flag from items that no longer match 2019-06-19 06:41:33 -04:00
Dan Stillman
b4b19c4811 Add Zotero.Retractions.getRetractionsFromJSON()
Addresses #1703
2019-06-12 01:05:49 -04:00
Dan Stillman
d353439980 Add retractions.enabled hidden pref to disable retraction checking 2019-06-11 21:24:17 -04:00
Dan Stillman
5c03813d81 Add Retracted Items virtual collection
Shown automatically when retracted items are detected
2019-06-10 02:38:42 -04:00
Dan Stillman
79a8917bdc Fix test failure after 3727d0f559 2019-06-07 03:17:26 -04:00
Dan Stillman
7f4f2770ba Add notification banner when retracted items are found
And other retraction tweaks
2019-06-07 01:15:17 -04:00
Dan Stillman
985f10d5ed Fix WebDAV test after 1f1b2e9b2 2019-06-03 03:35:32 -04:00
Dan Stillman
c0e2aa03d3 Fix test for delay after second library version conflict 2019-05-31 02:44:41 -04:00
Dan Stillman
e33a2d730d Fix tag type handling when merging items
Most importantly, don't change all tags to manual on the merged item.
2019-05-29 05:43:33 -04:00
Adomas Ven
48778f2847 Document export-import UI and integration code (#1501) 2019-05-14 21:06:18 -04:00
Dan Stillman
3fbb17a2e6 Properly upload local changes after automatic conflict resolution
If an object changed on both sides and the changes were either
non-conflicting or identical but there were other local changes, the
local object was incorrectly being marked as synced, causing it not to
be uploaded until it was next modified locally.
2019-05-08 04:34:21 -04:00
Dan Stillman
ec0b993a17 Fix erroneous sync conflict if creators are changed in the same way 2019-05-03 01:15:27 -04:00
Adomas Venčkauskas
78c3d5808b Fix citeproc errors due to improper citeproc state updates
The error is triggered upon initial interaction with a doc after Zotero
restart or if new external citations (copied into the document) are
peresnt and `session.updateSession()` is called without a subsequent
`session.updateDocument()` call. `session.updateSession()` is called
without a subsequent `session.updateDocument()` call every time the
user cancels a citation insert.

More specifically, `session.updateSession()` is called every time a
citation dialog is invoked. It retrieves all citations and writes them
into a local `session.citationsByIndex` object. Moreover, it marks
each citation that hasn't seen before in a `session.newIndices` object.
`session.newIndices` is there to ensure that we load every new citation
into citeproc upon document update. This object is built by marking any
citation that does not appear in the previous invocation's list of
citations as new. However, if the document is never updated (because the
user cancels the insertion) then the new indices are not loaded
into citeproc. This commit fixes that, by excluding citeproc unloaded
items from the previous invocation's citation list.
2019-04-08 17:43:52 +03:00
Frank Bennett
e618410eb2 Export CSL JSON with title-short rather than shortTitle 2019-04-04 00:45:32 +09:00
Dan Stillman
9ed48f3837 Keep earliest Date Added when merging items
Closes #1669
2019-04-02 14:01:25 -04:00
Dan Stillman
d9cee322cd Tag selector performance overhaul
- Use react-virtualized to render tags on demand, reducing the number
  of DOM elements from potentially tens of thousands to <100. This
  requires tags to be absolutely positioned, so sizing and
  positioning need to be precomputed rather than relying on CSS.
- Avoid unnecessary refreshes, speed up tag retrieval, and optimize
  sorting
- Debounce reflowing when resizing tag selector

Also:

- Scroll to top when changing collections
- Allow tags to take up full width of tag selector without truncation

Closes #1649
Closes #281
2019-03-28 06:28:34 -04:00
Dan Stillman
71f9420cff Fix linked-file rename pref when retrieving metadata
Due to a typo in d0f7fd6df7, linked files were still being renamed even
with the pref off if metadata was found for the file. The test I added
was only for adding a file to an existing item, which didn't trigger
metadata retrieval.

This also adds a hook for stubbing the actual PDF recognition process so
we can test certain behaviors without making HTTP requests.
2019-03-27 06:43:07 -04:00
Dan Stillman
49226edd07 Temporarily ignore collection-title for podcast in itemFromCSLJSON test
See #1667
2019-03-21 02:22:07 -04:00
Dan Stillman
8f1f1f1fba Update itemFromCSLJSON test for podcast broadcast mapping
Follow up to bf4deeff8f
2019-03-21 02:20:38 -04:00
Dan Stillman
03941dafe0 Fix tests after 42667e7090
It seems like the Zotero.Utilities.debounce() on handleSearch() in
tagSelector.jsx was somehow causing the function to be run without being
triggered from the onSearch events, resulting in an extra render. I'm
not sure why that was happening, but it's fixed now that there's no
longer a debounce() there.
2019-03-20 08:54:48 -04:00
Dan Stillman
a92b29cebf Fix race conditions in tag selector tests
And take an optional second parameter in waitForTagSelector() to
indicate how many updates to wait for, since certain operations trigger
two updates, one from notify() and the other from onItemViewChanged().
2019-03-18 04:58:50 -04:00
Dan Stillman
b30d31c4ee Fix tag selector test 2019-03-15 21:57:38 -04:00
Dan Stillman
7717df637e Fix hang in server_connector test after 94ccba45b
The test incorrectly contained an item wait for an attachment that
doesn't get saved to the group library, but the test was passing because
the main item was being moved to the group twice. 94ccba45b somehow
fixed that, and since the behavior during the test is now correct, I'm
not going to worry about this unless we notice a problem.
2019-03-15 21:01:12 -04:00
Dan Stillman
94ccba45b9 Avoid unnecessary tag queries (regression from React tag selector) 2019-03-15 12:46:27 -04:00
Adomas Venčkauskas
72fb67d15b Restore "Delete All Automatic Tags" menu option for tag selector. Closes #1660 2019-03-06 16:22:05 +02:00
Dan Stillman
b20bf345e6 Address race condition in tag selector tests (#1659) 2019-03-06 07:27:11 -05:00
Adomas Venčkauskas
e8f186479f Address tag selector tests race conditions. Addresses #1659 2019-03-06 13:13:40 +02:00
Dan Stillman
d7dc5670d5 When clearing automatic tags, don't delete manual tags with same name 2019-03-05 07:55:00 -05:00
Dan Stillman
fc43514ff0 Deselect tags when deleted (regression from Reactification) 2019-03-05 07:55:00 -05:00
Dan Stillman
04779d8d1c Add import option for storing or linking files
This allows files in Mendeley imports to be stored and files in
RIS/BibTeX/etc. to be linked.

Closes #329
2019-03-02 06:24:33 -05:00
Dan Stillman
d0f7fd6df7 Add separate "Rename linked files" pref for auto-renaming
Defaulting to false

People who've been relying on auto-renaming of linked files for the last
year will have to turn this back on.
2019-02-26 22:23:43 -05:00
Dan Stillman
945c413c42 Closes #1644, Allow selecting multiple items with zotero://select
This adds selectItems() to ZoteroPane and collectionTreeView and removes
the ancient, unused 'expand' argument to selectItem(), which didn't
really make sense there. It also includes a new
itemTreeView::ensureRowsAreVisible() that tries to scroll to an
appropriate place (or, better yet, not scroll at all) given the
specified rows and page size.
2019-02-23 17:35:09 -05:00
Dan Stillman
4d3625f101 Fix tests after removing RECOGNIZE_URL 2019-02-19 04:37:22 -05:00
Dan Stillman
1061893998 "Attachment Content" search improvements
- Fix incorrect results for ANY search with multiple "Attachment
  Content" conditions and no other conditions
- Dramatically speed up single-word searches by avoiding unnecessary
  text scans (which probably addresses #1595)
- Clean up code
2019-02-19 04:10:25 -05:00
Frank Bennett
2c136ffbb0 Add test of multiple new insertions with implicit update on the first 2019-02-13 10:18:05 +02:00
Frank Bennett
bffbe00399 Test of implicit citation update 2019-02-13 10:17:58 +02:00
Frank Bennett
0e18c5f1de Test for bibliography sort update on change to item 2019-02-10 11:07:10 +09:00
Dan Stillman
3942b0e0d5 Fix a tag selector test after b1fad505d7 2019-01-30 04:44:16 -05:00
Dan Stillman
b1fad505d7 Fix sorting of colored tags after Reactification
Regressed to alphabetic sorting instead of number-key sorting
2019-01-29 22:18:49 -05:00
Dan Stillman
2d71b13ce0 Fix some spurious failures in PDF retrieval test 2019-01-29 07:35:39 -05:00
Dan Stillman
c2a637472e Fix foreign key constraint error for collections with all-numeric keys
This was like a lottery where instead of winning something your library
stopped syncing.
2019-01-29 05:09:00 -05:00
Dan Stillman
8b4bb62efc Don't load linked URLs during import
Regression from 5a6a772ca2, I think
2019-01-26 00:20:26 -05:00
Adomas Venčkauskas
f59f3c5f7e Fix a failing /connector/updateSession test (regr 90a70f7) 2019-01-22 12:25:15 +02:00
Dan Stillman
eb5d28b626
Merge pull request #1606 from adomasven/feature/reactify
Reactified Tag Selector
2019-01-21 04:07:28 -05:00
Adomas Venčkauskas
a24cada451 React Tag Selector polish, i18n and tests
- Added icon-button UI code for the menubutton
- Upgrade to React 16 to allow non-standard attrs, such as `tooltiptext`
to support XUL tooltips
- Add i18n support for React UI elements
- Update tests for reactified tag selector
2019-01-21 11:01:27 +02:00
Philipp Zumstein
97e53c5230 Fix some typos found by codespell (#1570)
$ codespell --skip="./.git,./styles,./translators,./chrome/locale,./resource,./chrome/content/zotero/xpcom/xregexp,./chrome/content/zotero/xpcom/rdf,./chrome/content/zotero/xpcom/citeproc.js,./test/resource,./node_modules" -q 2 -i 3 -w
2019-01-20 20:39:27 -05:00
Dan Stillman
9c2d0d7272 Add skipped test for importing related items from Zotero RDF
This is hard to do currently because the natural place to do it (and
where the previous seeAlso stuff was done) is translate_item.js, but
with async import translators that now only gets one item at a time,
whereas saving item relations requires all items to be saved. So this
would probably need to be done in the import code in translate.js.

It might also require undoing
https://github.com/zotero/zotero/pull/453 so that getResourceURI() works
on notes and figuring out another solution for the problem that was
trying to solve.
2019-01-14 02:36:59 -05:00
Dan Stillman
c8fb2ecd1f Add test for exporting related items in Zotero RDF
These aren't currently imported, but better to at least include them.
2019-01-14 02:29:09 -05:00
Dan Stillman
0d5abb018a Ignore invalid paths during export
Invalid paths, including Windows UNC paths on other OSes, caused exports
to fail. Now they're ignored, which is what we do for other missing
attachment files.

Fixes #1622
2019-01-12 02:34:31 -05:00
Dan Stillman
9c0f5998a3 Use correct time-zone offset for access dates in bibliographies
Fixes #1069
2019-01-10 03:14:28 -05:00
Dan Stillman
5a6a772ca2 Fix linked-URL attachments not being saved to groups without files
It looks like this may have been broken for years.
2019-01-08 03:45:26 -05:00
Dan Stillman
506ed313da Fix moving item via connector to library with files from one without
Broken in ce5be0bc75 for non-webpage items
2019-01-07 05:16:17 -05:00
Dan Stillman
7575cd8b29 Don't accept keyboard input before new-collection prompt appears
Fixes #1613
2018-12-27 07:11:15 -05:00
Adomas Venčkauskas
4072d444e7 Ensure Test Import Translator.js #doImport() does not rely on #detectImport() 2018-12-21 15:16:27 +02:00
Dan Stillman
37c51242d2 Add test for copying collection to another library 2018-12-16 02:24:21 -05:00
Dan Stillman
80f6b857f6 Add Utilities.Internal.getNextName(name, existingNames)
Function to get the next available name when duplicating something
2018-12-16 02:24:21 -05:00
Dan Stillman
223f582aa7 Fix search error on nonexistent collection in recursive mode
And don't return results for a nonexistent parent search
2018-11-28 15:31:57 -07:00
Dan Stillman
6137aeddb8 Follow meta redirects for Find Available PDF
This fixes direct and VPN-based retrieval of PDFs for Elsevier (e.g.,
ScienceDirect) items that have a DOI but no URL, since Elsevier resolves
DOIs through an intermediate page.
2018-11-26 00:57:48 -07:00
Dan Stillman
4b81e03f28 Improve reliability of PDF retrieval delay tests 2018-10-09 19:14:59 -04:00
Dan Stillman
477c28b1e3 Fix another PDF retrieval test after be8db4fc50 2018-10-09 19:03:06 -04:00
Dan Stillman
f3aef41c90 Update tests for auto-recognizing dragged PDFs after be8db4fc50 2018-10-09 18:25:30 -04:00
Dan Stillman
1b9811c31d Fix test failures after 18f79f9796 2018-10-06 01:38:32 -04:00
Martynas Bagdonas
ae7ef109b7 Add universal progress queues (#1573) 2018-10-05 01:56:46 -04:00
Martynas Bagdonas
ec5424d5ae Follow meta redirects and use the final URL when processing documents (#1568) 2018-10-03 22:24:39 -04:00
Dan Stillman
d899134e7c Automatically delay between PDF retrieval requests to the same domain
Delay requests to the same domain by 1 second, respect a Retry-After
header if present for 429 and 503, and delay for 10 seconds on 429 or
5xx otherwise.
2018-09-22 04:03:25 -04:00
Dan Stillman
b782120840 Add followRedirects: false option to Zotero.HTTP.request()
Currently only .status and .getResponseHeader() (for getting 'Location')
are available in the returned object, but we could make the body
available if necessary.
2018-09-22 04:03:25 -04:00
Dan Stillman
7cf466a0b6 Save OA PDFs when the DOI resolves directly to the file 2018-09-06 16:44:11 -04:00
Dan Stillman
18821984e0 Accept a Blob as input in Zotero.File.putContentsAsync() 2018-09-06 16:44:11 -04:00
Dan Stillman
05d8e7a8a3 Check Extra field for DOIs for PDF retrieval
E.g., a book with a DOI in Extra

Closes #1551
2018-08-30 16:52:24 -04:00
Dan Stillman
e7e681c113 Set retrieved OA PDF title to reflect version
"Full Text", "Accepted Version", or "Submitted Version"
2018-08-25 16:48:17 -04:00
Dan Stillman
d5e7bae254 Fix broken test after ec76575645 2018-08-19 04:31:40 -04:00
Dan Stillman
e48a1a2abb Remove lots of old Zotero for Firefox code
We still run tests in Firefox, so not everything can be removed.
2018-08-17 03:17:08 -04:00
Dan Stillman
c2ca70055c Merge branch 'electron' 2018-08-16 19:16:46 -04:00
Dan Stillman
ce5be0bc75 Automatically download open-access PDFs when saving via the connector
If there's no translated PDF or the translated PDF fails and the item
has a DOI, check Zotero's Unpaywall mirror for possible sources and try
to download one of those.

Unlike with "Add Item by Identifier" and "Find Available PDF" in the
item context menu, this does not try the DOI/URL page, since it would
result in more data leakage and most of the time you'd be saving from
the DOI page already. We could consider offering it as an option, but
for it to be useful, you'd have to have an institutional subscription,
be on-campus or connected via VPN (for now), and be saving from
somewhere other than the main page.

A new connector endpoint, sessionProgress, takes the place of
attachmentProgress. Unlike attachmentProgress, sessionProgress can show
new attachments that have been added to the save, and with a little more
work should also be able to show when a parent item has been recognized
for a directly saved PDF.

This also adds support for custom PDF resolvers, available to all PDF
retrieval methods. I'll document those separately.

Closes #1542
2018-08-16 00:57:22 -04:00
Dan Stillman
bcf94942f9 Don't include "Zotero" in User-Agent sent to non-Zotero sites 2018-08-16 00:57:22 -04:00
Adomas Venčkauskas
01f3159b2f Ensure all failing integration tests are fixed (8d8410ff). Closes #1546 2018-08-14 11:40:19 +03:00
Dan Stillman
11f8bd20d6 Fix title display when installing style via connector (broken in 0f1687f1c7) 2018-08-13 18:17:35 -04:00
Adomas Venčkauskas
8d8410ffa0 Fix failing integration test (regr 0f1687f). Closes #1546 2018-08-13 14:22:15 +03:00
Dan Stillman
6478a375e3 Fix WebDAV tests after 276bb99fc9 2018-08-09 18:20:02 -04:00
Dan Stillman
679a6d5cc7 PDF retrieval improvements
- Add the ability to extract a PDF URL from a given webpage using the
  translation framework
- Add the ability to get open-access PDFs from landing pages from
  Unpaywall data in addition to direct PDF URLs
- Use the above functionality to improve PDF retrieval for "Add Item by
  Identifier"
- Add "Find Available PDFs" option to the item context menu to retrieve
  PDFs for existing items from the DOI or URL page or using Unpaywall
  data. The option appears for single items with a DOI or URL and no PDF,
  and it always appears when selecting multiple top-level items (but
  skips ineligible items).

PDF extraction from DOI/URL pages will currently only work with
unauthenticated access (i.e., on-campus or VPN, but not via a web-based
proxy).

Supersedes and closes #948
2018-08-07 04:58:15 -04:00
Sylvester Keil
aa908516eb
Use spy.resetHistory instead of reset
Reset was deprecated in 4.1.4 and removed in 5.0.
2018-08-04 22:07:27 +02:00
Sylvester Keil
37412c68ff
Fix style download stub 2018-08-03 13:42:15 +02:00
Sylvester Keil
aa94a62a74
Make test more independent of db state
If there are library ids above 9 string sorting may not put the
highest id last in the array.
2018-08-03 11:53:54 +02:00
Sylvester Keil
824e9c78d6
Move localeCompare tests to Zotero.Intl tests 2018-08-01 15:41:54 +02:00
Sylvester Keil
8e6896c27d
Move Zotero.getString() test to Zotero.Intl tests 2018-07-30 10:20:05 +02:00
Sylvester Keil
143fdd5f2b
Revert "Remove legacy iterator protocol use in Zotero.DB"
Reverting this until we can use Fx60 for testing.

This reverts commit dfe412d448.
2018-07-30 10:18:11 +02:00