Commit graph

6892 commits

Author SHA1 Message Date
Dan Stillman
d899134e7c Automatically delay between PDF retrieval requests to the same domain
Delay requests to the same domain by 1 second, respect a Retry-After
header if present for 429 and 503, and delay for 10 seconds on 429 or
5xx otherwise.
2018-09-22 04:03:25 -04:00
Dan Stillman
536c07e9f4 Pass URL to Zotero.HTTP.UnexpectedStatusException
This allows the URL to be retrieved at .url in the thrown error.
2018-09-22 04:03:25 -04:00
Dan Stillman
b25d2d9c56 Update Zotero.HTTP.request() documentation 2018-09-22 04:03:25 -04:00
Dan Stillman
b782120840 Add followRedirects: false option to Zotero.HTTP.request()
Currently only .status and .getResponseHeader() (for getting 'Location')
are available in the returned object, but we could make the body
available if necessary.
2018-09-22 04:03:25 -04:00
Dan Stillman
b8db83af08 Split out a couple helper functions for PDF retrieval 2018-09-21 21:12:15 -04:00
Dan Stillman
c6a2af7ab5 Use 'load' instead of 'pageshow' in Zotero.HTTP.loadDocuments()
This seems to fix snapshots in Twitter, which need to wait for a
JS-based self-redirect [1].

[1] https://blog.twitter.com/engineering/en_us/topics/insights/2018/twitter_silhouette.html
2018-09-21 21:12:15 -04:00
Philipp Zumstein
2957db2aa1 Delete two duplicate switch cases (#1569) 2018-09-14 17:01:47 -04:00
Dan Stillman
f4ac511b50 Copy items across libraries in batches of 100
This will hopefully fix failures when copying huge numbers of items.
2018-09-14 04:14:08 -04:00
Adomas Venčkauskas
0cb683ab18 Ensure we don't maintain document sessions across different doc processors 2018-09-10 08:40:48 +03:00
Maecenas Li
dd9ee66212 Add support for marginnoteapp:// URI protocol (#1567)
https://forums.zotero.org/discussion/comment/316070/#Comment_316070
2018-09-08 06:01:02 -04:00
Dan Stillman
d8b2d24524 Better fix for invalid file localUrl in Mendeley DB 2018-09-07 01:22:09 -04:00
Dan Stillman
3f2ee35d59 Tweak debug line 2018-09-06 17:22:03 -04:00
Dan Stillman
7cf466a0b6 Save OA PDFs when the DOI resolves directly to the file 2018-09-06 16:44:11 -04:00
Dan Stillman
18821984e0 Accept a Blob as input in Zotero.File.putContentsAsync() 2018-09-06 16:44:11 -04:00
Dan Stillman
36afb34cf6 Show warning if Zotero is started from Mac disk image
Doing so can cause the Word for Mac extension not to load, and possibly
other problems.

https://forums.zotero.org/discussion/73470/word-could-not-communicate-with-zotero-with-three-full-uninstalls-and-reinstalls
2018-09-06 00:53:50 -04:00
Dan Stillman
4b64cc312c Handle possible null file URL in Mendeley import 2018-09-05 14:41:13 -04:00
Dan Stillman
03a15d47af Remove redundant OA PDF search in Add Item by Identifier 2018-09-04 14:15:25 -04:00
Adomas Venčkauskas
66ed33b9fb Fix must insert citation prompt in new doc with copied fields. Closes #1564 2018-09-04 16:37:46 +03:00
Dan Stillman
05d8e7a8a3 Check Extra field for DOIs for PDF retrieval
E.g., a book with a DOI in Extra

Closes #1551
2018-08-30 16:52:24 -04:00
Adomas Venčkauskas
944188fa63 Fix RTF dontUpdate=true citations updating on LO. Closes #1561 2018-08-30 10:55:54 +03:00
Dan Stillman
5a008cf861 Don't save collections from Scaffold
Fixes zotero/scaffold#60
2018-08-27 17:45:42 -04:00
Dan Stillman
4eb4490723 Don't limit PDF retrieval to dev builds 2018-08-25 17:00:53 -04:00
Dan Stillman
862ac5ab40 Handle missing 'attachments' array in /connector/saveItems 2018-08-25 16:55:13 -04:00
Dan Stillman
e7e681c113 Set retrieved OA PDF title to reflect version
"Full Text", "Accepted Version", or "Submitted Version"
2018-08-25 16:48:17 -04:00
Dan Stillman
abc4cd9dfe Clear translatorCache table when resetting translators 2018-08-21 22:36:20 -04:00
Dan Stillman
91a3c27235 Fail schemaUpdatePromise and show alert on initialization failure
This will also cause the word processor plugin to throw an error isntead
of hanging if there was a problem loading translators or styles.

Closes #1557
2018-08-21 22:36:20 -04:00
Dan Stillman
475bd17b71 Fix 'title is null' error
Not sure why item.title would ever be null, but this was in a user's
error report and seemed to be causing a restart error.
2018-08-21 22:36:20 -04:00
Adomas Venčkauskas
0fd28959bf Ensure the doc prefs window is smaller than screen height
Dynamically resizes the style picker up to the window height of 700px
2018-08-20 12:42:45 +03:00
Dan Stillman
8d5338db2d Additional debug output for setting base directory 2018-08-19 22:49:09 -04:00
Dan Stillman
4f98ad9567 Vacuum database automatically after clearing full-text index 2018-08-19 04:06:04 -04:00
Dan Stillman
926e2f7317 Reduce memory consumption of full-text index clear 2018-08-19 04:05:38 -04:00
Dan Stillman
18021d1aeb Disable rebuild/clear buttons in Search prefs while in progress 2018-08-19 03:51:00 -04:00
Dan Stillman
c9e80319bb Support "Show File" on evicted iCloud Drive files 2018-08-19 02:41:45 -04:00
Dan Stillman
ec76575645 Trigger downloading and opening of evicted iCloud Drive files 2018-08-19 01:44:49 -04:00
Dan Stillman
e48a1a2abb Remove lots of old Zotero for Firefox code
We still run tests in Firefox, so not everything can be removed.
2018-08-17 03:17:08 -04:00
Dan Stillman
c2ca70055c Merge branch 'electron' 2018-08-16 19:16:46 -04:00
Dan Stillman
6d898e4774 Add extra debug logging for file merging 2018-08-16 17:09:14 -04:00
Martynas Bagdonas
7a3dc61892 Add an utility to fix author name capitalization (#1550) 2018-08-16 05:22:39 -04:00
Dan Stillman
1ffc34a8ac Don't try to retrieve PDFs when importing from a file 2018-08-16 05:21:00 -04:00
Dan Stillman
6d5abb55f0 Fix regression from 9fec063f4e 2018-08-16 05:20:03 -04:00
Adomas Venčkauskas
50a4bae945 Remove unused ZP.addItemFromPage() 2018-08-16 11:48:48 +03:00
Adomas Venčkauskas
044b08c9af Fix broken DataDirectory.removeSubdirectory() (regression 7f2d83a6) 2018-08-16 11:48:48 +03:00
Adomas Venčkauskas
2cf940763f Remove usage of HTTP.loadDocuments() in electron shared code 2018-08-16 11:48:48 +03:00
Dan Stillman
9fec063f4e Tweak display of attachment status lines in connector for PDF retrieval
Generally speaking, don't show a status line unless we have reason to
believe we'll be saving something.
2018-08-16 04:04:20 -04:00
Dan Stillman
ce5be0bc75 Automatically download open-access PDFs when saving via the connector
If there's no translated PDF or the translated PDF fails and the item
has a DOI, check Zotero's Unpaywall mirror for possible sources and try
to download one of those.

Unlike with "Add Item by Identifier" and "Find Available PDF" in the
item context menu, this does not try the DOI/URL page, since it would
result in more data leakage and most of the time you'd be saving from
the DOI page already. We could consider offering it as an option, but
for it to be useful, you'd have to have an institutional subscription,
be on-campus or connected via VPN (for now), and be saving from
somewhere other than the main page.

A new connector endpoint, sessionProgress, takes the place of
attachmentProgress. Unlike attachmentProgress, sessionProgress can show
new attachments that have been added to the save, and with a little more
work should also be able to show when a parent item has been recognized
for a directly saved PDF.

This also adds support for custom PDF resolvers, available to all PDF
retrieval methods. I'll document those separately.

Closes #1542
2018-08-16 00:57:22 -04:00
Dan Stillman
7a646a292b Pass response data to Zotero.WebProgressFinishListener callback
Pass an object with 'status' and 'contentType' if available
2018-08-16 00:57:22 -04:00
Dan Stillman
bcf94942f9 Don't include "Zotero" in User-Agent sent to non-Zotero sites 2018-08-16 00:57:22 -04:00
Dan Stillman
65bdef42a1 Fix double PDF when adding arXiv paper with Add Item by Identifier 2018-08-16 00:57:22 -04:00
Dan Stillman
e38725cf65 Localize new PDF retrieval strings
Closes #1541
2018-08-15 21:32:12 -04:00
Adomas Venčkauskas
850d2d2977 Expose private Attachments functions for electron 2018-08-15 17:11:30 +03:00