Commit graph

6875 commits

Author SHA1 Message Date
Dan Stillman
bb46d72fd1 Add DB integrity check repair step for TEXT userID
Follow-up to 26056c87f1
2018-08-09 04:18:30 -04:00
Dan Stillman
c45c9cc23b Fix error backing up database
Broken since f7e411d561. This affected automatic backups and DB
integrity check repairs (which do a backup first).
2018-08-09 04:17:45 -04:00
Dan Stillman
63d8d57f83 Add userID fix to earlier schema update
Since the userID is used in relations updates moving from Zotero 4 to 5
2018-08-09 03:42:28 -04:00
Dan Stillman
26056c87f1 Fix non-integer userID in settings table
A submitted database had a text userID with a trailing "A0. Not sure how
that happened -- it doesn't appear to be possible in current code -- but
it caused group permissions not to be properly synced.
2018-08-09 03:30:47 -04:00
Dan Stillman
8d8109dbac Rename/move some of the PDF retrieval functions 2018-08-07 15:40:52 -04:00
Dan Stillman
679a6d5cc7 PDF retrieval improvements
- Add the ability to extract a PDF URL from a given webpage using the
  translation framework
- Add the ability to get open-access PDFs from landing pages from
  Unpaywall data in addition to direct PDF URLs
- Use the above functionality to improve PDF retrieval for "Add Item by
  Identifier"
- Add "Find Available PDFs" option to the item context menu to retrieve
  PDFs for existing items from the DOI or URL page or using Unpaywall
  data. The option appears for single items with a DOI or URL and no PDF,
  and it always appears when selecting multiple top-level items (but
  skips ineligible items).

PDF extraction from DOI/URL pages will currently only work with
unauthenticated access (i.e., on-campus or VPN, but not via a web-based
proxy).

Supersedes and closes #948
2018-08-07 04:58:15 -04:00
Adomas Venčkauskas
51e181cb81 Fix some XPCOM Zotero breakage from previous commits 2018-08-06 12:24:17 +03:00
Sylvester Keil
a8dce25cef
Deprecate Z.File.getBinaryContents 2018-08-03 22:37:09 +02:00
Sylvester Keil
fa7a330b0e
Use Services.wm getter 2018-08-03 13:29:08 +02:00
Sylvester Keil
ad0c8aad53
Use Z.File.getResourceAsync to load local resources 2018-08-03 13:22:22 +02:00
Sylvester Keil
b0fe7d546d
Make Z.Styles.validate more portable 2018-08-03 13:21:12 +02:00
Sylvester Keil
07464d5c84
Make File.getResource expect full URLs
This change makes is easier to support different URL schemes
when overriding on other platforms.
2018-08-03 10:49:15 +02:00
Sylvester Keil
6fd25419a9
Use Services.io getter 2018-08-02 23:05:17 +02:00
Sylvester Keil
dc688a876f
Keep returning nsIFile from old directory getters 2018-08-02 17:56:57 +02:00
Sylvester Keil
91ddec5bd9
Move lazy, spawn, and serial to Utilities.Internal 2018-08-02 17:20:09 +02:00
Sylvester Keil
95559a0c62
Use Services.io getter 2018-08-02 13:11:15 +02:00
Sylvester Keil
7f2d83a602
Move data dire getters to Z.DataDirectory
Add Z.DataDirectory.getSubdirectory which, optionally, creates
the directory.

Add async Z.DataDirectory.removeSubdirectory and use it for
Z.removeTempDirectory (was sync call before!).
2018-08-02 13:02:28 +02:00
Sylvester Keil
fc8037a83b
Accept strings in File.createDirectoryIfNecessary 2018-08-02 12:52:04 +02:00
Sylvester Keil
e9e6a8313d
Move prviliged public methods to Utilities.Internal
Moving flattenArguments, setFontSize and getAncestorByTagName
2018-08-02 11:59:58 +02:00
Sylvester Keil
caaaed0af1
Move privileged public functions to Z.Utilities 2018-08-01 23:57:02 +02:00
Adomas Venčkauskas
e6d5be755a Move some Utility functions from Zotero to Z.U.Internals 2018-08-01 16:30:17 +03:00
Sylvester Keil
887913de09
Fix use of bluebird's .delay 2018-08-01 11:49:00 +02:00
Sylvester Keil
cc47a3aab5
Use hard-coded list for default script direction 2018-07-30 13:23:54 +02:00
Sylvester Keil
94539ceac5
Do not bind to Zotero.Intl before it has been imported 2018-07-30 13:22:20 +02:00
Sylvester Keil
4a97529b66
Move getLocaleCollation and localeCompare to Zotero.Intl 2018-07-30 12:35:11 +02:00
Sylvester Keil
3a8d04f467
Remove Zotero.localeJoin 2018-07-30 12:33:08 +02:00
Sylvester Keil
143fdd5f2b
Revert "Remove legacy iterator protocol use in Zotero.DB"
Reverting this until we can use Fx60 for testing.

This reverts commit dfe412d448.
2018-07-30 10:18:11 +02:00
Sylvester Keil
2dce5f4842
Import PluralForm.jsm in Zotero.Intl.
PluralForm is not used directly anywhere else.
2018-07-30 10:04:14 +02:00
Sylvester Keil
e24f3414b8
Zotero.intl -> Zotero.Intl
Use upper-case namespace for consitency.
2018-07-30 10:00:53 +02:00
Sylvester Keil
38166d4e05
Add Zotero.intl namespace
Move `Zotero.getString()` and intl init code to `Zotero.intl` to make
it easier to re-use.

Link `Zotero.getString()` to `Zotero.intl.getString()`.

Do not expose `getStringFromBundle`, `pluralFormGet`, and
`pluralFormNumForms` because they are not used.
2018-07-28 21:31:20 +02:00
Sylvester Keil
e8cd4c9338
Use Services.io to access IOService 2018-07-27 16:44:28 +02:00
Sylvester Keil
3f2dd94752
Load schema resources via Zotero.File.getResourceAsync 2018-07-27 15:56:34 +02:00
Sylvester Keil
aa1adc8817
Add Zotero.File.getResourceAsync 2018-07-27 15:56:00 +02:00
Dan Stillman
cdda572728 Support additional data from OA PDF lookup service
Handle an array of objects with 'url' and 'version' rather than just an
array of URLs.

Also:

- Don't throw an error from addOpenAccessPDF() if there's an error from
  getOpenAccessPDFURLs()
- Make addPDFFromURLs() a separate function so URL lookup can be done
  separately from download
2018-07-27 00:55:19 -04:00
Adomas Venčkauskas
c3bdf72f35 Misc nodejs environment compatibility 2018-07-25 17:45:06 +03:00
Adomas Venčkauskas
b62b1968cf Change string throws to Error throws (no stack trace with former) 2018-07-25 17:45:06 +03:00
Dan Stillman
2652fac24b Throw translation error on >=400 status code for doGet()/doPost()
Previously the handler would be called even on error pages, which often
meant that an import translator (e.g., BibTeX) would fail to find
anything on the page and the save popup would just close silently. The
popup will now show an error message as soon as the error occurs.
2018-07-23 07:05:44 -04:00
Sylvester Keil
dfe412d448
Remove legacy iterator protocol use in Zotero.DB 2018-07-19 12:21:01 +02:00
Dan Stillman
1f8cf22a35 Delete group files when deleting group
Fixes #1532
2018-07-15 23:09:34 -04:00
Dan Stillman
8390f30be9 Update citeproc-js to 1.1.210 2018-07-15 09:09:23 -04:00
Sylvester Keil
9737a4d974
Initialize Dates in Electron 2018-07-13 14:03:23 +02:00
Adomas Venčkauskas
28535ac64d Throw proper error in Prefs.get(). Use double-quotes in default prefs 2018-07-11 19:35:14 +03:00
Adomas Venčkauskas
856bbf49ab Remove redundant QueryInterface for PrefBranch2 2018-07-11 16:44:04 +03:00
Adomas Venčkauskas
cb8280f0ff Move Zotero.Prefs into its own file 2018-07-11 15:53:46 +03:00
Sylvester Keil
53522c2cbe Add Zotero.File.getResource for local resources
Use `getResource` in Zotero.Date.init (this turns it into a
synchronous function). Zotero.File.getResource makes it easier
to load local files on platforms that do not support the
`resource://` URLs.
2018-07-11 14:08:45 +03:00
Sylvester Keil
06cb9aff98 Prefer use of Services.appinfo 2018-07-11 13:58:41 +03:00
Sylvester Keil
b2a1977d5a Gecko 16 and later sets topic to 'active' 2018-07-11 13:58:41 +03:00
Sylvester Keil
e406ddc54b Prefer Services.jsm getters for common services 2018-07-11 13:58:41 +03:00
Sylvester Keil
3f026239f1 Remove unused methods
`.getAsyncStatement()` and `.executeAsyncStatement()` are not used
anywhere.
2018-07-11 13:58:41 +03:00
Dan Stillman
99a1ac62c7 Fix a couple cases of open item box field changes not being saved
When clicking directly from an open field to the creator delete button
or a creator type in the creator type menu
2018-07-09 14:54:50 +02:00
Dan Stillman
62e403bc55 Limit OA PDF lookup to dev builds 2018-07-07 13:12:33 +02:00
Brenton M. Wiernik
baafd49b8c Update link to import formats documentation (#1525)
The existing destination has never really been the appropriate page for this dialogue, as it has never actual listed the supported formats.
2018-07-04 21:09:45 +02:00
Dan Stillman
da9537c5e4 Mendeley import: Properly handle "Mendeley Ltd." folder with period
At some point Mendeley seems to have changed the default path to the
data directory on Windows to remove the period, and for people with the
old directory we were linking rather than storing attachment files from
"Downloaded".
2018-07-02 01:06:17 +02:00
Adomas Venčkauskas
c1a9735081 Fix citations updating even after user asks for them not to be updated 2018-07-01 14:23:49 +03:00
Dan Stillman
99584dc918 Import base-directory-relative linked files
Zotero RDF contained 'attachments:' paths when files weren't included
but they weren't imported properly
2018-06-30 09:19:09 +02:00
Brenton M. Wiernik
b4ee2fbd98 Add support for bear:// URI protocol (#1520)
https://forums.zotero.org/discussion/comment/310717#Comment_310717
2018-06-29 02:27:19 +02:00
Adomas Venčkauskas
5425c272b2 Update translation-server-v2 dependent files to be commonJS compatible
Also some minor misc changes to translate.js
2018-06-27 17:31:10 +03:00
Dan Stillman
3aefcdf2d9 Mendeley import: Import folders with parentId of 0 instead of -1
Top-level folders should have a parentId of -1, but at least in one user
report there's a folder with 0.
2018-06-27 02:19:00 +02:00
Dan Stillman
f715c02a66 Throw proper error when collection is moved into one of its descendents 2018-06-26 21:06:11 +02:00
Dan Stillman
3c1cdd57d2 Fix error with nested saved searches containing post-search filters
Addresses #1518
2018-06-26 00:48:08 +02:00
Dan Stillman
cbf63f4e72 Skip blank tags when importing from Mendeley 2018-06-24 20:04:37 +02:00
Dan Stillman
7785b8df14 Restore open-access PDF retrieval for Add Item by Identifier
This reverts commit faa310e0f0.
2018-06-20 17:19:39 -04:00
Dan Stillman
faa310e0f0 Temporarily disable Add Item by Identifier open-access PDF retrieval
Not going out in 5.0.52
2018-06-20 17:07:11 -04:00
Dan Stillman
a9cd47dc3f Avoid hang on invalid attachment during import
E.g., if a .pdf is really an HTML file, we try to load it in a hidden
browser (because we properly detect the content type), but then the .pdf
extension causes the hidden browser to launch it via the OS and the
hidden browser never finishes loading it. This adds a 5-second timeout
to abort the process.
2018-06-19 18:26:34 -04:00
Dan Stillman
2d6332212c Make attachment post-process failures non-fatal
This is generally just detecting the character set and/or indexing, so
it's not the end of the world if it fails.
2018-06-19 18:25:53 -04:00
Dan Stillman
2a7f31813e Disable JS in hidden browser when indexing HTML files without a charset
This could cause imports that linked to HTML files to hang, possibly
from network requests that failed.
2018-06-18 20:19:02 -04:00
Dan Stillman
9b9fe098d9 Fix display of error on corrupted DB 2018-06-17 18:08:33 -04:00
Dan Stillman
4554998798 Add Zotero.File.moveToUnique(), replacing Zotero.moveToUnique()
New function takes string paths instead of nsIFile instances
2018-06-17 18:08:03 -04:00
Dan Stillman
f3a62f5a63 Automatically download open-access PDFs via Add Item by Identifier
When the associated-files pref is enabled, Add Item by Identifier uses a
Zotero Unpaywall mirror to find available open-access PDFs. No details
about the contents of searches are logged.
2018-06-16 14:35:44 -04:00
Dan Stillman
72481b072e Clean up extra files from Mendeley imports
Follow-up to cdee741a6
2018-06-16 03:07:46 -04:00
Dan Stillman
cdee741a6d Mendeley import: Fix duplicate PDF copying for PDFs in Downloaded
For each PDF with an associated URL in the Downloaded directory, we were
copying all files in the directory (!) to the attachment's storage
directory. (Zotero imports always have files in separate directories,
and this was a function used to save both single files and HTML
snapshots.)

We'll clean up the extra files in a separate step.
2018-06-16 01:51:37 -04:00
Dan Stillman
0383f104dd Fix "Import into new collection" option when handling importable file 2018-06-14 16:41:33 -04:00
Dan Stillman
d38d55e2b4 Mendeley import: Don't use single transaction 2018-06-13 10:27:26 -04:00
Dan Stillman
5ddbe433b9 Fix Backoff and Retry-After header parsing 2018-06-12 15:17:46 -04:00
Dan Stillman
d4c201fbc0 Correct Mendeley data dir path on Windows 2018-06-12 07:04:32 -04:00
Dan Stillman
e67db436c6 Mendeley import: Add link to support page on encrypted DB
Also localize the name of the collection created during import
2018-06-12 06:16:59 -04:00
Dan Stillman
cfbb3d3d47 Mendeley import: Fix "Collection undefined not found" error
If a folder appeared in the DocumentFolders table but not in
Folders/RemoteFolders, it caused an error.
2018-06-11 12:16:09 -04:00
Dan Stillman
bd5f2525dc Add star tag for favorite items 2018-06-11 09:44:01 -04:00
Dan Stillman
2831e8be3e Mendeley import: Fix "keys is undefined" error
An error would occur if a document existed in RemoteDocuments but not
Documents.
2018-06-10 02:50:52 -04:00
Dan Stillman
8547dedca9 Add debug output for unrecognized Mendeley DB issue 2018-06-09 02:26:12 -04:00
Dan Stillman
46fc28dc16 Remove unused conditional 2018-06-09 02:13:00 -04:00
Dan Stillman
06a819bf34 Fix Import from Clipboard breakage 2018-06-09 02:09:46 -04:00
Dan Stillman
3c70dee5c1 Create import collection by default if there are non-trash items
Follow-up to 7494e4d88c, which only checked for collections
2018-06-08 05:57:20 -04:00
Dan Stillman
337144a5eb Mendeley import: Reuse existing collections
If "Place imported collections and items into new collection" is
unchecked, previously imported collections will be reused when they're
in the right place in the hierarchy rather than creating new ones.
2018-06-08 04:52:18 -04:00
Dan Stillman
7494e4d88c Don't create import collection by default if no collections in library
If the selected library doesn't have collections, "Place imported
collections and items into new collection" will be unchecked in the
import wizard.
2018-06-08 02:44:13 -04:00
Dan Stillman
2b49e94a61 Fix sync breakage from 78e87a351 2018-06-07 17:18:28 -04:00
Dan Stillman
854cb05883 Mendeley import: Support repeated imports without duplicating items
Existing items are overwritten with new data (or skipped in the case of
file attachments).

Addresses #1451
2018-06-07 06:37:50 -04:00
Dan Stillman
d98e89cf26 Add import option for creating a new collection
Use the new wizard for all imports (even if no Mendeley DB), and add a
page with a "Place imported collections and items into new collection"
option. If deselected, collections are added to the library root.
2018-06-05 22:39:23 -04:00
Dan Stillman
4008848c64 Mendeley import: Create imported_url attachments when possible
If there's a single PDF file and a single PDF URL and the file exists,
make an imported_url attachment instead of separate file and linked_url
attachments.

Addresses #1451
2018-06-05 19:57:28 -04:00
Dan Stillman
024b428fe6 Mendeley import: Add "Other…" button for selecting another DB
E.g., for selecting a DB from a backup

Addresses #1451
2018-06-05 19:57:20 -04:00
Dan Stillman
17f2025c53 Fix import into selected library from wizard 2018-06-05 08:39:17 -04:00
Dan Stillman
6ac65373a3 Mendeley import: Look for Downloaded files relative to DB if not found
Addresses #1451
2018-06-05 08:30:14 -04:00
Dan Stillman
2939b3ae95 Merge branch 'mendeley-import' (#1451) 2018-06-05 07:52:12 -04:00
Dan Stillman
0f4e5ef508 Mendeley import
Accept Mendeley SQLite databases via File → Import… and perform a
direct import, including collections, timestamps, notes, attachments,
and extracted annotations.

When a Mendeley database is present, File → Import… shows a wizard that
lets you choose between a file and Mendeley for the source, and choosing
the latter shows a list of available databases in the Mendeley data
directory.

Known fields that aren't valid for a type are stored in Extra.

Files in the Mendeley 'Downloaded' folder are stored. Files elsewhere
are linked.
2018-06-05 07:52:01 -04:00
Dan Stillman
c5ba2e55f3 Add a debug line before submitting output
This will help clarify whether the person waited to submit or just
submitted while stuff was happening.
2018-06-04 19:58:10 -04:00
Dan Stillman
f7e411d561 Add support for databases in other directories
Previously you could use Zotero.DBConnection to open another database in
the data directory, but not one stored elsewhere in the filesystem. This
allows an absolute path to be passed instead. Various operations
(backups, corrupt DB recovery, pragma commands) are disabled for
external databases.
2018-06-02 04:10:39 -04:00
Dan Stillman
603388c79d Add missing quote to error 2018-06-02 02:27:24 -04:00
Adomas Venčkauskas
547f1c29e3 Integration: Fix non-existent session errors preventing displayError from working 2018-06-01 23:26:24 +03:00
Dan Stillman
9220b2d9c2 Fix inconsequential bug in Zotero.MIME.sniffForMIMEType()
`undefined` was being passed as an argument to slice(), but 0 is the
only offset that's used anyway, and that's what happens if you pass
`undefined`.
2018-05-27 21:23:38 -04:00
Dan Stillman
7271fdf6b7 Add Zotero.Sync.Runner.delayIndefinite()
Delays syncing until the returned function is run
2018-05-27 21:23:38 -04:00