Commit graph

481 commits

Author SHA1 Message Date
Dan Stillman
6994c63a0e Tweak recognizePDF fix, add test coverage, and enable manual tests
Still disabled in Travis until we mock the remote services
2016-12-17 08:05:33 -05:00
Dan Stillman
7447525d75 Remove test for moving existing target dir during data dir migration
Functionality removed in 0be67dbda5
2016-12-14 03:36:35 -05:00
Adomas Ven
d6d4e4b33e Saner feed database management (#1131)
* Mark feedItems read in a single batch SQL update
* Automatically remove old feed items
* User-facing preference globally and per-feed for feed item expiration
2016-12-13 09:07:43 -05:00
Dan Stillman
7fc352b9b7 Fix "Add Item by Identifier" (broken by 78b1d2ee3)
And make lookup tests run when not on Travis, though we should still mock the
HTTP requests.
2016-12-13 05:23:21 -05:00
Adomas Ven
747c11c917 Improves proxy support (#1129)
Improves proxy support

- Automatically detect and dehyphenise https proxies which use EZProxy
  HttpsHyphens
- Web translators now pass around Zotero.Proxy instances which can
  proxify/deproxify urls passed to `translate.setLocation()` before calling
  `translate.getTranslators()`/ translate.detect()`. The proxy passing is
  done within connector background/injected processes and between
  standalone and connectors.
- Proxy protocol unified with connectors. Connectors can now pass
  proxies to `/connector/save_items`. The proxies will be used to resolve
  true item and attachment urls when saving.

Closes zotero/zotero#578, zotero/zotero#721

Relevant zotero/zotero#34, zotero/zotero#556
2016-12-12 14:29:59 +02:00
Dan Stillman
80f888f374 Fix replacement of existing item storage directories
- Make Zotero.Attachments.createDirectoryForItem() delete existing
  directory instead of moving it to orphaned-files; also now returns a
  string path instead of an nsIFile
- Use above function during file sync instead of
  _deleteExistingAttachmentFiles(), which was partly broken
- Fix throwing on errors when saving some attachment types
2016-12-12 04:06:01 -05:00
Dan Stillman
85776a99b2 Fix test 2016-12-11 02:55:47 -05:00
Dan Stillman
78b1d2ee35 Import translation improvements
- Don't block the UI with a progress meter during imports. Instead, show
  a popup in the bottom right when the import is done that shows how
  many items were saved.
- Fix hang when importing some files
- Fix various problems with asynchronous operations/transactions
    - Use the save queue for imports instead of creating concurrent
      transactions that can time out
    - Wait for the save to finish before returning from the translate()
      promise. All save modes now use the save queue, so code that
      handled the non-save-queue process can probably be removed.
    - Serialize child attachments instead of running them concurrently.
      This might make multi-attachment saves a little slower, since they
      can't download at the same time, but it avoids problems with
      concurrent transactions. We might be able to improve this to allow
      concurrent downloads, or allow concurrent saves for a limited
      number of items (e.g., from web saving) if not for larger imports.
- Change collection handling during import, since UI is now active
  - Select the root collection at the beginning of the import
  - Assign items and collections to the root during the import instead
    of at the end
  - Don't select other collections
- Change a few ItemSaver functions to use promises and remove
  unnecessary callbacks. (This includes some connector code that needs
  to be tested.)
- Change some `parentID` variables in ItemSaver to `parentItemID` for
  clarity, since collections are now handled in more places

To-do:

- Save items in smaller batches instead of doing all in the same
  transaction
- Show progress meter in a bottom-right popup during the import
2016-12-11 02:30:34 -05:00
Dan Stillman
c61a9dc5f3 Localized plural form support for Zotero.getString()
Zotero.getString() now takes a third parameter, `num` (which should also
appear in `params`) to use when determining which plural form of the
string to use. Localized strings should include all forms in the order
specified in [1], separated by semicolons.

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_and_Plurals
2016-12-11 02:30:34 -05:00
Dan Stillman
9fbd84b27e Move ItemGetter test to correct section 2016-12-09 06:30:38 -05:00
Dan Stillman
fbd21bae16 Don't update feeds after syncs during tests
This reverts commit 60befe52e4 and adds a
better fix that leaves the notifier event in place. Feeds just don't
need to update after syncs during tests.
2016-12-08 05:58:46 -05:00
Adomas Venčkauskas
60befe52e4 Remove database closed errors in sync tests 2016-12-08 12:16:44 +02:00
Dan Stillman
2b9ef26c61 Fix renaming linked attachment files 2016-12-08 03:57:49 -05:00
Dan Stillman
b5bc18c7ed Add new init(options) signature for server endpoints
An endpoint can now take a single object containing 'method', 'pathname',
'query', 'headers', and 'data' and return an integer, an array containing
[statusCode, contentType, body], or a promise for either. This allows the
handlers to use the HTTP method and headers and removes the need for callbacks
when some handlers already use coroutine().

If init() returns a promise, it now has to use the new single-parameter
signature (because the check is done with Function.length, and combining
promises and callbacks doesn't make sense anyway).
2016-12-05 02:55:32 -05:00
Dan Stillman
9541a6cfcd Fix connector server test after 6900a5cc 2016-11-30 22:40:54 -05:00
Adomas Venčkauskas
45c944e731 Fix server_connector tests broken in 69ab4b0b
Don't display error prompt in standalone when style install fails.
Fixes an import bug when import input is invalid.
2016-11-30 14:58:43 +02:00
Dan Stillman
1b67ed071e Skip auto data dir migration if target dir exists and is non-empty 2016-11-30 02:09:07 -05:00
Adomas Ven
69ab4b0b1d Add server_connector endpoint to import styles and import translatable resources (#1120) 2016-11-29 14:59:58 -05:00
Dan Stillman
ec7fe773c3 Disable a test that fails too often on Travis
If someone can manage to reproduce this locally, we can try a real fix:

✖ [FAIL] should show tags in alphabetical order
  AssertionError: expected '' to equal 'A, B, C' at
        @resource://zotero-unit-tests/noteeditorTest.js:61:4
2016-11-29 04:36:59 -05:00
Dan Stillman
a114309625 Note editor test tweaks, which may or may not fix Travis failures 2016-11-29 04:26:58 -05:00
Dan Stillman
8f0ed56ae2 Test fixes after 5bceebf56c 2016-11-29 03:52:07 -05:00
Dan Stillman
8799bb4ebd Remove Travis debugging 2016-11-29 03:29:45 -05:00
Dan Stillman
30d0858ebf Travis debugging 2016-11-29 01:42:15 -05:00
Dan Stillman
fa859b45fe Fix running of syncLocal tests 2016-11-28 19:26:39 -05:00
Dan Stillman
a735d5dd41 Remove Travis debugging 2016-11-27 20:02:53 -05:00
Dan Stillman
837ae0444d Travis debugging 2016-11-27 19:51:48 -05:00
Dan Stillman
3a9cd31d42 Travis debugging 2016-11-27 19:46:12 -05:00
Dan Stillman
5a6f1eef63 Update deprecated uses of Zotero.getZoteroDirectory()/getZoteroDatabase()
The Zotero.DataDirectory equivalents return string paths instead of nsIFile
instances, so some of these calls now just use Zotero.File.pathToFile(), which
can be removed when the surrounding code is updated to OS.File,
2016-11-27 00:17:56 -05:00
Dan Stillman
4c0abb6816 Point other profiles to new data dir after migration (+ code reorg)
Look for other profiles, from both apps (Firefox and Standalone), that
point to the data directory being migrated and update prefs.js in those
profiles to point to the new location.

Also reorganize code into Zotero.Profile and Zotero.DataDirectory
namespaces
2016-11-27 00:17:56 -05:00
Dan Stillman
412a7990b1 Don't perform automatic migration if there's an existing pipe
This prevents us from moving the data directory if the other app
(Firefox or Standalone) is running from the same directory.

Also clean up stub code in migration tests
2016-11-26 23:09:24 -05:00
Dan Stillman
acb1be97d0 More Travis debugging 2016-11-23 00:11:01 -05:00
Dan Stillman
84bb61ab7b Fix test timeouts caused by Quick Copy initialization
When an export translator is selected for Quick Copy, Quick Copy
initialization triggers translator initialization a few seconds after
startup, because the translator code needs to be available synchronously
for drag/drop. A Quick Copy test was changing the setting to BibTeX,
which was resulting in random timeouts after subsequent resetDB() calls
due to slow translator loading. This change skips initialization in test
mode. This might actually fix a lot of timeouts on Travis in the second
half of the tests...

This also resets the Quick Copy pref in those tests so that it's left at
the default, though really we should automatically reset all prefs after
all test groups and in resetDB().
2016-11-22 20:31:58 -05:00
Dan Stillman
935d48013b Travis debugging 2016-11-22 19:35:30 -05:00
Dan Stillman
c7700f2b96 Fix warning in translate test 2016-11-22 19:06:47 -05:00
Dan Stillman
ef3e098586 Migrate data directory automatically on macOS and Linux
If data directory is within the profile directory and we can move the
subdirectories instantaneously with /bin/mv, just do it silently at startup.
2016-11-22 02:33:05 -05:00
Dan Stillman
288d0c7c06 Create 'profile' and 'Zotero' directories in temp dir for tests 2016-11-22 01:45:07 -05:00
Dan Stillman
98e3a9addf Allow immediate retry on partial data dir migration failure
A common cause of this on Windows will likely be open attachment files,
so suggest closing those and make retrying the default action.
2016-11-21 05:19:18 -05:00
Dan Stillman
ce5328cbc4 Don't trigger translator loading in download-on-demand test
This was causing later tests to time out.
2016-11-20 02:46:04 -05:00
Dan Stillman
6c6a775be2 Fix migration tests in Fx45
`arguments` wasn't iterable until 46
2016-11-20 01:54:09 -05:00
Dan Stillman
249521f12b Enable debugging for Travis error 2016-11-20 01:40:39 -05:00
Dan Stillman
6d07cb44df Fix Travis errors due to broken DirectoryIterator 2016-11-19 18:52:10 -05:00
Dan Stillman
79700969e1 Data directory migration
This adds a new button to the Advanced prefs to migrate the data directory to
$HOME/Zotero. The button only appears if the data directory is set to the
default location within a profile directory (including the other program from
the one running, even though that's technically stored as a custom data
directory).

On Mac/Linux, directories within the data directory are moved with /bin/mv.  On
Windows, or if that fails, they're copied recursively using OS.File.move()
(which annoyingly doesn't reliably support directory moving). The former should
be instantaneous on most systems (unless the data directory or 'storage' were
on a different filesystem from $HOME).

If the database fails to transfer, migration fails and the data directory
setting remains on the old directory. If the database transfers but other files
fail, the data directory setting is updated. In both cases, the user is
encouraged to migrate remaining files manually with a button that reveals the
directories and quits the program.

This isn't yet tested on Linux or Windows, and migration isn't yet suggested
automatically.

Adds Zotero.File.reveal(), Zotero.File.directoryIsEmpty(), and
Zotero.File.moveDirectory().
2016-11-13 04:55:51 -05:00
Dan Stillman
341d3d69b3 Fix export failure on missing file attachments
This relies on synchronous file access, but making it async would require
rewriting all export translators that save files.
2016-10-25 02:36:13 -04:00
Dan Stillman
79baac3158 Fix relinking of imported attachment with external file 2016-10-22 15:11:12 -04:00
Dan Stillman
0016c5ecf8 Fix editing of saved search in a group 2016-10-20 22:55:42 -04:00
Dan Stillman
bc47b6b171 Reset Collection search condition menu on library change
Reported here:
https://github.com/zotero/zotero/issues/145#issuecomment-252678872
2016-10-11 23:55:17 -04:00
Dan Stillman
9c52ebdf8b Show saved searches under "Collection" search condition
With icons to identify collections and searches

Also:

- `savedSearch` search condition in general
- Clean up some search window code
- Reorganize search tests
2016-10-06 01:17:06 -04:00
Dan Stillman
eb1aa43da2 Remove test for Extra field parsing after 30f7ef093 2016-09-28 11:45:18 -04:00
Dan Stillman
753252be77 Merge pull request #1090 from adomasven/feature/iframe-translator-rules
Changes for #1021 to support targetAll translator property.
2016-09-28 11:42:58 -04:00
Adomas Venčkauskas
5d17ff3f53 Implements #1021 on Z4fx extension 2016-09-27 17:43:56 +03:00