Commit graph

25 commits

Author SHA1 Message Date
Abe Jellinek
1f599283df
Fix indexing files with text content types that Firefox won't display (#3708) 2024-02-19 05:11:16 -05:00
Dan Stillman
a497f501e9 Update pdf-worker and update full-text test
Strip '\f' at end of extracted text and ignore '\n' in the middle of
text from the test PDF
2023-04-29 04:56:28 -04:00
Dan Stillman
43ba4c3620 Don't check for .zotero-ft-info in test
No longer created
2023-04-27 03:48:49 -04:00
Dan Stillman
f95a6d0f0a Rename Zotero.Fulltext tests to Zotero.FullText 2023-04-27 03:48:01 -04:00
Dan Stillman
76a1535a60 Full-text indexing improvements
- Use full-text cache file from syncing if available when reindexing via
  info pane or Rebuild Index → Index Unindexed Items. Only discard it for
  full index rebuild. This allows Index Unindexed Items to be used to
  force immediate processing of queued content from syncing and avoids
  unnecessary syncing back of identical content. Previously, the cache
  file was used for a manual index only when the local file didn't exist.
- When rebuilding index, don't clear indexed items with missing local
  file that are missing stats due to a pre-411180ef bug.
- indexItems() now takes an 'options' object as its second parameter
- Minor code cleanup
2020-03-09 01:19:52 -04:00
Dan Stillman
411180ef83 Full-text indexing fixes
- Don't clear item's index stats (and show "Unknown") when an item is
  reindexed remotely and the content matches the local content
- Always update an item's state and its stats in the same query, to
  avoid incorrect feedback immediately after indexing
- Clean up `setItemContent()` tests
2020-03-06 03:11:16 -05:00
Martynas Bagdonas
d810deaa9c Update PDF tools version and fix fulltext test 2018-02-14 04:00:02 -05:00
Dan Stillman
45ddf9827c Reset PDF tools path for tests in resetDB()
And include path on error when running PDF tool
2018-02-07 04:04:37 -05:00
Dan Stillman
db2ddfd493 Fix Zotero.Fulltext.setItemContent() test 2018-01-18 18:55:24 -05:00
Martynas Bagdonas
723b4d32e5 Fix Zotero.Fulltext tests 2018-01-18 18:54:54 -05:00
Martynas Bagdonas
0d5ea8520a Use the bundled PDF tools 2018-01-17 13:33:18 +02:00
Dan Stillman
c110e64293 Consider fulltextItems with SYNC_STATE_MISSING as unindexed
The items will still match full-text word searches, but they won't match
phrase searches (because those require cache files for non-text
attachments) and the full-text won't sync to other computers, so they
should really be reindexed.
2017-07-26 22:16:01 -04:00
Dan Stillman
67ccb632b4 Don't keep looking for unsynced full-text content that isn't available
This can happen if cache files are deleted for PDF attachments or if
text files are missing.
2017-07-26 22:16:01 -04:00
Dan Stillman
d527c340c6 Fix "text is not defined" error during full-text content sync 2017-04-26 02:44:01 -04:00
Dan Stillman
5c50bb00cf Don't save full-text cache files for linked files to linked directory
Regression from 80f888f374. Not entirely sure what I was trying to fix there.
2017-02-20 17:51:37 -05:00
Dan Stillman
5a6f1eef63 Update deprecated uses of Zotero.getZoteroDirectory()/getZoteroDatabase()
The Zotero.DataDirectory equivalents return string paths instead of nsIFile
instances, so some of these calls now just use Zotero.File.pathToFile(), which
can be removed when the surrounding code is updated to OS.File,
2016-11-27 00:17:56 -05:00
Dan Stillman
d857a06661 Use OS.File for file reads in Zotero.File.get(Binary)ContentsAsync()
This is the recommended approach (since NetUtil can still do some main-thread
I/O for files) and avoids warnings in the console.

For getContentsAsync(), also sends nsIURIs and string URIs to
Zotero.HTTP.request(), which should be used instead.

This makes getBinaryContentsAsync() much slower (due to the conversion from an
array of bytes to a binary string), but it's only used in tests. For one test
that compares two large files, use MD5 instead.
2016-09-24 06:42:30 -04:00
Dan Stillman
10181f7f56 Fix sync error on missing full-text
If a version is returned for an item's full-text content but a 404 is returned
for the content itself (because it's missing in Elasticsearch for some reason),
don't throw an error.

Also remove legacy array comprehensions in fulltext and syncFullTextEngine test
files, which apparently weren't being run.
2016-04-30 01:24:33 -04:00
Dan Stillman
a20a6c86bc Closes #899, Setting max chars to 0 should disable full-text indexing 2016-02-07 01:01:26 -05:00
Dan Stillman
e3033b056e Fix intermittent error in FullText.getUnsyncedContent() test 2015-12-09 04:51:48 -05:00
Dan Stillman
62aeb1da32 Full-text syncing support via API [DB reupgrade] 2015-11-12 02:54:51 -05:00
Aurimas Vinckevicius
d122323dbf Fix fullText tests on Windows 2015-09-21 17:08:22 -05:00
Dan Stillman
5ba344516e Update PDF tool handling in tests
The test runner now downloads and caches the PDF tools for the current
platform within the test data directory and only redownloads them when
out of date, and it updates the download URL so that the full-text code
pulls from the cache directory via a file:// URL.

The installPDFTools() support function now installs the files directly
instead of going through the prefs, and a new uninstallPDFTools()
function removes the tools. Since the presence of the PDF tools can
affect other tests, tests that need the tools should install them in a
before() and uninstall them in an after(), leaving most tests to run
without PDF indexing.

This also adds a callback to the waitForWindow() support function. If a
modal dialog is opened, it blocks the next promise handler from running,
so a callback has to be used to interact with and close the dialog
immediately.
2015-05-31 23:50:26 -04:00
Dan Stillman
8a0b4d92a6 Fix PDF tools installation on Linux 2015-05-26 04:44:02 -04:00
Dan Stillman
ebe41ac51a Fixes #724, PDF indexing binaries not downloading
And adds Zotero.File.download(uri, path)
2015-05-23 18:03:25 -04:00