And use them in new importTextAttachment() and importHTMLAttachment()
test support functions. These can be used to avoid needing a hidden
browser for determining the character set of the imported text
documents.
This is useful when trying to debug an error that only happens after a
number of other tests have run -- specify -e and run tests from either
an earlier file with -s or from the beginning.
* Use mocha, chai & sinon from the npm. As of sinon 2.0
sinon-as-promised is no longer required so it is removed
* Tweak code to re-use the same loader with the same environment
throghout the code
* Introduce browserify step for testing tools that only provide
node-compatible libraries (sinon, chai-as-promised)
* Introduce copy step for test data to resolve multiple issues with
tests depending on files not being symlinks
* Re-introduce custom implementation of setTimeout to resolve issues
with few tests
* Re-introduce custom Bluebird Promises config & monkey patch
The web library will probably still display the old tag in addition to
the new one, at least until browser restart. We'll have to deal with
that separately.
Closes#1205
Changes `libraryTreeView::addEventListener('load')` and similar to
`libraryTreeView::onLoad.addListener(listener, once)`, etc. `once` is an
optional boolean that, when true, causes the listener to fire once and
then be removed. This is implicit for 'load'.
'load' maintains its special behavior of running immediately if the
treeview has already been loaded.
Also adds `waitForLoad()` and `waitForSelect()` functions that return
promises on event completion, since most uses of those events were just
resolving deferreds.
This reverts c6b78da69d, which changed it to expect numbers when I
noticed the type being undefined in debug output, but apparently the
only tests where the type actually mattered passed it as a name.
At some point we should just change all tests to pass as a name.
* Mark feedItems read in a single batch SQL update
* Automatically remove old feed items
* User-facing preference globally and per-feed for feed item expiration
Improves proxy support
- Automatically detect and dehyphenise https proxies which use EZProxy
HttpsHyphens
- Web translators now pass around Zotero.Proxy instances which can
proxify/deproxify urls passed to `translate.setLocation()` before calling
`translate.getTranslators()`/ translate.detect()`. The proxy passing is
done within connector background/injected processes and between
standalone and connectors.
- Proxy protocol unified with connectors. Connectors can now pass
proxies to `/connector/save_items`. The proxies will be used to resolve
true item and attachment urls when saving.
Closeszotero/zotero#578, zotero/zotero#721
Relevant zotero/zotero#34, zotero/zotero#556
The Zotero.DataDirectory equivalents return string paths instead of nsIFile
instances, so some of these calls now just use Zotero.File.pathToFile(), which
can be removed when the surrounding code is updated to OS.File,
Look for other profiles, from both apps (Firefox and Standalone), that
point to the data directory being migrated and update prefs.js in those
profiles to point to the new location.
Also reorganize code into Zotero.Profile and Zotero.DataDirectory
namespaces
And don't skip alerts in Zotero.alert() during automated tests. (That
was intended to avoid long timeouts after unexpected failures, but,
e.g., PDF metadata lookups (which are currently disabled in automated
tests) should just be mocked so they don't intermittently fail.)
server_connectorTest - alternating port prevents
exceptions not catchable in JS for httpd server when the socket
and the port remains open after httpd.stop() callback
support - changes the window on which `setTimeout()` is ran in dialogs.
If the timeout is ran on the main window the `dialog` object appears to
lose certain properties and not respond to interactions completely.
Items in a library are now loaded only when a library is clicked on and
at sync time. There might be some other areas where they need to be
loaded or where this causes problems (e.g., drag and drop, word
processor integration).
- Hide notes, tags and related for feed items in itembox
- Add feed support for <enclosure> elements
- Add feed syncing methods for synced settings (additional work is
needed on the sync architecture to download synced settings from the
server)
- Change feed item clear policy to be less aggressive
- Adjust for deasyncification
- Disable translate-on-select
- Closeadomasven/zotero#7, Remove context menu items from feeds
While trying to get translation and citing working with asynchronously
generated data, we realized that drag-and-drop support was going to
be...problematic. Firefox only supports synchronous methods for
providing drag data (unlike, it seems, the DataTransferItem interface
supported by Chrome), which means that we'd need to preload all relevant
data on item selection (bounded by export.quickCopy.dragLimit) and keep
the translate/cite methods synchronous (or maintain two separate
versions).
What we're trying instead is doing what I said in #518 we weren't going
to do: loading most object data on startup and leaving many more
functions synchronous. Essentially, this takes the various load*()
methods described in #518, moves them to startup, and makes them operate
on entire libraries rather than individual objects.
The obvious downside here (other than undoing much of the work of the
last many months) is that it increases startup time, potentially quite a
lot for larger libraries. On my laptop, with a 3,000-item library, this
adds about 3 seconds to startup time. I haven't yet tested with larger
libraries. But I'm hoping that we can optimize this further to reduce
that delay. Among other things, this is loading data for all libraries,
when it should be able to load data only for the library being viewed.
But this is also fundamentally just doing some SELECT queries and
storing the results, so it really shouldn't need to be that slow (though
performance may be bounded a bit here by XPCOM overhead).
If we can make this fast enough, it means that third-party plugins
should be able to remain much closer to their current designs. (Some
things, including saving, will still need to be made asynchronous.)
Also:
- Remove last-sync-time mechanism for both WebDAV and ZFS, since it can
be determined by storage properties (mtime/md5) in data sync
- Add option to include synced storage properties in item toJSON()
instead of local file properties
- Set "Fake-Server-Match" header in setHTTPResponse() test support
function, which can be used for request count assertions -- see
resetRequestCount() and assertRequestCount() in webdavTest.js
- Allow string (e.g., 'to_download') instead of constant in
Zotero.Sync.Data.Local.setSyncState()
- Misc storage tweaks
Improves UX of sync authentication.
The account is now linked and unlinked and an API key related to
the client is generated transparently in the background.
The API key is deleted on unlinking.
No sync options are allowed before linking an account.
Closes#864
This adds a 'channel' property to Zotero.HTTP.UnexpectedStatusException,
because the 'channel' property of the XHR can be garbage-collected
before handling, and the channel's 'securityInfo' property is necessary
to detect certificate errors.
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
This will appear much less frequently, since non-conflicting field changes on
both sides can be resolved automatically, but genuine field conflicts still
require manual conflict resolution.
The merge pane is no longer editable, since the itembox code to do that is
async and can't run in a modal window, but it's not really necessary,
particularly with conflicts happening less frequently.
TODO:
- Remote item deletions
- File conflicts
- Maybe handle some edge cases where the conflicted items fail to save
Save uploaded data to cache, and update local object if necessary (which
it mostly shouldn't be except for invalid characters and HTML filtering
in notes)
Also add some upload and JSON tests
There's a lot more to do, and this isn't ready for actual usage, but the
basic functionality is mostly in place and has decent test coverage. It
can successfully upgrade a library last used with classic syncing and
pull down changes via the API. Uploading mostly works but is currently
disabled for safety until it has better test coverage.
Downloaded JSON is first saved to a cache table, which is then used to
populate other tables and later for generating PATCH requests and
automatically resolving conflicts (since it shows what was changed
locally and what was changed remotely). Objects with unmet dependencies
or unknown fields are skipped for now but don't block the rest of the
sync.
Some of the bigger remaining to-dos:
- Tests for uploading
- Re-do the preferences to get an API key
- File sync integration
- Full-text syncing integration
- Manual conflict resolution (though this already includes much smarter
conflict handling that automatically resolves many conflicts)
And use it in resetDB() test support function, mainly to allow
skipBundledFiles for resetDB calls. Translator installation and
initialization can take a long time, but tests that need a clean DB
don't necessarily rely on translators. Without this, running resetDB()
in beforeEach() for many tests is prohibitively slow.
This uses ISO 8601 dates for generateAllTypesAndFieldsData (and
changes populateDBWithSampleData to use Item#fromJSON), and makes
translators expect ISO 8601 accessDates, although SQL accessDates are
still supported with a deprecation warning. Canonicalization happens in
Zotero.Translate, so I need to remember to update connectors as well.
And add group.fromJSON(json, userID), which sets editable and
filesEditable properties based on the group JSON (libraryReading, role
lists, etc.) and the given user
Waits for an alert or confirmation dialog to open and closes it
automatically, optionally after running onOpen(dialog) to check its
contents (e.g., with dialog.document.documentElement.textContent) and
optionally clicking a button other than 'accept' (e.g., 'cancel',
extra1').
Supports delayed accept buttons
And simplify tree view load event handling, which may or may not have
been contributing to intermittent test failures, but is cleaner this way
regardless.
Relations are now properties of collections and items rather than
first-class objects, stored in separate collectionRelations and
itemRelations tables with ids for subjects, with foreign keys to the
associated data objects.
Related items now use dc:relation relations rather than a separate table
(among other reasons, because API syncing won't necessarily sync both
items at the same time, so they can't be stored by id).
The UI assigns related-item relations bidirectionally, and checks for
related-item and linked-object relations are done unidirectionally by
default.
dc:isReplacedBy is now dc:replaces, so that the subject is an existing
object, and the predicate is now named
Zotero.Attachments.replacedItemPredicate.
Some additional work is still needed, notably around following
replaced-item relations, and migration needs to be tested more fully,
but this seems to mostly work.
getGroup() can be used to access a default group library for general
group tests. createGroup() can be used to create one for a particular
test or set of tests.
The test runner now downloads and caches the PDF tools for the current
platform within the test data directory and only redownloads them when
out of date, and it updates the download URL so that the full-text code
pulls from the cache directory via a file:// URL.
The installPDFTools() support function now installs the files directly
instead of going through the prefs, and a new uninstallPDFTools()
function removes the tools. Since the presence of the PDF tools can
affect other tests, tests that need the tools should install them in a
before() and uninstall them in an after(), leaving most tests to run
without PDF indexing.
This also adds a callback to the waitForWindow() support function. If a
modal dialog is opened, it blocks the next promise handler from running,
so a callback has to be used to interact with and close the dialog
immediately.
