This output is basically only useful when something hangs, so it can
stay off all other times.
For now, 6 can be the new only-use-when-something-is-actively-broken
level. At some point we may want to move DB activity to 4 and make this
sort of thing 5, because we don't have much that's 4 right now.
This uses ISO 8601 dates for generateAllTypesAndFieldsData (and
changes populateDBWithSampleData to use Item#fromJSON), and makes
translators expect ISO 8601 accessDates, although SQL accessDates are
still supported with a deprecation warning. Canonicalization happens in
Zotero.Translate, so I need to remember to update connectors as well.
And add group.fromJSON(json, userID), which sets editable and
filesEditable properties based on the group JSON (libraryReading, role
lists, etc.) and the given user
sinon.useFakeXMLHttpRequest() doesn't work in our case, but if
Zotero.HTTP.mock is set, Zotero.HTTP.request() will create new instances of
that object instead of the built-in XMLHttpRequest, so it can be set to
FakeXMLHttpRequest:
Zotero.HTTP.mock = sinon.FakeXMLHttpRequest;
var server = sinon.fakeServer.create();
server.autoRespond = true;
server.respondWith("GET", "/users/1",
[200, {"Content-Type": "application/json"}, '{"userID": 1}']);
var userInfo = yield getUser();
Zotero.HTTP.mock = null;
server.restore(); // probably not necessary
Also adds Zotero.DataObjects::getLoaded(), which returns an array of all
loaded objects of the given type. This is useful for selective
reloading, for example with item.reload(['relations'], true).
In particular, 0 is kept as a value, and passing undefined to setField
now throws an error.
I'm not sure if we actually want to return an empty string in all cases
for missing/invalid fields, but that's what we do currently.
And simplify tree view load event handling, which may or may not have
been contributing to intermittent test failures, but is cleaner this way
regardless.
Relations are now properties of collections and items rather than
first-class objects, stored in separate collectionRelations and
itemRelations tables with ids for subjects, with foreign keys to the
associated data objects.
Related items now use dc:relation relations rather than a separate table
(among other reasons, because API syncing won't necessarily sync both
items at the same time, so they can't be stored by id).
The UI assigns related-item relations bidirectionally, and checks for
related-item and linked-object relations are done unidirectionally by
default.
dc:isReplacedBy is now dc:replaces, so that the subject is an existing
object, and the predicate is now named
Zotero.Attachments.replacedItemPredicate.
Some additional work is still needed, notably around following
replaced-item relations, and migration needs to be tested more fully,
but this seems to mostly work.
The test runner now downloads and caches the PDF tools for the current
platform within the test data directory and only redownloads them when
out of date, and it updates the download URL so that the full-text code
pulls from the cache directory via a file:// URL.
The installPDFTools() support function now installs the files directly
instead of going through the prefs, and a new uninstallPDFTools()
function removes the tools. Since the presence of the PDF tools can
affect other tests, tests that need the tools should install them in a
before() and uninstall them in an after(), leaving most tests to run
without PDF indexing.
This also adds a callback to the waitForWindow() support function. If a
modal dialog is opened, it blocks the next promise handler from running,
so a callback has to be used to interact with and close the dialog
immediately.
Character sets are now populated on demand, so they can't be run through
Zotero.CharacterSets.getName() in getContentsAsync(), since they might
not exist. (I'm also not sure why this was being done anyway.)
For now, this just calls Zotero.Utilities.generateObjectKey(), but this
function makes more sense in DataObjectUtilities. It does need to be
accessible to the connectors, but if it's possible to add an alias in
Zotero.Utilities just for the connectors, it'd probably be better to do
that and use Zotero.DataObjectUtilities.generateKey() elsewhere.
These previously returned an itemID, but now that new saved items can be edited
without a refetch, they should just return the new item.
(Possible I missed a few spots where these are called.)
Notifier.trigger() needs to be async, since if it actually runs it waits for
promises returned from observers. But the vast majority of trigger() calls are
in transactions where they just queue and can therefore be synchronous. This
replaces all such calls with Notifier.queue().
This should fix a race condition that was causing the emptyTrash() test to fail
intermittently.
citeproc-js relies on this in several locations. Seems that Zotero passes these IDs to citeproc from the item picker. We also need to consider existing embedded items in Word/LO documents, but they do have embedded URIs, so it shouldn't be a problem.
CC @fbennett
* Enable legacy mode for export translators compatible with pre-4.0.27:
* Add compatibility mappings, so that current translators don't break if they specify minVersion lower than 4.0.27. This does introduce non-compatible changes, specifically, "version" field in legacy mode is "versionNumber" in the new format. "version" in the new format corresponds to the "version" as specified for Zotero API JSON format. New translators should expect Zotero web API JSON format and should specify minVersion 4.0.27.
* Update CSL mappings to comply with new itemToExportFormat
* CSL JSON export translator needs to be updated to be compatible with 4.0.27 to export correct CSL JSON
* Use item URI for id in CSL JSON instead of item ID
* Fix note and attachment handling in itemToCSLJSON
After saving a new object and reloading primary data and any changed
data (which we can maybe reconsider at some point), mark all other data
types as loaded, since there's no other data we don't have. For example,
this allows for item.save() to be followed by item.setField() without
needing to call item.loadItemData() first.
updateBundledStyles() already has the contents of a new/updated
translator when it copies it into the data dir, so there's no need for
Zotero.Translators.reinit() to read it again from the just-copied file.
This passes the metadata from updateBundledStyles() to reinit() to avoid
the extra file read.
(Alas, this appears to make essentially zero difference on an OS X
system with an SSD, but maybe it will help elsewhere.)
Get rid of data_access.js, at long last. Existing calls to
Zotero.getCollections() will need to be replaced with
Zotero.Collections.getByLibrary() or .getByParent().
Also removes Zotero.Collection::getCollections(), which is redundant
with Zotero.Collections.getByLibrary(), and Zotero.Collections.add().
The latter didn't didn't include a libraryID anyway, so code might as
well just use 'new Zotero.Collection' instead.
If a view or other resources are destroyed while a promise is being
resolved, subsequent code can fail. This is generally harmless, but it
results in unnecessary errors being logged to the console.
To address this, promises can use a new function,
Zotero.Promise.check(), to test whether a value is truthy or 0 and
automatically throw a specific error that's ignored by the unhandled
rejection handler if not.
Example usage:
getAsync().tap(() => Zotero.Promise.check(this.win));
If this.win is cleaned up while getAsync() is being resolved, subsequent
lines won't be run, and nothing will be logged to the console.
Change all attachment functions to take parameter objects, including a
'collections' property to assign collections. (Previously, calling code
assigned collections separately, which required a nested transaction,
which is no longer possible.)
Fixes#723, Can't attach files by dragging
Wait for the pane's collectionSelected() to finish before returning from
collectionTreeView select methods (e.g., selectLibrary()), and wait for
previous items view to finish loading before creating a new one in
collectionSelected(). This ensures that the items view has been created (though
not loaded) before returning from a select. The tree can still get a bit
confused switching between collections, but I think we're getting closer to
fixing that.
Also switch the items tree to use the same pattern.
This also fixes dragging items to collections (#731).
Groups were already being loaded for the collections list, so we might
as well just store them initially and let Zotero.Libraries.getName() be
a synchronous call.
- Moved ::_get() and _set() from Collection/Search into DataObject, and
disabled in Item
- Don't disable new items after save. We now put new objects into the
DataObjects cache from save() so that changes made post-save are
picked up by other code using .get().
- Added 'skipCache' save() option to avoid reloading data on new objects
and adding them to the cache. (This will be used in syncing, where
objects might be in another library where they're not needed right
away.) Objects created with this option are instead disabled to
prevent reuse.
- Modified some tests to try to make sure we're reloading everything properly
after a save.
- Documented save() options
- Fixes some saving and erasing issues with collections and searches
- Adds Zotero.DataObject::eraseTx() to automatically start transaction,
and have .erase() log a warning like .save()
- Adds createUnsavedDataObject() and createDataObject() helper functions
for tests
The sync cache will have pristine copies of the existing versions of
local objects for better conflict resolution, but downloads will get
saved to the sync cache first before processing, so the cache needs to
be able to hold more than one version.
Replace Z.DataObjects::diff() with Z.DataObjectUtilities.diff(). Instead
of just returning two objects with the differing fields, the new diff()
generates a changeset with operations to apply with applyChanges(),
including at the array member level for collections and tags. This,
combined with cached pristine copies of objects, will allow for vastly
better conflict resolution, with automatic merging of non-conflicting
changes.
Creators currently don't show granular changes, and ordering might make
it too tough to do so. Relations diffing isn't yet implemented.
Also:
- Make .mode == 'patch' optional if .patchBase is provided.
- Remove requirement for item to be unchanged, which hopefully wasn't there for
a good reason
- Add a few tests, though more are needed
This allows calling code to do something other than call Zotero.debug()
on errors (like, say, nothing, in order to avoid logging certain
expected errors) before throwing.
Store and check the last selected items in ZoteroPane.itemSelected() to
see if it's necessary to refresh the item pane. This prevents loss of
textbox focus if another write occurs while editing a field.
Also optimize row adding/removing in itemTreeView.js
Fix a couple cases of lost text field focus after an edit, including
focusing of the Title field after using New Item when a field is already
being edited and has a changed value.
Also, in tests, select My Library and wait for items to load when using
the loadZoteroPane() support function. We could add a parameter to skip
that or move it to a separate function, but the code to detect it is a
bit convoluted and it's a prerequisite for many tests, so it's handy to
have a function for it.
If a new item is created manually when a text field is already open, the
hideEditor is run first on the html:textarea before bubbling up to the
textbox. During a normal blur, the event only happens on the textbox,
though I haven't looked into the reason for the difference.
- Add an 'exclusive' option to transactions that causes them to block other
transactions and wait for other transactions to finish before starting,
instead of nesting
- Resolve Zotero.DB.waitForTransaction() promise before returning from
executeTransaction()
- A side effect of the above: wait for a newly created item to be selected in
the middle pane and rendered in the right-hand pane before returning from
executeTransaction()
- Don't save items multiple times when adding/removing a non-final creator in
the Info pane
- Use a simpler, non-recursive method for focusing the next field in the Info
pane; this prevents "too much recursion" errors if something causes the
right-hand pane not to be rendered when expected
This fixes an issue where two transactions started around the same time
could run separately instead of nesting, causing the statements from one
to end up running not within a transaction
And use 'version' instead of 'itemVersion' for object version for items
Also add deferred foreign key checking to system.sql so that DROP TABLE
commands don't fail mid-transaction
Show a wizard after items are dragged to My Publications choosing
whether to include files and notes and choosing sharing settings for the
items. Sharing options are Creative Commons licenses, CC0, "All rights
reserved", or keeping the existing Rights field if available.
Also blocks collections, searches, linked file attachments, and
top-level attachments/notes from My Publications at the data layer, but
not yet from the UI in all places (so it can crash if you try).
Todo:
- Block certain UI actions with nice messages
- Show a nice scrollable list of items in the wizard to allow selecting
specific files/notes, instead of just having checkboxes for files and
notes that apply to all dragged items
- Show an explanation of My Publications in the right-hand pane when no
items are selected
- Maybe adjust handling when no attached files/notes, since it might be
a bit alarming at the moment to see sharing options for metadata
entries
Previously, object identifiers were registered in a commit callback, but that
meant that they wouldn't be available to getLibraryAndKeyFromID/
getIDFromLibraryAndKey within a transaction. Instead, register them before
saving and clear them in a tranasction rollback if necessary.
This changes Zotero.DataObject to call its own _finalizeSave() in addition to a
descendent one.
Object contains 'libraryID' and 'key' properties
This is due to changed array destructuring behavior in Firefox. Previously,
`var [foo, bar] = maybeArrayMaybeFalse()` always worked, leaving foo and bar
undefined if the function returned false. Now (with ES6, I assume), if the
function returns false it results in a "false[Symbol.iterator] is not a
function" error. But `var {libraryID, key} = false` works as expected, leaving
both values undefined, so instead we can just return an object with those
properties from getLibraryAndKeyFromID(). To assign to different variables, use
`var {libraryID, key: parentItemKey} = ...`.
Search condition ids are now indexed from 0, and always saved
contiguously (no more 'fixGaps' option), since they're just in an array
in the API. (They're still returned as an object from
Zotero.Search.prototype.getConditions() because it's easier for the
advanced search window to not have to deal with shifting ids between
saves.)
Also:
- Return an object from `Zotero.Search.prototype.getConditions()`
instead of an array.
- Add support function `getPromiseError(promise)` to return the error
thrown from a chain of promises, or false if none. (We could make an
`assert.throwsAsync()`, but this allows testing of various properties
such as `.name`, which even the built-in `assert.throws()` can't
test.)
- Clarify some search save errors
'b' for *b*undled files
Translators and styles take a long time to install and initialize in
source installations, and they're unnecessary for many tests.
This shaves about 10 seconds off each test run for me on one system (and
that's with some help from filesystem caching).
This is just a wrapper around nsIPromptService.alert() that takes
Zotero.noUserInput into consideration, which avoids long timeouts during
testing (e.g., for lookup failures).
Previously, getItemURI and getCollectionURI had to check the libraryID
to determine whether to return a local URI, and were doing so using an
obsolete check.
To deal with issues running .vbs scripts on some systems. If the 3.02
binaries haven't been reinstalled since this version, reinstall them as
3.02a automatically or on manual upgrade to fix corrupted binaries from
previous gzip issue.
Other platforms unchanged
Previously, if an editor was entered before an author, it would sort by
the editor. Now, as long as there's an author, it will sort by that
first (and, with 2f3d865f, favor left-bound matches).
Addresses the second issue on
https://forums.zotero.org/discussion/48047/
Courtesy of Pastel SVG (2x famfamfam)
I didn't add 2x versions for any composite icons or any Fugue icons.
Also:
- Removed some unused/redundant images
- Switched to shadowless versions for a couple Fugue icons
Use stdout redirection scripts for pdfinfo and, on Windows, a script to
run pdftotext hidden, which together allow for all unmodified binaries
(including, probably, symlinked system ones, though I didn't test that).
On Windows, using a .vbs does cause a brief wait cursor. The stock
pdfinfo needs the redirection script anyway, so that's unavoidable, but
on the async branch I think we'll be able to switch to pdf.js for the
page count, at which point maybe I'll try to remember how I modified the
Windows binaries to be hidden and use a modified version of pdftotext to
avoid VBScript. (We use the stock pdftotext elsewhere already.)
If automatic translator/style updates are enabled, at least one of the
PDF tools is installed, and the repo returns a more recent version
number than what's installed, automatically upgrade the tools. (Version
3.02 counts as lower, since Poppler's version numbers are lower.)
If an error occurs, wait increasing amounts of time to try the downloads
again, up to one week.
Fixes regression from 37921b0910
cleanISBN/ISSN now return first valid ISBN/ISSN from a list of valid/invalid ISBNs/ISSNs, even if invalid ISBN/ISSN comes first
toISBN13 now corrects the check digit for ISBN13 input
It turns out that the Cygwin console, unlike -console, is actually
usable, so developers on Windows can use that. Since we sometimes need
real-time debug output from end users (who won't have Cygwin installed),
keep logging to the Browser Console if only the -ZoteroDebug flag is
passed.
- Use async DB and OS.File for bundled file updates
- Remove support for translator/style ZIP files -- the two options are
now non-unpacked XPIs with subfolders or unpacked source installations
- Now that we have async file access, don't store translator code in
database cache -- just store metadata so that it's available without
reading each translator file
- Change the (previously partially asyncified) Zotero.Styles/Translators
APIs a bit -- while the getAll/getVisible methods are asynchronous and
will wait for loading, the get() methods are synchronous and require
styles/translators to be initialized before they're called. Most
places that end up calling get() probably call getAll/getVisible first
and should therefore be async, but if there's any way to trigger a
get() first, that will need to be adjusted.
- Asyncify various other style/translator-related code
XPI support is untested, as is style/translator usage, so there are
almost certainly bugs. The latter depends on updated export format
support (#659), since toArray() no longer exists on this branch.
Addresses #529 and #520
I don't think there's any way for this to happen short of manual
preference editing, and the pane width would have to be exactly 250px,
and it's pretty harmless anyway, but might as well be safe.
There are hundreds more, but these are all the ones that generate warnings in
the console at startup. XPCOM/XBL ones don't seem to do so, so we can ignore
those for now (and hopefully not bother with them on 4.0). Instances in
translators do generate warnings.
Addresses #656
Note that this loses conflicting changes to translate_item.js from 849803473a,
so those will need to be reapplied if applicable. /cc @aurimasv, @mtd91429
Adds a "My Publications" source after "My Library", implemented as a
separate library. Top-level items can be dragged in and removed.
(This doesn't currently work without disabling Quick Copy.)
Also:
- Make "Group Libraries" an unselectable header instead of a container,
and don't indent group libraries
- Fix relation purging, which maybe never worked
- Pass only libraryID/key on deletes (which should speed them up)
- Fix async item cloning/copying
- Fix miscellaneous other bugs
To-do:
- Confirmation dialog on drag
- API support
0 allowed for some accidental behavior due to old code expecting NULL,
and it prevented easy checks (``if (!libraryID)``) for a passed
libraryID. Code now uses Zotero.Libraries.userLibraryID instead of a
hard-coded value (except in schema.js). Functions can still make
libraryID optional, but they should then use
Zotero.Libraries.userLibraryID if that's to mean the user library.
There might be some code that still expects 0 that I missed.
Fixes#426: this allows the localization of the Word Processors Add-in
button and messages displayed in the Prefs, under the "Word Processors"
tab. These strings were previously hardcoded in the integration plugins.
(Unfortunately this wipes out "Shortcuts" in all locales, but pushing
non-English locales to Transifex scares me too much. Sorry, localizers!
I think Transifex makes it easy to use previously used translations, at
least.)
[and delete unused "#zotero-prefpane-keys checkbox" (there used to be a checkbox to "Try to override conflicting shortcuts", which has been definely removed by 5b34dce40f )]
- New dedicated Zotero button, available in the Customize palette
- Remove "16px" Z SVG and just render "32px" one at the smaller size
(there's a slight difference at the edges, but we should probably redo
it anyway so that dark lines reach all the way to the bottom)
- Change "Open Zotero" back to just "Zotero", since 1) it's for closing
too, 2) it can also just bring Standalone to the front, and 3) "Open
Zotero" looks weird and inconsistent in the menu panel next to things
like "Preferences"
- For now, show single large Z for combo buttons in customization
palette with "Zotero (Combo)", though we can probably do something
better.
- Fix some HiDPI bugs when moving items between areas
Still need a layout for the panel for the combo buttons
Previously, if you dragged a standalone attachment to a collection in another
library where the item already existed as a child item, it would crash Zotero
on a collection-item constraint.
