It does not appear to be possible to create a creator with no values in
Mendeley, however we got reports of these causing the imports to fail.
This tweak makes the importer more resilient by discarding empty/invalid
creators.
* Importer will now ask user for a login and password via form and will
perform sign-in directly using credentials rather than oauth
* Signing in this way enables importer to obtain desktop document ID
which is now stored for each item
* It's possible to switch back to the old method (ouath) by setting
`import.mendeleyUseOAuth` pref to `true`.
* New option to only import new items. This options only appears if
database contains previously imported items.
* Importer will now update mendeleyDB:documentUUID on existing items to
match value used in Mendeley Desktop if available
* Importer will no longer create collections when no new items are
imported * Importer will only report number of new items imported on
re-import * Importer will now preserve dateAdded on re-import
Co-authored-by: Dan Stillman <dstillman@zotero.org>
Because regex is built using a template string, \s* is actually escaped
into s*, i.e. literal "s" appearing 0 or more times. In most cases this
would mean that output can have spacing slightly off. In extreme case,
when identifier starts with letter "s", this could this could lead to
identifier being stored incorrectly.
Also adjusted tests to be more strict and mock data to cover this case.
Remove Zotero.Browser and add HiddenBrowser.jsm. Post-Fission, web/file
content loads in a separate process, so it's not possible (as best as I
can tell) to directly access the contents of a hidden browser -- it just
appears as about:blank in the parent process. We now use Mozilla's
JSWindowActor mechanism [1] to get page data, including character set
and body text for full-text indexing. We'll have to evaluate other uses
of hidden browsers to see how to handle them.
This also adds include.jsm for loading the Zotero object into a JSM.
[1] https://firefox-source-docs.mozilla.org/dom/ipc/jsactors.html
We follow a different merge procedure for each attachment type:
- For PDF attachments, compare by MD5. If no match, get the top 50 words
in the attachment's text and hash those, then check again for a match.
Update references to item keys in notes and annotations.
- For web (snapshot / link) attachments, compare by title and URL.
Prefer a title + URL match but accept a title-only match.
- For other attachment types, keep all attachments from all items being
merged.
Also:
- Move most merge tests from Duplicates to Items#merge(). It just doesn't
make sense to worry about the UI in these.
Extended Mendeley Import test to include a scenario where other users
attached an annotation to an item in a group library that also exists
in user's library.
This adds extra tests to check parsing behavior such as entities, tag handling,
CDATA, etc. This will help ensure the new feed processor matches the previous
behavior.
This changes the way item types, item fields, creator types, and CSL
mappings are defined and handled, in preparation for updated types and
fields.
Instead of being predefined in SQL files or code, type/field info is
read from a bundled JSON file shared with other parts of the Zotero
ecosystem [1], referred to as the "global schema". Updates to the
bundled schema file are automatically applied to the database at first
run, allowing changes to be made consistently across apps.
When syncing, invalid JSON properties are now rejected instead of being
ignored and processed later, which will allow for schema changes to be
made without causing problems in existing clients. We considered many
alternative approaches, but this approach is by far the simplest,
safest, and most transparent to the user.
For now, there are no actual changes to types and fields, since we'll
first need to do a sync cut-off for earlier versions that don't reject
invalid properties.
For third-party code, the main change is that type and field IDs should
no longer be hard-coded, since they may not be consistent in new
installs. For example, code should use `Zotero.ItemTypes.getID('note')`
instead of hard-coding `1`.
[1] https://github.com/zotero/zotero-schema
- Fix incorrect results for ANY search with multiple "Attachment
Content" conditions and no other conditions
- Dramatically speed up single-word searches by avoiding unnecessary
text scans (which probably addresses #1595)
- Clean up code
This is hard to do currently because the natural place to do it (and
where the previous seeAlso stuff was done) is translate_item.js, but
with async import translators that now only gets one item at a time,
whereas saving item relations requires all items to be saved. So this
would probably need to be done in the import code in translate.js.
It might also require undoing
https://github.com/zotero/zotero/pull/453 so that getResourceURI() works
on notes and figuring out another solution for the problem that was
trying to solve.
Use Atom namespace when getting fields, and use `<updated>` date before
`<published>`. (The dates are also available on the nsIFeedContainer
(`feedEntry`), but we're getting them directly from the fields for some
reason.)
WPD code hasn't been updated in many years, and there was an issue with
document permissions in 5.0. We'll need to replace nsIWBP in Electron,
but this will do for now.
Attachments are opened using file:// URIs instead of
zotero://attachment, which is what Standalone does anyway. Ancient HTML
annotations and highlights won't be displayed anymore, but I'm not sure
they worked anyway, and it hasn't been possible to create them in years.
We might be able to write out existing annotations to notes.
iframes are skipped during saving, in an attempt to reduce the number of
junk ad files. JS can still cause problems with viewing, so we might
still want to either disable scripts or force the viewed page offline
(if such a thing is possible).
There might be issues with auxiliary filename length/characters during
cross-platform file syncing. (We modified the WPD code to shorten/clean
them.)
