Commit graph

1029 commits

Author SHA1 Message Date
Dan Stillman
2c046a227b Fix sync error on collection/search conflict
> Invalid data error for collection [key] in [library]: Unknown
> collection property 'dateAdded'"

Regression from 4b60c6ca27, probably
2020-04-23 03:56:14 -04:00
Dan Stillman
81e2bee452 Fix 'false' filename after case-only rename on Macs
Zotero.File.move() now forces `overwrite` if the old and new filenames
differ only by case, since otherwise on a case-insensitive filesystem
OS.File.move() does an existence check and thinks that the target file
exists.
2020-04-22 00:01:28 -04:00
Dan Stillman
ad13313924 Don't change existing sync cache object on 'unchanged' response
Previously, if an object was uploaded but the API returned 'unchanged',
the uploaded data would be written to the sync cache, which, given that
most requests are patch requests, could result in an empty or mostly
empty object being saved to the sync cache. That would cause the next
sync to treat most/all local fields as changed and either upload them
unnecessarily or trigger a conflict instead of merging changes
automatically.
2020-04-14 05:07:50 -04:00
Dan Stillman
083588e211 Save correct data to cache when merging non-conflicting sync changes
When non-conflicting changes were automatically merged, the local object
would be correctly marked as unsynced, but the merged object rather than
the remote object would be saved to the sync cache. When the object was
then uploaded, it matched the cache version exactly, so an empty patch
object (other than an unchanged dateModified, which is always included)
would be uploaded and the local change wouldn't make it to the server.
The empty patch would result in an 'unchanged' response, which would
cause the empty patch object to be saved to the sync cache (which is a
bug that I'll fix separately). If the local object was modified again,
the patch would include all fields (since the cache object was empty)
and the local change would be uploaded, but there could also be
unnecessary conflicts due to it looking like all local fields had been
modified.

This patch causes the remote object to be saved to the sync cache
instead, so the local change looks like a local edit and is correctly
uploaded.
2020-04-13 03:42:28 -04:00
Dan Stillman
617564982c Include only related-item relations when duplicating items
Don't include linked-object or replaced-item relations. Previously, if
you duplicated an item, modified it to represent a different source, and
dragged it to another library where you had already copied the original
item, the new item wouldn't be transferred.

https://forums.zotero.org/discussion/comment/353246/#Comment_353246
2020-04-13 01:31:21 -04:00
Dan Stillman
e9ea9ae171 Leave Event Place and Publisher Place in Extra on import
These CSL variables are both currently mapped to the Zotero Place field,
but that's imprecise and will change:

https://github.com/citation-style-language/zotero-bits/issues/6

So for now, don't convert these to a Place line or make them eligible
for migration to a real field (once we turn that on).

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:51 -04:00
Dan Stillman
0e3d707576 Remove a couple redundant lines in Extra on RDF import
`publicationTitle`/`reporter` (and fields mapped to `publicationTitle`)
and `meetingName`/`conferenceName`

I assume these should just be base-field mappings, but since they're
not, they're not automatically deduplicated in `fromJSON()` and need to
be handled separately.

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:27 -04:00
Dan Stillman
91ca6d2ba6 extraToCSL(): Fix "zoteroField[1] is undefined" on single-character field
E.g., "a:" on a line

https://forums.zotero.org/discussion/82286/new-line-in-extra-field-induces-error-during-bibliography-generation
2020-04-03 18:13:10 -04:00
Dan Stillman
8b0dbc7708 Update fileInterface test after 0f212bdd76 2020-03-24 17:53:13 -04:00
Dan Stillman
679148d8d6 Don't store "Version Number" in Extra for books coming from RDF
The RDF translator assigns 'versionNumber' in addition to 'edition'.
2020-03-24 14:48:09 -04:00
Dan Stillman
d3601bba24 Remove incorrect test for Extra 'type:' extraction 2020-03-23 19:09:59 -04:00
Dan Stillman
52d5b68564 Update 'type:' handling when migrating Extra lines
- When changing type based on 'type:' line, move existing fields that
  are no longer valid to Extra
- Remove 'type:' line with CSL type if the item's existing type is one
  of the types mapped to it
2020-03-22 15:19:24 -04:00
Dan Stillman
02b43cbfac Refactor migrateExtraFields() tests 2020-03-21 00:56:55 -04:00
Dan Stillman
592188bfc5 Disabled test for keeping unfiled item selected on add to collection
A fix will need to wait for the Reactified items tree

Addresses #1799
2020-03-19 13:08:23 -04:00
Dan Stillman
f1816c12ad Fix test from 28ea77dae 2020-03-17 14:17:02 -04:00
Dan Stillman
28ea77dae1 Add test for ignoring invalid item type in Extra 2020-03-17 13:51:19 -04:00
Dan Stillman
3f2998bb6b Don't extract 'type: note' or 'type: attachment' from Extra 2020-03-17 13:50:45 -04:00
Dan Stillman
d8d07362ba Add additional test for creator migration from Extra 2020-03-17 10:37:16 -04:00
Dan Stillman
dd9ec54c3b Don't try to migrate Extra fields in read-only libraries 2020-03-15 19:52:56 -04:00
Dan Stillman
23c936f016 Fix creator migration from Extra 2020-03-15 13:51:04 -04:00
Dan Stillman
0679809735 Fix invalid collection nesting in DB integrity check
It shouldn't be possible to nest two collections inside each other, but
if it happens, fix it in the integrity check.

Also detect it from CollectionTreeView::expandToCollection() (used when
showing the collections containing an item) and crash Zotero with a flag
to run an integrity check after restart. Previously, this would result
in an infinite loop.

This may be the cause of some of the collection disappearances people
have reported. If parentCollectionID never leads to a null, the
collection won't appear anywhere in the tree.

TODO:

- Figure out how this is happening
- Detect and fix it automatically for people it's happened to
2020-03-13 18:07:20 -04:00
Dan Stillman
9175f9ade8 Don't use "Create Bib" locale for Quick Copy when no explicit locale set
If you'd never set a Quick Copy locale, the option would show as using
the current locale, but Quick Copy itself would use the last locale from
"Create Bibliography from Items". That was a side effect of behavior we
put in place in 2015 so that documents created before 4.0.27 that relied
on the removed bibliographyLocale pref would continue using the migrated
locale, but now that we've had an explicit locale option for years in
the document preferences I think we can stop doing that.
2020-03-13 17:06:29 -04:00
Dan Stillman
d389a71280 Download remotely updated files in "as needed" file sync mode
Previously, files updated remotely wouldn't be downloaded in "as needed"
mode if a copy of the file already existed locally and could only be
re-downloaded by deleting the file via Show File.

This causes remotely modified files that exist locally to be downloaded
at sync time, even in "as needed" mode, by marking them as
"force_download". While this might not be ideal for people who use "as
needed" to limit data transfer, it's better for people who use it simply
to limit local storage, and ending up with an outdated file while
offline seems worse than a little bit of extra data transfer.

In the future, we'll likely also provide ways to explicitly download and
remove files, so keeping chosen files in sync makes sense.

Files modified remotely before this change (which were marked as
"to_download" instead of "force_download") won't be downloaded as sync
time in "as needed" mode, but they'll now be re-downloaded on open.

Fixes #1322
2020-03-09 01:19:52 -04:00
Dan Stillman
76a1535a60 Full-text indexing improvements
- Use full-text cache file from syncing if available when reindexing via
  info pane or Rebuild Index → Index Unindexed Items. Only discard it for
  full index rebuild. This allows Index Unindexed Items to be used to
  force immediate processing of queued content from syncing and avoids
  unnecessary syncing back of identical content. Previously, the cache
  file was used for a manual index only when the local file didn't exist.
- When rebuilding index, don't clear indexed items with missing local
  file that are missing stats due to a pre-411180ef bug.
- indexItems() now takes an 'options' object as its second parameter
- Minor code cleanup
2020-03-09 01:19:52 -04:00
Dan Stillman
411180ef83 Full-text indexing fixes
- Don't clear item's index stats (and show "Unknown") when an item is
  reindexed remotely and the content matches the local content
- Always update an item's state and its stats in the same query, to
  avoid incorrect feedback immediately after indexing
- Clean up `setItemContent()` tests
2020-03-06 03:11:16 -05:00
Dan Stillman
da5e8c549e Convert Zotero fields in Extra to CSL fields for citeproc-js 2020-03-02 01:38:10 -05:00
Dan Stillman
83cc65eea6 Fix parsing of CSL date fields in extractExtraField() 2020-03-02 01:34:27 -05:00
Dan Stillman
a9dee2f487 Item::fromJSON(): Remove invalid-for-type Type-mapped fields
"Type: [val]` in Extra means Item Type to citeproc-js, and Type values
from translators mostly aren't going to be useful if the item type
doesn't have a Type-mapped field.

https://forums.zotero.org/discussion/comment/348864/#Comment_348864
2020-02-17 11:39:35 -05:00
Dan Stillman
b2bf60e1d7 Item.fromJSON(): Dedupe invalid-for-type fields when storing in Extra
3de54455f6 removed redundant base-mapped fields when a valid-for-type
field was also set, but that still left duplicate fields in Extra when a
valid field wasn't set. This will happen until translators (most notably
Embedded Metadata) are fixed to stop setting redundant fields.

https://forums.zotero.org/discussion/81262/translator-error-sage-lots-of-extra-data-in-extra
2020-02-17 00:22:34 -05:00
Dan Stillman
11caa1b719 Group strict-mode and non-strict-mode tests for Item::fromJSON() 2020-02-16 18:05:45 -05:00
Dan Stillman
b41734924d Further fixing of "Too many sync requests" error
Follow-up to 804a898c98

Addresses #1788
2020-02-16 18:05:45 -05:00
Dan Stillman
804a898c98 Hopefully fix "Too many sync requests" after file upload 412
Addresses #1788
2020-02-16 13:06:49 -05:00
Dan Stillman
a53f363b8d Additional fix for search crash with includeParentsAndChildren
Follow-up to 76081ab05
2020-02-11 13:09:54 -05:00
Dan Stillman
76081ab05f Fix crash when search uses no-op condition and includeParentsAndChildren
E.g., a nonexistent saved search
2020-02-11 00:23:45 -05:00
Dan Stillman
9e2ea008f4 Don't try to include jquery.js in tests HTML page 2020-02-11 00:23:45 -05:00
Dan Stillman
86a5c46b1e Find Available PDF: Don't mark URLs that redirect as tried
https://forums.zotero.org/discussion/81182
2020-02-02 23:47:40 -05:00
Dan Stillman
b4b33c07de Add a delay after the first browser initialization during tests
The window-loading process is different in Firefox, and running a test
that creates items while the window is still loading can cause things to
fail.
2020-02-02 23:47:40 -05:00
Dan Stillman
4e11c7927d Don't skip creator lines in Extra in fromJSON
Regression in 3de54455f6
2020-01-27 22:01:25 -05:00
Dan Stillman
3de54455f6 Automatically save unknown/invalid fields to Extra in non-strict mode
This is a prerequisite for starting to use new fields in translators,
since otherwise switching from, say, storing originalDate in Extra to
using an originalDate field would cause the value to be lost in clients
without the newer schema.

Closes #1504
2020-01-26 03:29:23 -05:00
Dan Stillman
3d2afa9c7f Additional adjustments to URL handling
Follow-up to df40ee7216

- Restore opening of non-HTTP URL schemes in link attachments
- Remove scheme whitelist for link attachments, since it's not enforced
  via the API anyway and we prompt before an external application is
  opened. Instead, just block a few schemes (e.g., 'javascript') from
  launching.

TODO:

- Provide some way to change/reset an application association if the
  user checks the box to automatically open that scheme.
- Show an error message if a link attachment with an invalid URL is
  double-clicked
2020-01-04 02:30:12 -05:00
Dan Stillman
6070743ff0 Fix Nextcloud WebDAV syncing
Add Zotero.HTTP.CookieBlocker mechanism to block storing and sending
of all cookies for a given URL prefix, and use that for the configured
WebDAV URL.

https://forums.zotero.org/discussion/80429/sync-error-in-5-0-80
2020-01-04 02:29:52 -05:00
Dan Stillman
12ccbf45c8 Make HTTP tests more reliable 2019-12-23 04:31:24 -05:00
Dan Stillman
9c380c362a Hopefully fix 503 from NextCloud (since 5.0.78)
https://forums.zotero.org/discussion/comment/344790/#Comment_344790
2019-12-22 16:32:36 -05:00
Dan Stillman
b7dc0d8b0a Fix parsing of pre-1000 years
If a leading zero (e.g., '068'), parse as actual year instead of
19xx/20xx.
2019-12-19 02:15:36 -05:00
Dan Stillman
df40ee7216 Adjust URL launching behavior
- Support launching plausible HTTP URLs without schemes from
  Zotero.loadURI(), and use that when launching from URL field
- Show correct cursor feedback on URL label -- only show pointer if the
  URL is launchable
- Don't launch non-HTTP URLs (e.g., zotero://) from URL field
- Don't open HTTP URLs in viewer when using ZoteroPane.launchURI()
2019-12-16 14:56:06 -05:00
Dan Stillman
bbbd02444b Restore 'yesterday'/'today'/'tomorrow' parsing for dates in searches
Follow-up to a549a64de9, which removed it from strToDate()
2019-12-06 03:12:48 -07:00
Dan Stillman
61cebbd8f7 Fix display of a couple WebDAV verification errors
"spec is undefined"

Fixes #1745
2019-12-01 03:39:54 -07:00
Dan Stillman
fbb54a7621 Fix "string is undefined" export error
Regression from a549a64de9
2019-11-29 01:50:43 -07:00
Dan Stillman
8c7677a009 Fix out-of-memory error syncing collections nested inside each other
It shouldn't be possible for collections to be nested this way, if it
happens, it shouldn't result in an infinite loop.

This removes one of the parent assignments at sync time.
2019-11-26 15:30:43 -07:00
Dan Stillman
5791ffeb16 Reactify item tags box
Improvements:

- Fixes autocomplete text remaining in field after selection in Fx60
- No more text or icon shifting on select (tested on macOS)

Changes:

- Tags are now selected on mousedown with no active state, as in web
  library

Regressions:

- Tooltip with tag type doesn't appear when hovering over icon
- Pressing Tab after modifying a tag loses focus
- Right-click in textbox shows custom menu instead of default text
  editing context menu (Cut/Copy/Paste)

To-do:

- Switch to this version for note tags box
- Style colored tags in autocomplete drop-down? Sort to top?
- Only show delete button on row hover, as in web library?
2019-11-08 06:41:06 -05:00