Commit graph

925 commits

Author SHA1 Message Date
Dan Stillman
abadbb2d77 Add fileInterface test for importing into selected collection
And update importFile() calls in tests to use current signature
2020-07-12 04:53:29 -04:00
Dan Stillman
6fd0b8245b Avoid logged error in retraction test from missing response data 2020-06-25 20:53:20 -04:00
Dan Stillman
7422c50076 Fix detection of retractions for items with DOI in Extra
Affected items should be detected on the next retraction updates check
2020-06-25 20:53:20 -04:00
Dan Stillman
bccf5ff0b2 Fix endless WebDAV loops if server has wrong mtimes but hash matches
Possibly caused by a third-party client uploading mtimes that then
aren't synced, or that differ from what get synced. When we detect this,
try to correct it by updating mtimes on WebDAV and the API to match the
local file.

https://forums.zotero.org/discussion/83554/zotero-loop-syncs-2000-items
2020-06-09 01:26:11 -04:00
Dan Stillman
a8c682bf4b Preserve linked-object and replaced-item relations when merging items
https://groups.google.com/d/msgid/zotero-dev/6f764822-ed7c-46eb-8068-ce9ed1a1538c%40googlegroups.com

Regression from 617564982c
2020-06-06 17:16:09 -04:00
Dan Stillman
b2e902746a Strip HTML tags from titles when generating filenames 2020-06-02 17:06:42 -04:00
Dan Stillman
d7ee3fdee2 strToDate(): A couple additional tests 2020-05-28 07:00:11 -04:00
Dan Stillman
0620b16d3e strToDate(): Don't parse 01/01/08 as the year 8
Regression from b7dc0d8b0a (5.0.81)
2020-05-28 06:19:42 -04:00
Dan Stillman
85bc5d168e strToDate(): Fix 'order' for just a number <=12
"m" instead of "m y"
2020-05-28 04:35:14 -04:00
Dan Stillman
70b361ed8b Fix stateful retractions tests
Closes #1824
2020-05-27 08:44:25 -04:00
Dan Stillman
f43df90225 Fix applying remote deletion of item in collection in read-only library
"Cannot edit item in read-only library"

I have no idea how this bug has gone undetected for so long.
2020-05-26 08:23:18 -04:00
Dan Stillman
c65322d0a4 Clarify warnings about data removal when switching accounts
And improve styling of hardConfirmationDialog.xul

Closes #1359
2020-05-23 03:23:32 -04:00
Dan Stillman
a79d1c2114 Fix test failure after 30cefca18e 2020-05-20 18:47:55 -04:00
Dan Stillman
30cefca18e Fix cross-library collection dragging from read-only library
This has apparently always been broken.
2020-05-20 08:45:37 -04:00
Dan Stillman
0199428c57 Fix error migrating Extra with empty author in citeproc-js cheater syntax
E.g., `{:author: }`

https://forums.zotero.org/discussion/83070/error-report-id-n-a-there-was-an-error-starting-zotero
2020-05-12 01:00:21 -04:00
Dan Stillman
cef47aaef6 Update test after AMA update 2020-05-09 19:30:41 -04:00
Dan Stillman
806ba2eb72 Fix test failure after 40fe85b274 2020-04-29 16:29:22 -04:00
Dan Stillman
2c046a227b Fix sync error on collection/search conflict
> Invalid data error for collection [key] in [library]: Unknown
> collection property 'dateAdded'"

Regression from 4b60c6ca27, probably
2020-04-23 03:56:14 -04:00
Dan Stillman
81e2bee452 Fix 'false' filename after case-only rename on Macs
Zotero.File.move() now forces `overwrite` if the old and new filenames
differ only by case, since otherwise on a case-insensitive filesystem
OS.File.move() does an existence check and thinks that the target file
exists.
2020-04-22 00:01:28 -04:00
Dan Stillman
ad13313924 Don't change existing sync cache object on 'unchanged' response
Previously, if an object was uploaded but the API returned 'unchanged',
the uploaded data would be written to the sync cache, which, given that
most requests are patch requests, could result in an empty or mostly
empty object being saved to the sync cache. That would cause the next
sync to treat most/all local fields as changed and either upload them
unnecessarily or trigger a conflict instead of merging changes
automatically.
2020-04-14 05:07:50 -04:00
Dan Stillman
083588e211 Save correct data to cache when merging non-conflicting sync changes
When non-conflicting changes were automatically merged, the local object
would be correctly marked as unsynced, but the merged object rather than
the remote object would be saved to the sync cache. When the object was
then uploaded, it matched the cache version exactly, so an empty patch
object (other than an unchanged dateModified, which is always included)
would be uploaded and the local change wouldn't make it to the server.
The empty patch would result in an 'unchanged' response, which would
cause the empty patch object to be saved to the sync cache (which is a
bug that I'll fix separately). If the local object was modified again,
the patch would include all fields (since the cache object was empty)
and the local change would be uploaded, but there could also be
unnecessary conflicts due to it looking like all local fields had been
modified.

This patch causes the remote object to be saved to the sync cache
instead, so the local change looks like a local edit and is correctly
uploaded.
2020-04-13 03:42:28 -04:00
Dan Stillman
617564982c Include only related-item relations when duplicating items
Don't include linked-object or replaced-item relations. Previously, if
you duplicated an item, modified it to represent a different source, and
dragged it to another library where you had already copied the original
item, the new item wouldn't be transferred.

https://forums.zotero.org/discussion/comment/353246/#Comment_353246
2020-04-13 01:31:21 -04:00
Dan Stillman
e9ea9ae171 Leave Event Place and Publisher Place in Extra on import
These CSL variables are both currently mapped to the Zotero Place field,
but that's imprecise and will change:

https://github.com/citation-style-language/zotero-bits/issues/6

So for now, don't convert these to a Place line or make them eligible
for migration to a real field (once we turn that on).

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:51 -04:00
Dan Stillman
0e3d707576 Remove a couple redundant lines in Extra on RDF import
`publicationTitle`/`reporter` (and fields mapped to `publicationTitle`)
and `meetingName`/`conferenceName`

I assume these should just be base-field mappings, but since they're
not, they're not automatically deduplicated in `fromJSON()` and need to
be handled separately.

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:27 -04:00
Dan Stillman
91ca6d2ba6 extraToCSL(): Fix "zoteroField[1] is undefined" on single-character field
E.g., "a:" on a line

https://forums.zotero.org/discussion/82286/new-line-in-extra-field-induces-error-during-bibliography-generation
2020-04-03 18:13:10 -04:00
Dan Stillman
8b0dbc7708 Update fileInterface test after 0f212bdd76 2020-03-24 17:53:13 -04:00
Dan Stillman
679148d8d6 Don't store "Version Number" in Extra for books coming from RDF
The RDF translator assigns 'versionNumber' in addition to 'edition'.
2020-03-24 14:48:09 -04:00
Dan Stillman
d3601bba24 Remove incorrect test for Extra 'type:' extraction 2020-03-23 19:09:59 -04:00
Dan Stillman
52d5b68564 Update 'type:' handling when migrating Extra lines
- When changing type based on 'type:' line, move existing fields that
  are no longer valid to Extra
- Remove 'type:' line with CSL type if the item's existing type is one
  of the types mapped to it
2020-03-22 15:19:24 -04:00
Dan Stillman
02b43cbfac Refactor migrateExtraFields() tests 2020-03-21 00:56:55 -04:00
Dan Stillman
592188bfc5 Disabled test for keeping unfiled item selected on add to collection
A fix will need to wait for the Reactified items tree

Addresses #1799
2020-03-19 13:08:23 -04:00
Dan Stillman
f1816c12ad Fix test from 28ea77dae 2020-03-17 14:17:02 -04:00
Dan Stillman
28ea77dae1 Add test for ignoring invalid item type in Extra 2020-03-17 13:51:19 -04:00
Dan Stillman
3f2998bb6b Don't extract 'type: note' or 'type: attachment' from Extra 2020-03-17 13:50:45 -04:00
Dan Stillman
d8d07362ba Add additional test for creator migration from Extra 2020-03-17 10:37:16 -04:00
Dan Stillman
dd9ec54c3b Don't try to migrate Extra fields in read-only libraries 2020-03-15 19:52:56 -04:00
Dan Stillman
23c936f016 Fix creator migration from Extra 2020-03-15 13:51:04 -04:00
Dan Stillman
0679809735 Fix invalid collection nesting in DB integrity check
It shouldn't be possible to nest two collections inside each other, but
if it happens, fix it in the integrity check.

Also detect it from CollectionTreeView::expandToCollection() (used when
showing the collections containing an item) and crash Zotero with a flag
to run an integrity check after restart. Previously, this would result
in an infinite loop.

This may be the cause of some of the collection disappearances people
have reported. If parentCollectionID never leads to a null, the
collection won't appear anywhere in the tree.

TODO:

- Figure out how this is happening
- Detect and fix it automatically for people it's happened to
2020-03-13 18:07:20 -04:00
Dan Stillman
9175f9ade8 Don't use "Create Bib" locale for Quick Copy when no explicit locale set
If you'd never set a Quick Copy locale, the option would show as using
the current locale, but Quick Copy itself would use the last locale from
"Create Bibliography from Items". That was a side effect of behavior we
put in place in 2015 so that documents created before 4.0.27 that relied
on the removed bibliographyLocale pref would continue using the migrated
locale, but now that we've had an explicit locale option for years in
the document preferences I think we can stop doing that.
2020-03-13 17:06:29 -04:00
Dan Stillman
d389a71280 Download remotely updated files in "as needed" file sync mode
Previously, files updated remotely wouldn't be downloaded in "as needed"
mode if a copy of the file already existed locally and could only be
re-downloaded by deleting the file via Show File.

This causes remotely modified files that exist locally to be downloaded
at sync time, even in "as needed" mode, by marking them as
"force_download". While this might not be ideal for people who use "as
needed" to limit data transfer, it's better for people who use it simply
to limit local storage, and ending up with an outdated file while
offline seems worse than a little bit of extra data transfer.

In the future, we'll likely also provide ways to explicitly download and
remove files, so keeping chosen files in sync makes sense.

Files modified remotely before this change (which were marked as
"to_download" instead of "force_download") won't be downloaded as sync
time in "as needed" mode, but they'll now be re-downloaded on open.

Fixes #1322
2020-03-09 01:19:52 -04:00
Dan Stillman
76a1535a60 Full-text indexing improvements
- Use full-text cache file from syncing if available when reindexing via
  info pane or Rebuild Index → Index Unindexed Items. Only discard it for
  full index rebuild. This allows Index Unindexed Items to be used to
  force immediate processing of queued content from syncing and avoids
  unnecessary syncing back of identical content. Previously, the cache
  file was used for a manual index only when the local file didn't exist.
- When rebuilding index, don't clear indexed items with missing local
  file that are missing stats due to a pre-411180ef bug.
- indexItems() now takes an 'options' object as its second parameter
- Minor code cleanup
2020-03-09 01:19:52 -04:00
Dan Stillman
411180ef83 Full-text indexing fixes
- Don't clear item's index stats (and show "Unknown") when an item is
  reindexed remotely and the content matches the local content
- Always update an item's state and its stats in the same query, to
  avoid incorrect feedback immediately after indexing
- Clean up `setItemContent()` tests
2020-03-06 03:11:16 -05:00
Dan Stillman
da5e8c549e Convert Zotero fields in Extra to CSL fields for citeproc-js 2020-03-02 01:38:10 -05:00
Dan Stillman
83cc65eea6 Fix parsing of CSL date fields in extractExtraField() 2020-03-02 01:34:27 -05:00
Dan Stillman
a9dee2f487 Item::fromJSON(): Remove invalid-for-type Type-mapped fields
"Type: [val]` in Extra means Item Type to citeproc-js, and Type values
from translators mostly aren't going to be useful if the item type
doesn't have a Type-mapped field.

https://forums.zotero.org/discussion/comment/348864/#Comment_348864
2020-02-17 11:39:35 -05:00
Dan Stillman
b2bf60e1d7 Item.fromJSON(): Dedupe invalid-for-type fields when storing in Extra
3de54455f6 removed redundant base-mapped fields when a valid-for-type
field was also set, but that still left duplicate fields in Extra when a
valid field wasn't set. This will happen until translators (most notably
Embedded Metadata) are fixed to stop setting redundant fields.

https://forums.zotero.org/discussion/81262/translator-error-sage-lots-of-extra-data-in-extra
2020-02-17 00:22:34 -05:00
Dan Stillman
11caa1b719 Group strict-mode and non-strict-mode tests for Item::fromJSON() 2020-02-16 18:05:45 -05:00
Dan Stillman
b41734924d Further fixing of "Too many sync requests" error
Follow-up to 804a898c98

Addresses #1788
2020-02-16 18:05:45 -05:00
Dan Stillman
804a898c98 Hopefully fix "Too many sync requests" after file upload 412
Addresses #1788
2020-02-16 13:06:49 -05:00
Dan Stillman
a53f363b8d Additional fix for search crash with includeParentsAndChildren
Follow-up to 76081ab05
2020-02-11 13:09:54 -05:00