Commit graph

917 commits

Author SHA1 Message Date
Dan Stillman
85bc5d168e strToDate(): Fix 'order' for just a number <=12
"m" instead of "m y"
2020-05-28 04:35:14 -04:00
Dan Stillman
70b361ed8b Fix stateful retractions tests
Closes #1824
2020-05-27 08:44:25 -04:00
Dan Stillman
f43df90225 Fix applying remote deletion of item in collection in read-only library
"Cannot edit item in read-only library"

I have no idea how this bug has gone undetected for so long.
2020-05-26 08:23:18 -04:00
Dan Stillman
c65322d0a4 Clarify warnings about data removal when switching accounts
And improve styling of hardConfirmationDialog.xul

Closes #1359
2020-05-23 03:23:32 -04:00
Dan Stillman
a79d1c2114 Fix test failure after 30cefca18e 2020-05-20 18:47:55 -04:00
Dan Stillman
30cefca18e Fix cross-library collection dragging from read-only library
This has apparently always been broken.
2020-05-20 08:45:37 -04:00
Dan Stillman
0199428c57 Fix error migrating Extra with empty author in citeproc-js cheater syntax
E.g., `{:author: }`

https://forums.zotero.org/discussion/83070/error-report-id-n-a-there-was-an-error-starting-zotero
2020-05-12 01:00:21 -04:00
Dan Stillman
cef47aaef6 Update test after AMA update 2020-05-09 19:30:41 -04:00
Dan Stillman
806ba2eb72 Fix test failure after 40fe85b274 2020-04-29 16:29:22 -04:00
Dan Stillman
2c046a227b Fix sync error on collection/search conflict
> Invalid data error for collection [key] in [library]: Unknown
> collection property 'dateAdded'"

Regression from 4b60c6ca27, probably
2020-04-23 03:56:14 -04:00
Dan Stillman
81e2bee452 Fix 'false' filename after case-only rename on Macs
Zotero.File.move() now forces `overwrite` if the old and new filenames
differ only by case, since otherwise on a case-insensitive filesystem
OS.File.move() does an existence check and thinks that the target file
exists.
2020-04-22 00:01:28 -04:00
Dan Stillman
ad13313924 Don't change existing sync cache object on 'unchanged' response
Previously, if an object was uploaded but the API returned 'unchanged',
the uploaded data would be written to the sync cache, which, given that
most requests are patch requests, could result in an empty or mostly
empty object being saved to the sync cache. That would cause the next
sync to treat most/all local fields as changed and either upload them
unnecessarily or trigger a conflict instead of merging changes
automatically.
2020-04-14 05:07:50 -04:00
Dan Stillman
083588e211 Save correct data to cache when merging non-conflicting sync changes
When non-conflicting changes were automatically merged, the local object
would be correctly marked as unsynced, but the merged object rather than
the remote object would be saved to the sync cache. When the object was
then uploaded, it matched the cache version exactly, so an empty patch
object (other than an unchanged dateModified, which is always included)
would be uploaded and the local change wouldn't make it to the server.
The empty patch would result in an 'unchanged' response, which would
cause the empty patch object to be saved to the sync cache (which is a
bug that I'll fix separately). If the local object was modified again,
the patch would include all fields (since the cache object was empty)
and the local change would be uploaded, but there could also be
unnecessary conflicts due to it looking like all local fields had been
modified.

This patch causes the remote object to be saved to the sync cache
instead, so the local change looks like a local edit and is correctly
uploaded.
2020-04-13 03:42:28 -04:00
Dan Stillman
617564982c Include only related-item relations when duplicating items
Don't include linked-object or replaced-item relations. Previously, if
you duplicated an item, modified it to represent a different source, and
dragged it to another library where you had already copied the original
item, the new item wouldn't be transferred.

https://forums.zotero.org/discussion/comment/353246/#Comment_353246
2020-04-13 01:31:21 -04:00
Dan Stillman
e9ea9ae171 Leave Event Place and Publisher Place in Extra on import
These CSL variables are both currently mapped to the Zotero Place field,
but that's imprecise and will change:

https://github.com/citation-style-language/zotero-bits/issues/6

So for now, don't convert these to a Place line or make them eligible
for migration to a real field (once we turn that on).

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:51 -04:00
Dan Stillman
0e3d707576 Remove a couple redundant lines in Extra on RDF import
`publicationTitle`/`reporter` (and fields mapped to `publicationTitle`)
and `meetingName`/`conferenceName`

I assume these should just be base-field mappings, but since they're
not, they're not automatically deduplicated in `fromJSON()` and need to
be handled separately.

https://groups.google.com/d/msgid/zotero-dev/806a22e3-3d6a-4d86-8747-10c787291a93%40googlegroups.com
2020-04-04 04:17:27 -04:00
Dan Stillman
91ca6d2ba6 extraToCSL(): Fix "zoteroField[1] is undefined" on single-character field
E.g., "a:" on a line

https://forums.zotero.org/discussion/82286/new-line-in-extra-field-induces-error-during-bibliography-generation
2020-04-03 18:13:10 -04:00
Dan Stillman
8b0dbc7708 Update fileInterface test after 0f212bdd76 2020-03-24 17:53:13 -04:00
Dan Stillman
679148d8d6 Don't store "Version Number" in Extra for books coming from RDF
The RDF translator assigns 'versionNumber' in addition to 'edition'.
2020-03-24 14:48:09 -04:00
Dan Stillman
d3601bba24 Remove incorrect test for Extra 'type:' extraction 2020-03-23 19:09:59 -04:00
Dan Stillman
52d5b68564 Update 'type:' handling when migrating Extra lines
- When changing type based on 'type:' line, move existing fields that
  are no longer valid to Extra
- Remove 'type:' line with CSL type if the item's existing type is one
  of the types mapped to it
2020-03-22 15:19:24 -04:00
Dan Stillman
02b43cbfac Refactor migrateExtraFields() tests 2020-03-21 00:56:55 -04:00
Dan Stillman
592188bfc5 Disabled test for keeping unfiled item selected on add to collection
A fix will need to wait for the Reactified items tree

Addresses #1799
2020-03-19 13:08:23 -04:00
Dan Stillman
f1816c12ad Fix test from 28ea77dae 2020-03-17 14:17:02 -04:00
Dan Stillman
28ea77dae1 Add test for ignoring invalid item type in Extra 2020-03-17 13:51:19 -04:00
Dan Stillman
3f2998bb6b Don't extract 'type: note' or 'type: attachment' from Extra 2020-03-17 13:50:45 -04:00
Dan Stillman
d8d07362ba Add additional test for creator migration from Extra 2020-03-17 10:37:16 -04:00
Dan Stillman
dd9ec54c3b Don't try to migrate Extra fields in read-only libraries 2020-03-15 19:52:56 -04:00
Dan Stillman
23c936f016 Fix creator migration from Extra 2020-03-15 13:51:04 -04:00
Dan Stillman
0679809735 Fix invalid collection nesting in DB integrity check
It shouldn't be possible to nest two collections inside each other, but
if it happens, fix it in the integrity check.

Also detect it from CollectionTreeView::expandToCollection() (used when
showing the collections containing an item) and crash Zotero with a flag
to run an integrity check after restart. Previously, this would result
in an infinite loop.

This may be the cause of some of the collection disappearances people
have reported. If parentCollectionID never leads to a null, the
collection won't appear anywhere in the tree.

TODO:

- Figure out how this is happening
- Detect and fix it automatically for people it's happened to
2020-03-13 18:07:20 -04:00
Dan Stillman
9175f9ade8 Don't use "Create Bib" locale for Quick Copy when no explicit locale set
If you'd never set a Quick Copy locale, the option would show as using
the current locale, but Quick Copy itself would use the last locale from
"Create Bibliography from Items". That was a side effect of behavior we
put in place in 2015 so that documents created before 4.0.27 that relied
on the removed bibliographyLocale pref would continue using the migrated
locale, but now that we've had an explicit locale option for years in
the document preferences I think we can stop doing that.
2020-03-13 17:06:29 -04:00
Dan Stillman
d389a71280 Download remotely updated files in "as needed" file sync mode
Previously, files updated remotely wouldn't be downloaded in "as needed"
mode if a copy of the file already existed locally and could only be
re-downloaded by deleting the file via Show File.

This causes remotely modified files that exist locally to be downloaded
at sync time, even in "as needed" mode, by marking them as
"force_download". While this might not be ideal for people who use "as
needed" to limit data transfer, it's better for people who use it simply
to limit local storage, and ending up with an outdated file while
offline seems worse than a little bit of extra data transfer.

In the future, we'll likely also provide ways to explicitly download and
remove files, so keeping chosen files in sync makes sense.

Files modified remotely before this change (which were marked as
"to_download" instead of "force_download") won't be downloaded as sync
time in "as needed" mode, but they'll now be re-downloaded on open.

Fixes #1322
2020-03-09 01:19:52 -04:00
Dan Stillman
76a1535a60 Full-text indexing improvements
- Use full-text cache file from syncing if available when reindexing via
  info pane or Rebuild Index → Index Unindexed Items. Only discard it for
  full index rebuild. This allows Index Unindexed Items to be used to
  force immediate processing of queued content from syncing and avoids
  unnecessary syncing back of identical content. Previously, the cache
  file was used for a manual index only when the local file didn't exist.
- When rebuilding index, don't clear indexed items with missing local
  file that are missing stats due to a pre-411180ef bug.
- indexItems() now takes an 'options' object as its second parameter
- Minor code cleanup
2020-03-09 01:19:52 -04:00
Dan Stillman
411180ef83 Full-text indexing fixes
- Don't clear item's index stats (and show "Unknown") when an item is
  reindexed remotely and the content matches the local content
- Always update an item's state and its stats in the same query, to
  avoid incorrect feedback immediately after indexing
- Clean up `setItemContent()` tests
2020-03-06 03:11:16 -05:00
Dan Stillman
da5e8c549e Convert Zotero fields in Extra to CSL fields for citeproc-js 2020-03-02 01:38:10 -05:00
Dan Stillman
83cc65eea6 Fix parsing of CSL date fields in extractExtraField() 2020-03-02 01:34:27 -05:00
Dan Stillman
a9dee2f487 Item::fromJSON(): Remove invalid-for-type Type-mapped fields
"Type: [val]` in Extra means Item Type to citeproc-js, and Type values
from translators mostly aren't going to be useful if the item type
doesn't have a Type-mapped field.

https://forums.zotero.org/discussion/comment/348864/#Comment_348864
2020-02-17 11:39:35 -05:00
Dan Stillman
b2bf60e1d7 Item.fromJSON(): Dedupe invalid-for-type fields when storing in Extra
3de54455f6 removed redundant base-mapped fields when a valid-for-type
field was also set, but that still left duplicate fields in Extra when a
valid field wasn't set. This will happen until translators (most notably
Embedded Metadata) are fixed to stop setting redundant fields.

https://forums.zotero.org/discussion/81262/translator-error-sage-lots-of-extra-data-in-extra
2020-02-17 00:22:34 -05:00
Dan Stillman
11caa1b719 Group strict-mode and non-strict-mode tests for Item::fromJSON() 2020-02-16 18:05:45 -05:00
Dan Stillman
b41734924d Further fixing of "Too many sync requests" error
Follow-up to 804a898c98

Addresses #1788
2020-02-16 18:05:45 -05:00
Dan Stillman
804a898c98 Hopefully fix "Too many sync requests" after file upload 412
Addresses #1788
2020-02-16 13:06:49 -05:00
Dan Stillman
a53f363b8d Additional fix for search crash with includeParentsAndChildren
Follow-up to 76081ab05
2020-02-11 13:09:54 -05:00
Dan Stillman
76081ab05f Fix crash when search uses no-op condition and includeParentsAndChildren
E.g., a nonexistent saved search
2020-02-11 00:23:45 -05:00
Dan Stillman
86a5c46b1e Find Available PDF: Don't mark URLs that redirect as tried
https://forums.zotero.org/discussion/81182
2020-02-02 23:47:40 -05:00
Dan Stillman
4e11c7927d Don't skip creator lines in Extra in fromJSON
Regression in 3de54455f6
2020-01-27 22:01:25 -05:00
Dan Stillman
3de54455f6 Automatically save unknown/invalid fields to Extra in non-strict mode
This is a prerequisite for starting to use new fields in translators,
since otherwise switching from, say, storing originalDate in Extra to
using an originalDate field would cause the value to be lost in clients
without the newer schema.

Closes #1504
2020-01-26 03:29:23 -05:00
Dan Stillman
3d2afa9c7f Additional adjustments to URL handling
Follow-up to df40ee7216

- Restore opening of non-HTTP URL schemes in link attachments
- Remove scheme whitelist for link attachments, since it's not enforced
  via the API anyway and we prompt before an external application is
  opened. Instead, just block a few schemes (e.g., 'javascript') from
  launching.

TODO:

- Provide some way to change/reset an application association if the
  user checks the box to automatically open that scheme.
- Show an error message if a link attachment with an invalid URL is
  double-clicked
2020-01-04 02:30:12 -05:00
Dan Stillman
6070743ff0 Fix Nextcloud WebDAV syncing
Add Zotero.HTTP.CookieBlocker mechanism to block storing and sending
of all cookies for a given URL prefix, and use that for the configured
WebDAV URL.

https://forums.zotero.org/discussion/80429/sync-error-in-5-0-80
2020-01-04 02:29:52 -05:00
Dan Stillman
12ccbf45c8 Make HTTP tests more reliable 2019-12-23 04:31:24 -05:00
Dan Stillman
9c380c362a Hopefully fix 503 from NextCloud (since 5.0.78)
https://forums.zotero.org/discussion/comment/344790/#Comment_344790
2019-12-22 16:32:36 -05:00