Commit graph

899 commits

Author SHA1 Message Date
Dan Stillman
679148d8d6 Don't store "Version Number" in Extra for books coming from RDF
The RDF translator assigns 'versionNumber' in addition to 'edition'.
2020-03-24 14:48:09 -04:00
Dan Stillman
d3601bba24 Remove incorrect test for Extra 'type:' extraction 2020-03-23 19:09:59 -04:00
Dan Stillman
52d5b68564 Update 'type:' handling when migrating Extra lines
- When changing type based on 'type:' line, move existing fields that
  are no longer valid to Extra
- Remove 'type:' line with CSL type if the item's existing type is one
  of the types mapped to it
2020-03-22 15:19:24 -04:00
Dan Stillman
02b43cbfac Refactor migrateExtraFields() tests 2020-03-21 00:56:55 -04:00
Dan Stillman
592188bfc5 Disabled test for keeping unfiled item selected on add to collection
A fix will need to wait for the Reactified items tree

Addresses #1799
2020-03-19 13:08:23 -04:00
Dan Stillman
f1816c12ad Fix test from 28ea77dae 2020-03-17 14:17:02 -04:00
Dan Stillman
28ea77dae1 Add test for ignoring invalid item type in Extra 2020-03-17 13:51:19 -04:00
Dan Stillman
3f2998bb6b Don't extract 'type: note' or 'type: attachment' from Extra 2020-03-17 13:50:45 -04:00
Dan Stillman
d8d07362ba Add additional test for creator migration from Extra 2020-03-17 10:37:16 -04:00
Dan Stillman
dd9ec54c3b Don't try to migrate Extra fields in read-only libraries 2020-03-15 19:52:56 -04:00
Dan Stillman
23c936f016 Fix creator migration from Extra 2020-03-15 13:51:04 -04:00
Dan Stillman
0679809735 Fix invalid collection nesting in DB integrity check
It shouldn't be possible to nest two collections inside each other, but
if it happens, fix it in the integrity check.

Also detect it from CollectionTreeView::expandToCollection() (used when
showing the collections containing an item) and crash Zotero with a flag
to run an integrity check after restart. Previously, this would result
in an infinite loop.

This may be the cause of some of the collection disappearances people
have reported. If parentCollectionID never leads to a null, the
collection won't appear anywhere in the tree.

TODO:

- Figure out how this is happening
- Detect and fix it automatically for people it's happened to
2020-03-13 18:07:20 -04:00
Dan Stillman
9175f9ade8 Don't use "Create Bib" locale for Quick Copy when no explicit locale set
If you'd never set a Quick Copy locale, the option would show as using
the current locale, but Quick Copy itself would use the last locale from
"Create Bibliography from Items". That was a side effect of behavior we
put in place in 2015 so that documents created before 4.0.27 that relied
on the removed bibliographyLocale pref would continue using the migrated
locale, but now that we've had an explicit locale option for years in
the document preferences I think we can stop doing that.
2020-03-13 17:06:29 -04:00
Dan Stillman
d389a71280 Download remotely updated files in "as needed" file sync mode
Previously, files updated remotely wouldn't be downloaded in "as needed"
mode if a copy of the file already existed locally and could only be
re-downloaded by deleting the file via Show File.

This causes remotely modified files that exist locally to be downloaded
at sync time, even in "as needed" mode, by marking them as
"force_download". While this might not be ideal for people who use "as
needed" to limit data transfer, it's better for people who use it simply
to limit local storage, and ending up with an outdated file while
offline seems worse than a little bit of extra data transfer.

In the future, we'll likely also provide ways to explicitly download and
remove files, so keeping chosen files in sync makes sense.

Files modified remotely before this change (which were marked as
"to_download" instead of "force_download") won't be downloaded as sync
time in "as needed" mode, but they'll now be re-downloaded on open.

Fixes #1322
2020-03-09 01:19:52 -04:00
Dan Stillman
76a1535a60 Full-text indexing improvements
- Use full-text cache file from syncing if available when reindexing via
  info pane or Rebuild Index → Index Unindexed Items. Only discard it for
  full index rebuild. This allows Index Unindexed Items to be used to
  force immediate processing of queued content from syncing and avoids
  unnecessary syncing back of identical content. Previously, the cache
  file was used for a manual index only when the local file didn't exist.
- When rebuilding index, don't clear indexed items with missing local
  file that are missing stats due to a pre-411180ef bug.
- indexItems() now takes an 'options' object as its second parameter
- Minor code cleanup
2020-03-09 01:19:52 -04:00
Dan Stillman
411180ef83 Full-text indexing fixes
- Don't clear item's index stats (and show "Unknown") when an item is
  reindexed remotely and the content matches the local content
- Always update an item's state and its stats in the same query, to
  avoid incorrect feedback immediately after indexing
- Clean up `setItemContent()` tests
2020-03-06 03:11:16 -05:00
Dan Stillman
da5e8c549e Convert Zotero fields in Extra to CSL fields for citeproc-js 2020-03-02 01:38:10 -05:00
Dan Stillman
83cc65eea6 Fix parsing of CSL date fields in extractExtraField() 2020-03-02 01:34:27 -05:00
Dan Stillman
a9dee2f487 Item::fromJSON(): Remove invalid-for-type Type-mapped fields
"Type: [val]` in Extra means Item Type to citeproc-js, and Type values
from translators mostly aren't going to be useful if the item type
doesn't have a Type-mapped field.

https://forums.zotero.org/discussion/comment/348864/#Comment_348864
2020-02-17 11:39:35 -05:00
Dan Stillman
b2bf60e1d7 Item.fromJSON(): Dedupe invalid-for-type fields when storing in Extra
3de54455f6 removed redundant base-mapped fields when a valid-for-type
field was also set, but that still left duplicate fields in Extra when a
valid field wasn't set. This will happen until translators (most notably
Embedded Metadata) are fixed to stop setting redundant fields.

https://forums.zotero.org/discussion/81262/translator-error-sage-lots-of-extra-data-in-extra
2020-02-17 00:22:34 -05:00
Dan Stillman
11caa1b719 Group strict-mode and non-strict-mode tests for Item::fromJSON() 2020-02-16 18:05:45 -05:00
Dan Stillman
b41734924d Further fixing of "Too many sync requests" error
Follow-up to 804a898c98

Addresses #1788
2020-02-16 18:05:45 -05:00
Dan Stillman
804a898c98 Hopefully fix "Too many sync requests" after file upload 412
Addresses #1788
2020-02-16 13:06:49 -05:00
Dan Stillman
a53f363b8d Additional fix for search crash with includeParentsAndChildren
Follow-up to 76081ab05
2020-02-11 13:09:54 -05:00
Dan Stillman
76081ab05f Fix crash when search uses no-op condition and includeParentsAndChildren
E.g., a nonexistent saved search
2020-02-11 00:23:45 -05:00
Dan Stillman
86a5c46b1e Find Available PDF: Don't mark URLs that redirect as tried
https://forums.zotero.org/discussion/81182
2020-02-02 23:47:40 -05:00
Dan Stillman
4e11c7927d Don't skip creator lines in Extra in fromJSON
Regression in 3de54455f6
2020-01-27 22:01:25 -05:00
Dan Stillman
3de54455f6 Automatically save unknown/invalid fields to Extra in non-strict mode
This is a prerequisite for starting to use new fields in translators,
since otherwise switching from, say, storing originalDate in Extra to
using an originalDate field would cause the value to be lost in clients
without the newer schema.

Closes #1504
2020-01-26 03:29:23 -05:00
Dan Stillman
3d2afa9c7f Additional adjustments to URL handling
Follow-up to df40ee7216

- Restore opening of non-HTTP URL schemes in link attachments
- Remove scheme whitelist for link attachments, since it's not enforced
  via the API anyway and we prompt before an external application is
  opened. Instead, just block a few schemes (e.g., 'javascript') from
  launching.

TODO:

- Provide some way to change/reset an application association if the
  user checks the box to automatically open that scheme.
- Show an error message if a link attachment with an invalid URL is
  double-clicked
2020-01-04 02:30:12 -05:00
Dan Stillman
6070743ff0 Fix Nextcloud WebDAV syncing
Add Zotero.HTTP.CookieBlocker mechanism to block storing and sending
of all cookies for a given URL prefix, and use that for the configured
WebDAV URL.

https://forums.zotero.org/discussion/80429/sync-error-in-5-0-80
2020-01-04 02:29:52 -05:00
Dan Stillman
12ccbf45c8 Make HTTP tests more reliable 2019-12-23 04:31:24 -05:00
Dan Stillman
9c380c362a Hopefully fix 503 from NextCloud (since 5.0.78)
https://forums.zotero.org/discussion/comment/344790/#Comment_344790
2019-12-22 16:32:36 -05:00
Dan Stillman
b7dc0d8b0a Fix parsing of pre-1000 years
If a leading zero (e.g., '068'), parse as actual year instead of
19xx/20xx.
2019-12-19 02:15:36 -05:00
Dan Stillman
df40ee7216 Adjust URL launching behavior
- Support launching plausible HTTP URLs without schemes from
  Zotero.loadURI(), and use that when launching from URL field
- Show correct cursor feedback on URL label -- only show pointer if the
  URL is launchable
- Don't launch non-HTTP URLs (e.g., zotero://) from URL field
- Don't open HTTP URLs in viewer when using ZoteroPane.launchURI()
2019-12-16 14:56:06 -05:00
Dan Stillman
bbbd02444b Restore 'yesterday'/'today'/'tomorrow' parsing for dates in searches
Follow-up to a549a64de9, which removed it from strToDate()
2019-12-06 03:12:48 -07:00
Dan Stillman
61cebbd8f7 Fix display of a couple WebDAV verification errors
"spec is undefined"

Fixes #1745
2019-12-01 03:39:54 -07:00
Dan Stillman
fbb54a7621 Fix "string is undefined" export error
Regression from a549a64de9
2019-11-29 01:50:43 -07:00
Dan Stillman
8c7677a009 Fix out-of-memory error syncing collections nested inside each other
It shouldn't be possible for collections to be nested this way, if it
happens, it shouldn't result in an infinite loop.

This removes one of the parent assignments at sync time.
2019-11-26 15:30:43 -07:00
Dan Stillman
5791ffeb16 Reactify item tags box
Improvements:

- Fixes autocomplete text remaining in field after selection in Fx60
- No more text or icon shifting on select (tested on macOS)

Changes:

- Tags are now selected on mousedown with no active state, as in web
  library

Regressions:

- Tooltip with tag type doesn't appear when hovering over icon
- Pressing Tab after modifying a tag loses focus
- Right-click in textbox shows custom menu instead of default text
  editing context menu (Cut/Copy/Paste)

To-do:

- Switch to this version for note tags box
- Style colored tags in autocomplete drop-down? Sort to top?
- Only show delete button on row hover, as in web library?
2019-11-08 06:41:06 -05:00
Dan Stillman
cbfa4be437 Add 'Type:' to extraToCSL() test
Follow-up to #1744
2019-10-26 17:05:45 -04:00
Dan Stillman
c25fbe7c1c Remove all colored tags on selected items if 0 is pressed
Like Thunderbird
2019-10-26 16:45:04 -04:00
Dan Stillman
52fd91950d Fix deleted item fields reappearing on sync
This was a regression from 4b60c6ca27. The original plan for introducing
new fields was to have them save to the sync cache even if they weren't
supported by the current Zotero version and process them on upgrade, and
so I changed `DataObjectUtilities.patch()` to omit fields that didn't
exist locally when patching the sync cache JSON so they wouldn't be
wiped on the server. That caused this bug where locally deleted fields
were restored on every sync. It's also no longer necessary now that
we decided to just reject unknown fields from saving, so we can just
revert to the previous behavior of blanking out locally missing fields
(with the tweak that fields that are already false or empty in the base
version can be omitted).
2019-10-24 01:22:00 -04:00
Dan Stillman
513f7d6555 Fx60: Fix localization
Also fixes #1690 (at least on Catalina), and possibly other things

general.useragent.locale and intl.locale.matchOS are no longer used.
2019-10-23 19:08:13 -04:00
Dan Stillman
d4f682aa88 Merge branch 'fx60' 2019-10-21 21:44:01 -04:00
Dan Stillman
956813ac1f Fix "err is undefined" on 200 response for nonexistent WebDAV file
Fixes #1741
2019-10-20 15:03:18 -04:00
Dan Stillman
1710eb1c4b Don't store unknown/invalid fields in Extra in non-strict mode
And fix a couple things for if we turn it back on

This code came along with the type/field handling overhaul, but I think
it was originally intended for handling unknown fields during sync
before we decided on strict mode, so it wasn't finished and causes
various problems [1]. It could still be useful for preserving fields
from translators before they're available on items, but the better fix
there is just to add the missing fields, so I'm not sure if we'll end up
needing it.

[1] https://groups.google.com/d/msg/zotero-dev/a1IPUJ2m_3s/hfmdK2P3BwAJ
2019-10-18 03:37:24 -04:00
Dan Stillman
89672ed0a4 Fix "c1 is undefined" sync CR error 2019-10-09 18:29:04 -04:00
Dan Stillman
44d4586242 Fix test after 8aeb6f7fe4 2019-09-28 07:40:01 -04:00
Dan Stillman
5723683b3b Fix "attachmentSyncedModificationTime must be a number" sync error
https://forums.zotero.org/discussion/79011/zotero-error-report
2019-09-21 01:59:09 -04:00
Dan Stillman
4b7cdddb4a Change apply-to-all checkbox text for some conflicts
Say "Use the [local|remote] version for all remaining conflicts" for
everything instead of saying "Use [local|remote] fields for all
remaining conflicts" for some conflicts.

This also fixes a test failure after 54343c49fb.
2019-09-21 01:35:36 -04:00