It shouldn't be possible for collections to be nested this way, if it
happens, it shouldn't result in an infinite loop.
This removes one of the parent assignments at sync time.
- Properly truncate and ellipsize long tags
- Show scrollbar if tags go off the bottom of the pane
These both improve on the pre-React version, which didn't properly
truncate tags (at least in the current version, though I think it used
to) and scrolled the Add button off the top of the screen.
I don't think this ever should've been here. It was added in 2010 but
then added to itembox.xml a couple weeks later in d500b50e2, since
that's where the conversion should happen.
Closes#1749
Improvements:
- Fixes autocomplete text remaining in field after selection in Fx60
- No more text or icon shifting on select (tested on macOS)
Changes:
- Tags are now selected on mousedown with no active state, as in web
library
Regressions:
- Tooltip with tag type doesn't appear when hovering over icon
- Pressing Tab after modifying a tag loses focus
- Right-click in textbox shows custom menu instead of default text
editing context menu (Cut/Copy/Paste)
To-do:
- Switch to this version for note tags box
- Style colored tags in autocomplete drop-down? Sort to top?
- Only show delete button on row hover, as in web library?
`intl.regional_prefs.use_os_locales` isn't actually equivalent to
`intl.locale.matchOS` -- an empty string for `intl.locale.requested` is,
and that's set by calling `setRequestedLocales(null)`.
Mozilla documentation:
https://firefox-source-docs.mozilla.org/intl/locale.html
- When upgrading from Fx52, don't show "Automatic" for
a manually selected locale.
- Make sure Zotero.locale is always set to a known language tag
- Fix inconsistencies between Zotero.Locale.availableLocales and
chrome.manifest
When we added the View menu pane options in 3aeef69ad3, we removed
automatic reopening of the panes on startup, since the View menu is more
discoverable than the narrow collapsed splitters. Unfortunately we're
now getting many reports of people not knowing where their panes are.
It turns out the View menu pane options in 5.0.76 are actually broken,
so it's not a great test, but enough people are doing this by accident
and enough of them don't seem to be finding the View menu options
(broken or otherwise) that I think it's better to return to reopening
the panes. We can always add a hidden pref to change that behavior, but
no one has complained in years, so it's probably not necessary.
This was a regression from 4b60c6ca27. The original plan for introducing
new fields was to have them save to the sync cache even if they weren't
supported by the current Zotero version and process them on upgrade, and
so I changed `DataObjectUtilities.patch()` to omit fields that didn't
exist locally when patching the sync cache JSON so they wouldn't be
wiped on the server. That caused this bug where locally deleted fields
were restored on every sync. It's also no longer necessary now that
we decided to just reject unknown fields from saving, so we can just
revert to the previous behavior of blanking out locally missing fields
(with the tweak that fields that are already false or empty in the base
version can be omitted).
- Include file path
- Show separate message for linked files mentioning Linked Attachment
Base Directory
- Link to separate support page for linked files
And fix a couple things for if we turn it back on
This code came along with the type/field handling overhaul, but I think
it was originally intended for handling unknown fields during sync
before we decided on strict mode, so it wasn't finished and causes
various problems [1]. It could still be useful for preserving fields
from translators before they're available on items, but the better fix
there is just to add the missing fields, so I'm not sure if we'll end up
needing it.
[1] https://groups.google.com/d/msg/zotero-dev/a1IPUJ2m_3s/hfmdK2P3BwAJ
We're not currently setting a timeout, so this shouldn't happen, but
someone [1] is getting a timeout error with undefined ms and
NS_BINDING_ABORTED from the channel. It also happened during a /fulltext
upload, so it's not limited to S3, but this fix should give a more
helpful error message for such errors during file syncing.
[1] https://forums.zotero.org/discussion/79286/
We're not migrating fields yet, but when we do, I think marking them as
changed will be the safer option. We'll just have to make sure that
conflicts without differences are resolved automatically on all
platforms.
Say "Use the [local|remote] version for all remaining conflicts" for
everything instead of saying "Use [local|remote] fields for all
remaining conflicts" for some conflicts.
This also fixes a test failure after 54343c49fb.
This allows Zotero.Prefs to be used instead of Services.prefs for pref
observing in plugins.
Zotero.Prefs.prefBranch was replaced by Zotero.Prefs.rootBranch.
Default prefs are no longer read automatically, so we need to do it
manually. Preferences may not be read before extensions load, so they
should wait for Zotero.Schema.schemaUpdatePromise or
Zotero.uiReadyPromise before initializing.
This changes the way item types, item fields, creator types, and CSL
mappings are defined and handled, in preparation for updated types and
fields.
Instead of being predefined in SQL files or code, type/field info is
read from a bundled JSON file shared with other parts of the Zotero
ecosystem [1], referred to as the "global schema". Updates to the
bundled schema file are automatically applied to the database at first
run, allowing changes to be made consistently across apps.
When syncing, invalid JSON properties are now rejected instead of being
ignored and processed later, which will allow for schema changes to be
made without causing problems in existing clients. We considered many
alternative approaches, but this approach is by far the simplest,
safest, and most transparent to the user.
For now, there are no actual changes to types and fields, since we'll
first need to do a sync cut-off for earlier versions that don't reject
invalid properties.
For third-party code, the main change is that type and field IDs should
no longer be hard-coded, since they may not be consistent in new
installs. For example, code should use `Zotero.ItemTypes.getID('note')`
instead of hard-coding `1`.
[1] https://github.com/zotero/zotero-schema
- Move 5xx retries and connection checking out of the sync API client
and into HTTP.request() so that they apply to all requests. 429 handling
remains in the API client, since not all callers necessarily want to
handle that the same way. Callers can still handle 5xx themselves by
including the relevant 5xx status codes in `successCodes` or by passing
`errorDelayMax: 0`.
- Add `cancellerReceiver` option, which is a callback that receives a
function that will cancel the request, whether it's an active request
or an automatic delay before a 5xx retry.
This also updates Sinon to 7.3.2.
- nsIURI is now immutable, so it's necessary to use nsIURIMutator via
mutate() to change it
- .path is replaced with .pathQueryRef
- Only nsIURL has .fileName
The XBL prefwindow bindings are removed in Firefox 60, so this adds them
back, along with necessary styling, to allow the existing preferences to
work until the preferences are rewritten with React. The preferences.xml
file in the Mozilla source has platform ifdefs, but since this is a
temporary hack I've just duplicated the file for each platform with the
necessary lines included.
I haven't yet tested the styling on Windows/Linux.
In particular, remove code related to opening/closing the Zotero pane,
which affects tests. The pane is now opened by default in Firefox, which
brings its behavior closer to the main version.
As of Fx60, XPathResult is no longer available as nsIDOMXPathResult in
XPCOM, so just shim its constants, which are all we need, when adding it
to the sandbox.
`nsIFilePicker::show()` is removed in Firefox 60 in favor of `open()`,
which takes a callback (and apparently has been preferred for a long
time).
There's no point switching to that, so this module is a version of
nsIFilePicker with an async `show()` that returns a promise and some
XPCOM-isms replaced (e.g., string paths instead of nsIFile).
Legacy/unsigned add-on warnings are now hidden via CSS in the Firefox
build, so remove the code that tries to do that, and instead add a class
to any other warnings so they can be selectively shown.
StopIteration is no longer supported in Firefox 60, so instead of taking
a generator function that might throw StopIteration for the second
parameter, take a function that is passed to iterator.forEach() that
receives an OS.File.DirectoryIterator.Entry for each directory entry. If
the function returns a promise, it's waited for.
Also update other direct uses of OS.File.DirectoryIterator to remove
StopIteration use.
nsIURL doesn't seem to work anymore, so add Zotero.Utilities.parseURL(),
which uses the `url` package from NPM and adds fileName, fileExtension,
and fileBaseName.
fetch_xulrunner.sh updates the built-in path to point to this file, but
it's not currently being read properly for some reason (and I'm not sure
it does anything we need).
- getStringPref/setStringPref are now used for strings instead of
getComplexValue/setComplexValue
- Remove nsIPrefBranch2 reference
- If there was a pref failure during initialization, nothing was logged
to the terminal
- The Mozilla CommonJS loader is no longer available, so bundle the
Fx52 version of it
- Strict mode is enforced
- `this` is only defined as a global object in .jsm files, not .js files
- `this` can't be converted to a string for BackstagePass test, so check
for presence of Components.utils.import instead
- The return value from import() is no longer available
The menu includes all user interface options from the General
preferences (now removed from the preferences), toggles for the
collections pane, item pane, and tag selector, and, at long last, a
toggle for recursive collections ("Display Items from Subcollections").
The collections pane and item pane no longer reopen automatically when
restarting Zotero. People might still close them by mistake and not find
this menu, but we'll see how it goes.
Closes#1372
When the filename limit isn't 255 on Linux, it's probably because of
eCryptfs, but we were checking the character length instead of the byte
length before shortening the filename to 143 bytes.
If a parent item was expanded and all items in the tree, including the
item's child items, were selected, Cmd + Left Arrow would break the
items tree until restart.
This will cause a conflict and need to be applied to the new tree.
(Worth noting that collapseSelectedRows() is currently only called on
Cmd-left-arrow because the XUL tree seemingly swallows a regular
left-arrow (and only applies the collapse to the last-selected row). The
comment in the keypress listener where collapseSelectedRows() is called
suggests that that was meant to be used for all left-arrow keypresses,
but either that stopped working at some point or it only ever worked for
Cmd + Left Arrow because it bypasses the normal tree handling of left-
arrow. In any case, it would be better if left-arrow always collapsed
selected rows, with or without Cmd.)
https://forums.zotero.org/discussion/78515/bug-collapsing-all-items-in-a-collection-breaks-display-of-items-in-all-collections
And move Error Console and Run JavaScript to a Developer menu that's
shown in Tools if that option is enabled.
The Memory Info button is now behind its own undefined
extensions.zotero.debug.memoryInfo pref.
Zotero.Translate::setTranslatorProviderMethods(methods) can be used to
provide custom 'get' and 'getAllForType' methods that override the
default Zotero.Translators methods.
I don't know how this is happening -- people are ending up with an
updated DB version but missing schema changes, despite the DB version
being updated after the schema updates and in the same transaction --
but until we figure it out, apply the schema update steps in
Zotero.Retractions.init() if necessary.
Separate flags for hiding the retraction altogether and for hiding
citation warnings for it
New functions:
Zotero.Retractions.hideRetraction(item)
Zotero.Retractions.shouldShowCitationWarning(item)
Zotero.Retractions.disableCitationWarningsForItem(item)
Addresses #1710
Without an accompanying "Restore from Online Library" option, which
doesn't currently exist, "Restore to Online Library" can be interpreted
two ways. It's clear if you read the text below or the confirmation
prompt, but if you don't you might accidentally do something very bad.
This was almost certainly also translated in misleading ways in other
locales.
If a retracted item was in the trash at startup or when detected and
there were no other retracted items in the library, an error would occur
if the item was modified.
This shouldn't be possible, but there've been a couple reports of people
ending up on version 103 without the table, so create it again with IF
NOT EXISTS. This is obviously a bad fix, but until we know how this
happened it's the best we can do.
- Fix don't-show-again checkbox for non-DB items
- Tweak warning text
- Don't show comma after year if year is missing (which it shouldn't be)
- Remove redundant try/catch
I'm not sure a double-quote actually appears in any DOIs, but it's
technically valid and would result in an invalid URL.
Follow-up to 782c2a1d1
Addresses #295
- Fix list download logic
- Close bar when clicking "View Item"
- Don't show "Move to Trash" if item isn't editable
- Download new list if cached version differs from client
- Reduce height of notification bar
- Switch to a slightly darker red
- Check for retracted items using data from Retraction Watch
- Show an X next to retracted items in the items list, and show a
scary message at the top of the item pane with more info and links.
- Lookup is done in a privacy-preserving manner using k-anonymity --
the server is unable to determine the specific items that exist in
the client, so people who don't sync don't need to share any library
data (though the server doesn't log the lookups anyway).
TODO:
- Pop up an alert when new items are found
- Show a confirmation prompt when citing a retracted item
- Support items without DOIs or PMIDs
- Add a proper PMID field and expand DOI to more item types so these
values don't need to be parsed out of Extra
- Clear the banner immediately when all possible fields are cleared
instead of waiting a few seconds
When doing Restore to Online Library from a backup that was never
file-synced, 'mtime' and 'md5' can be null, but we don't want to clear
existing properties on the server.
- Support items within collections and searches:
zotero://select/library/collections/:collectionKey/items/:itemKey
zotero://select/groups/:groupID/collections/:collectionKey/items/:itemKey
- Fix the 'itemKey' parameter:
zotero://select/library/collections/:collectionKey/items?itemKey=:itemKey1,:itemKey2
- Select library root if collection/search not specified
If an object changed on both sides and the changes were either
non-conflicting or identical but there were other local changes, the
local object was incorrectly being marked as synced, causing it not to
be uploaded until it was next modified locally.
- Revert to single-line mode after closing
- Don't add newline if Shift-Enter is pressed in an empty textbox
- Hide progress meter in multi-line mode until beginning search
Applied to all styles with ids matching 'apa($|-)'
Currently limited to colons, but Juris-M/citeproc-js#74 may extend this
to other punctuation
Closes#1681
This adds "English (UK)" to the locale list, which allows for "tag
colours" and allows dates to be recognized in d/m/y form.
I changed "color" and "license" on Transifex, but I'll leave other words
for others to change. If we stick to Oxford spelling, there probably
won't be too much else.
The error is triggered upon initial interaction with a doc after Zotero
restart or if new external citations (copied into the document) are
peresnt and `session.updateSession()` is called without a subsequent
`session.updateDocument()` call. `session.updateSession()` is called
without a subsequent `session.updateDocument()` call every time the
user cancels a citation insert.
More specifically, `session.updateSession()` is called every time a
citation dialog is invoked. It retrieves all citations and writes them
into a local `session.citationsByIndex` object. Moreover, it marks
each citation that hasn't seen before in a `session.newIndices` object.
`session.newIndices` is there to ensure that we load every new citation
into citeproc upon document update. This object is built by marking any
citation that does not appear in the previous invocation's list of
citations as new. However, if the document is never updated (because the
user cancels the insertion) then the new indices are not loaded
into citeproc. This commit fixes that, by excluding citeproc unloaded
items from the previous invocation's citation list.
- Don't cut off bottom of tags on Windows
- Fix appearance of search bar on Windows/Linux
- Fix cancel button on Windows/Linux
- Don't cut off right-hand pane of bottom bar when narrowing pane
- Use react-virtualized to render tags on demand, reducing the number
of DOM elements from potentially tens of thousands to <100. This
requires tags to be absolutely positioned, so sizing and
positioning need to be precomputed rather than relying on CSS.
- Avoid unnecessary refreshes, speed up tag retrieval, and optimize
sorting
- Debounce reflowing when resizing tag selector
Also:
- Scroll to top when changing collections
- Allow tags to take up full width of tag selector without truncation
Closes#1649Closes#281
Due to a typo in d0f7fd6df7, linked files were still being renamed even
with the pref off if metadata was found for the file. The test I added
was only for adding a file to an existing item, which didn't trigger
metadata retrieval.
This also adds a hook for stubbing the actual PDF recognition process so
we can test certain behaviors without making HTTP requests.
Since 9c0f5998a3, depending on your timezone the day could be off by
one if the access date didn't have a timestamp (so only for manual
entries or imports).
Using a regexp meant that an invalid regexp pattern would crash the tag
selector, and even if we caught that it would produce unexpected results
for some searches (e.g., anything with a period).
Even after cdf9d7ff32, the tag selector was still being initialized if
it was closed at startup, which meant that keeping it closed didn't fix
performance problems in large libraries. This hopefully finally brings
the tag selector in line with pre-Reactification behavior.
This also moves initIntlStrings() logic to Zotero.Intl so that strings
are accessible from React components in separate windows, and it moves
container initialization to ZoteroPane since most of what it does will
need to interact with ZoteroPane anyway.
This adds selectItems() to ZoteroPane and collectionTreeView and removes
the ancient, unused 'expand' argument to selectItem(), which didn't
really make sense there. It also includes a new
itemTreeView::ensureRowsAreVisible() that tries to scroll to an
appropriate place (or, better yet, not scroll at all) given the
specified rows and page size.
- Fix incorrect results for ANY search with multiple "Attachment
Content" conditions and no other conditions
- Dramatically speed up single-word searches by avoiding unnecessary
text scans (which probably addresses #1595)
- Clean up code
Adds a "Run as async function" checkbox that wraps the code in an async
function and displays the value returned by a `return` statement.
This also properly catches errors and displays them in the results pane.
This is a replacement for the Execute JS extension that could be used
with Zotero for Firefox.
To enable, go to the Config Editor in the Advanced pane of the
preferences and set devtools.chrome.enabled to true, and then restart
Zotero. A "Run JavaScript" option will appear in the Tools menu.
- Added icon-button UI code for the menubutton
- Upgrade to React 16 to allow non-standard attrs, such as `tooltiptext`
to support XUL tooltips
- Add i18n support for React UI elements
- Update tests for reactified tag selector
Invalid paths, including Windows UNC paths on other OSes, caused exports
to fail. Now they're ignored, which is what we do for other missing
attachment files.
Fixes#1622
Translators can include `proxy: false` in the attachment object to
indicate that the URL should be used as is, without further proxying.
This generally isn't necessary, but sometimes it is (e.g., on
EBSCOhost), and in theory we should start using this whenever a PDF URL
is extracted from the page instead of being constructed manually by the
translator.
Closes#1612
The 'application/pdf' Content-Type shouldn't have any parameters, but a
site was returning 'application/pdf;charset=ISO-8859-1', so instead just
look for the 'application/pdf' prefix.
This should have been obvious, but we've been seeing Terminal when testing
because we're launching Zotero from the Terminal. Users with installed
Zotero version actually see 'Zotero' in instead of Terminal.
Also:
- Actually use the cookie sandbox passed to processDocuments()
For zotero/translation-server#16, we want to include Accept-Language
(and maybe other headers) from the client request in upstream requests,
which requires passing it to both non-translate processDocuments().
translation-server's non-translate processDocuments() is defined in that
repo, but it's called by the cross-repo translate processDocuments() in
utilities_translate.js, so the signature needs to be changed in both
repos.
We also apparently weren't using the cookieSandbox in client
processDocuments() calls, though I think that only would've affected
translator testing.
This fixes direct and VPN-based retrieval of PDFs for Elsevier (e.g.,
ScienceDirect) items that have a DOI but no URL, since Elsevier resolves
DOIs through an intermediate page.
This reverts commit e526a8fa89.
This is a repeat of a0ca67d879. The reverted changes break Scaffold.
This will need to be fixed in a way that allows Scaffold to continue to
work.
In 2652fac24 we started failing the translator if doGet/doPost returned
a >=400 status code, but at least one translator (Primo) relied on doGet
continuing after a 404. This allows translators to specify the status
codes that are allowed, similar to the same argument in
Zotero.HTTP.request().
(We'll clean up the signature at some point or just get rid of these
functions, but this is a quick fix for Primo.)
I didn't look into the details, but `this` isn't defined when this file
is loaded within Scaffold, so this line caused an error and the
translator tester was broken.
Delay requests to the same domain by 1 second, respect a Retry-After
header if present for 429 and 503, and delay for 10 seconds on 429 or
5xx otherwise.
Currently only .status and .getResponseHeader() (for getting 'Location')
are available in the returned object, but we could make the body
available if necessary.
If there's no translated PDF or the translated PDF fails and the item
has a DOI, check Zotero's Unpaywall mirror for possible sources and try
to download one of those.
Unlike with "Add Item by Identifier" and "Find Available PDF" in the
item context menu, this does not try the DOI/URL page, since it would
result in more data leakage and most of the time you'd be saving from
the DOI page already. We could consider offering it as an option, but
for it to be useful, you'd have to have an institutional subscription,
be on-campus or connected via VPN (for now), and be saving from
somewhere other than the main page.
A new connector endpoint, sessionProgress, takes the place of
attachmentProgress. Unlike attachmentProgress, sessionProgress can show
new attachments that have been added to the save, and with a little more
work should also be able to show when a parent item has been recognized
for a directly saved PDF.
This also adds support for custom PDF resolvers, available to all PDF
retrieval methods. I'll document those separately.
Closes#1542
If you had Quick Copy set to a missing translator, had a site-specific
Quick Copy setting, and loaded a page in the browser, the next click on
an item would result in a restart error.
- Show a clearer message on a certificate error that includes the
underlying error, which should make debugging much simpler. (No more
checking in a browser and hoping it's the same connection.)
- Mention proxy server in message on startup proxy-check failure
- Include link to connection-error KB page on sync connection failure
Closes#1191Closes#1513
If a proxy is required and we can't connect to S3, show the (sync) error
icon. For SSL certificate errors, which are the most common cause of
this, the panel includes a link to the SSL cert troubleshooting page.
A submitted database had a text userID with a trailing "A0. Not sure how
that happened -- it doesn't appear to be possible in current code -- but
it caused group permissions not to be properly synced.
- Add the ability to extract a PDF URL from a given webpage using the
translation framework
- Add the ability to get open-access PDFs from landing pages from
Unpaywall data in addition to direct PDF URLs
- Use the above functionality to improve PDF retrieval for "Add Item by
Identifier"
- Add "Find Available PDFs" option to the item context menu to retrieve
PDFs for existing items from the DOI or URL page or using Unpaywall
data. The option appears for single items with a DOI or URL and no PDF,
and it always appears when selecting multiple top-level items (but
skips ineligible items).
PDF extraction from DOI/URL pages will currently only work with
unauthenticated access (i.e., on-campus or VPN, but not via a web-based
proxy).
Supersedes and closes#948
Add Z.DataDirectory.getSubdirectory which, optionally, creates
the directory.
Add async Z.DataDirectory.removeSubdirectory and use it for
Z.removeTempDirectory (was sync call before!).
Move `Zotero.getString()` and intl init code to `Zotero.intl` to make
it easier to re-use.
Link `Zotero.getString()` to `Zotero.intl.getString()`.
Do not expose `getStringFromBundle`, `pluralFormGet`, and
`pluralFormNumForms` because they are not used.
Handle an array of objects with 'url' and 'version' rather than just an
array of URLs.
Also:
- Don't throw an error from addOpenAccessPDF() if there's an error from
getOpenAccessPDFURLs()
- Make addPDFFromURLs() a separate function so URL lookup can be done
separately from download
