If there's no translated PDF or the translated PDF fails and the item
has a DOI, check Zotero's Unpaywall mirror for possible sources and try
to download one of those.
Unlike with "Add Item by Identifier" and "Find Available PDF" in the
item context menu, this does not try the DOI/URL page, since it would
result in more data leakage and most of the time you'd be saving from
the DOI page already. We could consider offering it as an option, but
for it to be useful, you'd have to have an institutional subscription,
be on-campus or connected via VPN (for now), and be saving from
somewhere other than the main page.
A new connector endpoint, sessionProgress, takes the place of
attachmentProgress. Unlike attachmentProgress, sessionProgress can show
new attachments that have been added to the save, and with a little more
work should also be able to show when a parent item has been recognized
for a directly saved PDF.
This also adds support for custom PDF resolvers, available to all PDF
retrieval methods. I'll document those separately.
Closes#1542
Shows a prompt once per restart or once per day, whichever is longer,
with an option to not show again for a month. Can be disabled completely
by setting extensions.zotero.showConnectorVersionWarning to false.
Currently prompts for connector versions below 5.0.35.
This is a temporary solution for #1489 until the connector checks and
warns on its own when it's outdated and most people are on a version
that does that (particularly Safari users, who don't auto-upgrade).
Configurable via extensions.zotero.fileHandler.pdf hidden pref for now,
though we'll probably make it a visible pref.
We also appear to have been doing blocking launches when launch()
failed, which may have been causing UI hangs when opening files on some
Linux installations. (I'm not sure if that's an issue with recent
Firefox builds. launch() works on Ubuntu 17.10.) All launches are now
async.
This is a rewritten version of PR #1450 by @ehhc.
Closes#1450
Applies to dragging to the collections pane or the items pane, adding
via New Item menu, or saving via the connector server
If the renaming pref is enabled, the PDF is renamed after recognition.
Can be disabled in the preferences
Closes#917
Automatic renaming is now done for dragging of an external file onto an
item (as added in 7cb95f41) as well as dragging as a linked file,
dragging a non-native (not text or HTML) URL, "Attach Link to File…",
"Attach Stored Copy of File…", and "Retrieve Metadata for PDF". It only
applies if a single file is being added and if the parent item has no
non-HTML attachments. By default, the renaming only applies to PDFs, but
this can be changed with the renameAttachmentFiles.automatic.fileTypes
hidden pref.
A new General pref, "Automatically rename attachment files using parent
metadata", controls whether the renaming happens.
Files saved via web translators are renamed regardless of this pref,
because they would often be gibberish otherwise.
Closes#113
And try all, since it doesn't do any of this unless there's a PAC file,
it's not actually a network request unless the site should be proxied,
and it should only make a single HEAD request unless a previous one
fails.
There's a possibility some proxies are limited to HTTP, in which case
changing these to HTTPS might stop this from working. If so, we'd need
to add back HTTP addresses.
Closes#1273
Previously, if a translator or style was fixed, people didn't get the
fix until their client checked the repository for updates, which could
take up to 24 hours. Now, in addition to checking once a day, we check
every time Zotero is started and also when we receive a notification
from the streaming server, which happens immediately after a translators
or style is updated on GitHub. To avoid DDoSing ourselves, the
notification includes a random delay (within a given period) before the
update is triggered by the client.
The streaming server connection is now made when either "Automatically
check for updated translators and styles" or "Sync automatically" is
enabled. It can be disabled via the extensions.zotero.streaming.enabled
pref.
- Add Cmd/Ctrl-Shift-I for "Import…"
- Cmd/Ctrl-Shift-V apparently pastes without formatting, so don't override it.
Instead, add a shortcut key to the menu option, defaulting to
Cmd-Option-Shift-I/Ctrl-Alt-Shift-I
- The I in both is now localizable
Immediate sync triggering on remote library change using WebSocket API.
Currently kicks off a normal sync process for the modified library --
actual object data isn't pushed.
(This might not stay enabled for 5.0 Final.)
* Mark feedItems read in a single batch SQL update
* Automatically remove old feed items
* User-facing preference globally and per-feed for feed item expiration
Instead, just always show the (greatly shortened, after 0609b62a) menu
on export when the translator supports it (which probably fewer
translators actually need to do).
This leaves the pref in place, but it can be removed eventually (and
changing the default for a while I believe will cause the existing pref
line to be removed for anyone who has it set to true already).
.persistentDescriptor now appears to return (and parse) a string path anyway on
macOS, which is the only place where it didn't use a string path to begin with,
so this will only affect earlier users.
- Allow clicking on non-editable abstract field to expand/collapse it
- Change cursor when hovering over abstract to show it can be toggled
- Default abstracts to expanded
- Fix persistence of last translation target
- Add checkbox to menuitem of selected target
- Remove unnecessary flex attributes
Also:
- Move collectionTreeView row id (e.g., "L1", "C123") and image
generation to Zotero.Library and Zotero.Collection properaties,
.collectionTreeViewID and .collectionTreeViewImage -- currently used
only for the feed add-to button, but could be expanded for use in
collectionTreeView
- Hide notes, tags and related for feed items in itembox
- Add feed support for <enclosure> elements
- Add feed syncing methods for synced settings (additional work is
needed on the sync architecture to download synced settings from the
server)
- Change feed item clear policy to be less aggressive
- Adjust for deasyncification
- Disable translate-on-select
- Closeadomasven/zotero#7, Remove context menu items from feeds
Display a warning when choosing data directory
Or on opening Zotero Pane for existing users with data dir in dropbox
Also:
Fix a bug where it won't use custom path if "Choose..." button is
pressed instead of radio button.
Change filepicker to show current data directory on display
Since modal windows (e.g., the Create Bib window and the Quick Copy site
editor window) can't use yield, style retrieval
(Zotero.Styles.getVisible()/getAll()) is now synchronous, depending on a
previous async Zotero.Styles.init(). The translator list is generated in
the prefs window and passed into the Quick Copy site editor, but it's
possible the translators API should be changed to make getTranslators()
synchronous with a prior init() as well.
- Add high-res webpage icon
- Show webpage icon in grayscale when no translator (except on hover, for fun)
- Remove pre-Australis icons
- Switch to CustomizableUI API for toolbar icon
- Move icon generation code to separate file
- Add Zotero.hiRes flag for Retina/etc. displays (available only after a window
has loaded)
Known issues:
- While the gray is mostly to be less distracting, the gray/color distinction
will probably be lost on most people. A separate guidance panel for the gray
icon might help.
- On pages with frames, the webpage icon appears first and then is replaced
with a translator icon.
Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
- Each column in the middle pane can now have its own persistent
secondary sort column, configurable from a new submenu in the column
picker menu (top right of items list). The settings are stored in
extensions.zotero.secondarySort.[primaryField]. The submenu title
includes the current primary field (e.g., "Secondary Sort (Creator)"),
which is pretty weird, and I'm not sure I want to keep it, but it does
convey that the setting is specific to the selected column.
- The fallback sort fields (firstCreator, date, title, dateAdded) are
now configurable via the extensions.zotero.fallbackSort. Setting that
pref to an empty string avoids all fallback sorts, which
allows reverse-order clicking to set the order, as requested by
@aurimasv in #275.
- The previous behavior of sorting based on the exact Creator string
(rather than the actual creators) can now be restored with the
extensions.zotero.sortCreatorAsString pref. (It simply circumvents all
the newer code, so it's pretty safe.) This setting should result in
faster sorting in large libraries that have many items with the same
Creator string.
- Some of the lesser fields in the column picker menu are now in the
More Columns submenu (which is now alphabetical)
- The "Type" column is now the less-ambiguous "Item Type".
- This uses a different method to modify the column picker menu that is
simultaneously less and more hacky. (It no longer has to duplicate
Mozilla code in a custom XBL binding that wouldn't reflect future
upstream changes, and instead it bushwhacks its way through various
boxObject properties to get to the underlying menupopup.)
FT sync is enabled by default for new users and configurable in the Sync
prefs.
Also disable downgrades once full-text sync is enabled, since otherwise
someone switching back and forth between old and new versions could miss
full-text content updates.
Trigger a proxy authentication prompt at startup if a PAC file is
installed and one of a few randomly chosen big sites requires a proxy.
This also improves general proxy detection by not making a request
to S3 unless it would actually be proxied.
It's not clear when this became an issue, but our usual background HTTP
requests (set that way to avoid triggering auth prompts when saving from
websites) weren't triggering proxy authentication dialogs, which was breaking
most network activity in Standalone. To fix this, we now make a foreground
request at startup to a file on S3 and resolve the Zotero.proxyAuthComplete
promise when we're done. Any network requests that want to wait for proxy
authentication can wait for that promise.
This behavior can be disabled via the triggerProxyAuthentication hidden pref.
Adds three new preferences controlling whether a new collection is created upon import:
- extensions.zotero.import.createNewCollection.fromFile
Controls whether a new collection is created by import from cog menu
- extensions.zotero.import.createNewCollection.fromClipboard
Controls whether a new collection is created by import from clipboard
- extensions.zotero.import.createNewCollection.fromFileOpenHandler
Controls whether a new collection is created when import is initiated by
double-clicking a file or by dragging it over Zotero. This preference applies only to
Zotero Standalone, and is configurable in the dialog that appears asking the user
to confirm the import.
Closes#19, [papercuts] (26) Clipboard import into an open collection
Falls back to /usr/bin/xdg-open by default, configurable via
extensions.zotero.fallbackLauncher.unix (or .windows, though that
should be much rarer). Also used for file revealing.
If xdg-open fails, files launch via the external helper app dialog and
URLs just don't work.
Can choose to download files "at sync time" or "as needed"
On-demand defaults to on, but remains off for existing users
To-do:
- Handling of local and remote file changes on on-demand download
(currently if a file exists it isn't downloaded, which means a
remotely modified file won't be redownloaded in on-demand mode)
- Additional control over file downloading and retention
Other changes:
- Overhauled entire file syncing architecture
- Replaced numAttachments column with Note and Attachment columns with
dynamic icons to indicate status
- Double-clicking a parent with a missing best attachment and on-demand
downloading off no longer loads the parent URL
- Bugs
Does not currently support sorting or citation editor, and has not been tested on any platform but OS X, so off by default for now. Can be enabled with the hidden pref extensions.zotero.integration.quickFormat.
- Closes#1831, Connectors should be able to save via API in the absence of Zotero Standalone
- Fixes Zotero.Utilities.deepCopy() for arrays
- Fixes some circumstances where an error would not be saved for future error reporting
- Fixes connector status checking
- Implement connector for Firefox (should switch in/out of connector mode automatically when Standalone is launched or closed, although this has only been tested extensively on OS X)
- Share core translation code between Zotero and connectors
Still to be done:
- Run translators in non-Fx connectors (this works in theory, but it's not currently enabled for any translators)
- Show translation results in non-Fx connectors
- Ability to translate to server when Zotero Standalone is not running
also adds a parameter to disable proxy redirection by domain (although if I had known how much work would be involved in implementing this before I started, I wouldn't have bothered)
- Store one item per IA bucket, with attachments stored as objects
- Use proper mediatype field based on Zotero item type
- Commons list is now pulled dynamically based on RDF stored at IA, without need for corresponding local item (which may have been deleted, etc.)
- Once available, OCRed PDFs can be pulled down by right-clicking on Commons and selecting Refresh
- Downloaded OCRed PDFs are now named the same as the existing attachment, with "(OCR)" appended
- The relations table is used to link downloaded OCRed PDFs to the IA file, so the downloaded file can be renamed without triggering another download
- The Commons view is marked for automatic refresh after an item is uploaded
- Added some progress notifications, though more are probably needed
- Other things
Also:
- Added Zotero.File.getBinaryContents(file)
- Erase an item's relations when the item is deleted, and purge orphaned ones
- Zotero.URI.eraseByPathPrefix(prefix) no longer prepends 'http://zotero.org' (which has been moved to Zotero.URI.defaultPrefix)
- New function Zotero.URI.eraseByURI(prefix)
Known Issues:
- Slow (some IA changes should be able to speed it up)
- Identifier format is likely temporary
- Sometimes it stops during setTimeout() calls for no apparent reason whatsoever
- Didn't test items with multiple attachments
- Not sure if Commons view will auto-refresh if you switch to it before the upload is done
- IA translator not yet updated
- Deleting items not supported by IA
- Date Added/Date Modified don't show up properly in Zotero for Commons items
- Group file sync via Zotero File Storage
- Split file syncing into separate modules for ZFS and WebDAV
- Dragging items between libraries copies child notes, snapshots/files, and links based on checkboxes for each (enabled by default) in the Zotero preferences
- Sync errors now trigger an exclamation/error icon separate from the sync icon, with a popup window displaying the error and an option to report it
- Various errors that could cause perpetual sync icon spinning now stop the sync properly
- Zotero.Utilities.md5(str) is now md5(strOrFile, base64)
- doPost(), doHead(), and retrieveSource() now takes a headers parameter instead of requestContentType
- doHead() can now accept an nsIURI (with login credentials), is a background request, and isn't cached
- When library access or file writing access is denied during sync, display a warning and then reset local group to server version
- Perform additional steps (e.g., removing local groups) when switching sync users to prevent errors
- Compare hash as well as mod time when checking for modified local files
- Don't trigger notifications when removing groups from the client
- Clear relation links to items in removed groups
- Zotero.Item.attachmentHash property to get file MD5
- importFromFile() now takes libraryID as a third parameter
- Zotero.Attachments.getNumFiles() returns the number of files in the attachment directory
- Zotero.Attachments.copyAttachmentToLibrary() copies an attachment item, including files, to another library
- Removed Zotero.File.getFileHash() in favor of updated Zotero.Utilities.md5()
- Zotero.File.copyDirectory(dir, newDir) copies all files from dir into newDir
- Preferences shuffling: OpenURL to Advanced, import/export character set options to Export, "Include URLs of paper articles in references" to Styles
- Other stuff I don't remember
Suffice it to say, this could use testing.
- dropping Zotero items into a bucket puts them in that IA bucket
- double clicking a bucket takes you to that IA bucket
In order to enable Zotero Commons:
1) Get an access key and secret key at http://www.archive.org/account/s3.php
2) Go to about:config
3) Search "commons" (no quotes)
4) Set "extensions.zotero.commons.enabled" to true
5) Enter your S3 access key into "extensions.zotero.commons.accessKey"
6) Enter your S3 secret key into "extensions.zotero.commons.secretKey"
7) Enter your buckets into "extensions.zotero.commons.buckets" as a comma separated list
Note: Steps 4-7 take effect in new windows
- Make note font configurable through extensions.zotero.note.fontFamily hidden pref (for now)
- Additional CSS rules can be added through extensions.zotero.note.css hidden pref
- Give TinyMCE note body #zotero-tinymce-note id for more specific external customization
- Remove status bar position option from visible preferences...
