Instead of My Publications being a separate library, have it be a
special collection inside My Library. Top-level items can be dragged
into it as before, and child items can be toggled off and on with a
button in the item pane. Newly added child items won't be shown by
default.
For upgraders, items in the My Publications library will be moved into
My Library, which might result in their being duplicated if the items
weren't removed from My Library. The client will then upload those new
items into My Library.
The API endpoint will continue to show items in the separate My
Publications library until My Publications items are added to My
Library, so the profile page will continue to show them.
This allows for imported items to be saved individually instead of being
saved in a batch after processing the entire imported file (which for
large imports would hang the UI, even if the actual saving was
asynchronous). This also fixes the progress meter during asynchronous
saves.
To take advantage of this, import translators will need to return a
promise when available (using the native Promise object) from doImport()
and wait for optional promises from item.complete().
The logic here can probably be streamlined further. (E.g., we might be
able to say that item.complete() always returns a promise.) It's
complicated by the fact that, at the moment, Promise isn't available in
child sandboxes, though this can probably be fixed.
Tests forthcoming, but they require a translator that supports this,
which needs to be committed separately.
View with -w for a cleaner diff.
- Add Cmd/Ctrl-Shift-I for "Import…"
- Cmd/Ctrl-Shift-V apparently pastes without formatting, so don't override it.
Instead, add a shortcut key to the menu option, defaulting to
Cmd-Option-Shift-I/Ctrl-Alt-Shift-I
- The I in both is now localizable
- Make version copyable
- Include "What's new" link to changelog
- Update some links and descriptions
To-do:
- Replace dev credits with a nice statement like in the Firefox About pane
And change "Changes take effect in new windows" to "Changes take effect
after restart"
(The note font size still requires a restart, as do the shortcut keys in
the Advaned pane.)
And treat an existing non-empty directory as a failure during the
migration, since it should now never happen
Also suggest manually moving on Windows if more than 100 attachments
* Mark feedItems read in a single batch SQL update
* Automatically remove old feed items
* User-facing preference globally and per-feed for feed item expiration
- Don't block the UI with a progress meter during imports. Instead, show
a popup in the bottom right when the import is done that shows how
many items were saved.
- Fix hang when importing some files
- Fix various problems with asynchronous operations/transactions
- Use the save queue for imports instead of creating concurrent
transactions that can time out
- Wait for the save to finish before returning from the translate()
promise. All save modes now use the save queue, so code that
handled the non-save-queue process can probably be removed.
- Serialize child attachments instead of running them concurrently.
This might make multi-attachment saves a little slower, since they
can't download at the same time, but it avoids problems with
concurrent transactions. We might be able to improve this to allow
concurrent downloads, or allow concurrent saves for a limited
number of items (e.g., from web saving) if not for larger imports.
- Change collection handling during import, since UI is now active
- Select the root collection at the beginning of the import
- Assign items and collections to the root during the import instead
of at the end
- Don't select other collections
- Change a few ItemSaver functions to use promises and remove
unnecessary callbacks. (This includes some connector code that needs
to be tested.)
- Change some `parentID` variables in ItemSaver to `parentItemID` for
clarity, since collections are now handled in more places
To-do:
- Save items in smaller batches instead of doing all in the same
transaction
- Show progress meter in a bottom-right popup during the import
It would be good for people not to kill the process indiscriminately, but the
migration will theoretically pick up where it left off, so it shouldn't be too
big a deal.
Would be nice to have a determinate progress meter on Windows, but that
will require enumerating all files, which maybe would just make things
slower. Might be safer, though, by discouraging people from
force-quitting.
This adds a new button to the Advanced prefs to migrate the data directory to
$HOME/Zotero. The button only appears if the data directory is set to the
default location within a profile directory (including the other program from
the one running, even though that's technically stored as a custom data
directory).
On Mac/Linux, directories within the data directory are moved with /bin/mv. On
Windows, or if that fails, they're copied recursively using OS.File.move()
(which annoyingly doesn't reliably support directory moving). The former should
be instantaneous on most systems (unless the data directory or 'storage' were
on a different filesystem from $HOME).
If the database fails to transfer, migration fails and the data directory
setting remains on the old directory. If the database transfers but other files
fail, the data directory setting is updated. In both cases, the user is
encouraged to migrate remaining files manually with a button that reveals the
directories and quits the program.
This isn't yet tested on Linux or Windows, and migration isn't yet suggested
automatically.
Adds Zotero.File.reveal(), Zotero.File.directoryIsEmpty(), and
Zotero.File.moveDirectory().
- Use $HOME/Zotero if zotero.sqlite exists, and set it as a custom data
directory so that 4.0 uses it if loaded (resulting in a
newer-db-version error instead of an empty database)
- Don't prompt whether to use data directory from the other version's
profile directory (i.e., Firefox or Standalone) -- just do it
This could be extended to address #118/#145, but for now it makes some of the
more common search conditions (Creator, Collection, etc.) more prominent.
Clicking it cancels the current window, opens the Cite pane of the
prefs, and selects the Styles tab. (This will be more useful once we
have inline style installation from that pane.)
On reset, items are overwritten with pristine versions if available and deleted
otherwise, and then the library is marked for a full sync. Unsynced/changed
files are deleted and marked for download.
Closes#1002
Todo:
- Handle API key access change (#953, in part)
- Handle 403 from data/file upload for existing users (#1041)
This is necessary because you can copy a database synced with a
different account into the data directory without affecting the stored
pref.
Also tweak the text to use proper quotes and remove quaint references to
"the server".
- Only show first pane (without Sharing pane) if no files or files
aren't included
- Update authorship checkbox to reflect file include setting to avoid
confusion
- Clarify in intro text that license applies to files, not notes --
notes here are no different than notes in any public library, so it's
sort of up to the user to clarify those if they're substantial enough
for it to matter
- Adjust alignment of authorship checkbox if more than one line
The on-change auto-sync now syncs only the modified library, and does so
quite efficiently (and should be able to be made more efficient), so we
might be able to reduce the timeout below 15 seconds.
- Hide notes, tags and related for feed items in itembox
- Add feed support for <enclosure> elements
- Add feed syncing methods for synced settings (additional work is
needed on the sync architecture to download synced settings from the
server)
- Change feed item clear policy to be less aggressive
- Adjust for deasyncification
- Disable translate-on-select
- Closeadomasven/zotero#7, Remove context menu items from feeds
Fixes the incompatible-version dialog and adds info on the version used
to upgrade the database, which may be helpful for troubleshooting.
Also fixes showing of the Zotero toolbar icons even in case of a startup
error.
Also:
- Remove last-sync-time mechanism for both WebDAV and ZFS, since it can
be determined by storage properties (mtime/md5) in data sync
- Add option to include synced storage properties in item toJSON()
instead of local file properties
- Set "Fake-Server-Match" header in setHTTPResponse() test support
function, which can be used for request count assertions -- see
resetRequestCount() and assertRequestCount() in webdavTest.js
- Allow string (e.g., 'to_download') instead of constant in
Zotero.Sync.Data.Local.setSyncState()
- Misc storage tweaks
Improves UX of sync authentication.
The account is now linked and unlinked and an API key related to
the client is generated transparently in the background.
The API key is deleted on unlinking.
No sync options are allowed before linking an account.
Display a warning when choosing data directory
Or on opening Zotero Pane for existing users with data dir in dropbox
Also:
Fix a bug where it won't use custom path if "Choose..." button is
pressed instead of radio button.
Change filepicker to show current data directory on display
Disallow everything other than adding a child note in My Publications,
since the wizard only shows on drag (currently, at least). Need to
disable translation separately.
Also show "Delete Item[s]..." in the trash context menu, and hide rather
than disable various menu options.
Addresses #703 -- still need to disable saved search saving
This mostly gets ZFS file syncing and file conflict resolution working
with the API sync process. WebDAV will need to be updated separately.
Known issues:
- File sync progress is temporarily gone
- File uploads can result in an unnecessary 412 loop on the next data
sync
- This causes Firefox to crash on one of my computers during tests,
which would be easier to debug if it produced a crash log.
Also:
- Adds httpd.js for use in tests when FakeXMLHttpRequest can't be used
(e.g., saveURI()).
- Adds some additional test data files for attachment tests
Also:
* _finalizeErase in Zotero.DataObject is now inheritable
* Call _initErase before starting a DB transaction
* removes Zotero.Libraries.add and Zotero.Libraries.remove (doesn't seem like this is used any more)
This will appear much less frequently, since non-conflicting field changes on
both sides can be resolved automatically, but genuine field conflicts still
require manual conflict resolution.
The merge pane is no longer editable, since the itembox code to do that is
async and can't run in a modal window, but it's not really necessary,
particularly with conflicts happening less frequently.
TODO:
- Remote item deletions
- File conflicts
- Maybe handle some edge cases where the conflicted items fail to save
There's a lot more to do, and this isn't ready for actual usage, but the
basic functionality is mostly in place and has decent test coverage. It
can successfully upgrade a library last used with classic syncing and
pull down changes via the API. Uploading mostly works but is currently
disabled for safety until it has better test coverage.
Downloaded JSON is first saved to a cache table, which is then used to
populate other tables and later for generating PATCH requests and
automatically resolving conflicts (since it shows what was changed
locally and what was changed remotely). Objects with unmet dependencies
or unknown fields are skipped for now but don't block the rest of the
sync.
Some of the bigger remaining to-dos:
- Tests for uploading
- Re-do the preferences to get an API key
- File sync integration
- Full-text syncing integration
- Manual conflict resolution (though this already includes much smarter
conflict handling that automatically resolves many conflicts)
Since modal windows (e.g., the Create Bib window and the Quick Copy site
editor window) can't use yield, style retrieval
(Zotero.Styles.getVisible()/getAll()) is now synchronous, depending on a
previous async Zotero.Styles.init(). The translator list is generated in
the prefs window and passed into the Quick Copy site editor, but it's
possible the translators API should be changed to make getTranslators()
synchronous with a prior init() as well.
For new installations, a panel will show for the Z icon with a ">"
button to go to a second panel for the save icon. For existing
installations, a panel will show just for the save icon. In both cases,
the panels are non-auto-hiding but are dismissed if clicked on or when a
new content page is loaded.
There's also a new close button, but it's not enabled currently. Might
be useful for panels with embedded links where an accidental click on
the panel alone shouldn't close the panel (like the Firefox customize
mode wizard).
- When relinking a missing stored file, copy it into the attachment's
storage directory automatically
- Previously, selecting a file outside the attachment subdir would
just result in a missing attachment, since it only looks for stored
files within the subdir
- Display an error message if a Windows shortcut (.lnk) is added via
drag-and-drop or via a file dialog on non-Windows systems, until we
can figure out how to determine the original file
- Shortcuts can cause errors during syncing, for unclear reasons
- Neither nsIFile::copyToFollowingLinks() nor nsIFile::target work for
me to get the original file, even when nsIFile::isSymlink() returns
true
- Windows file dialogs seem to automatically resolve shortcuts, so
it's only an issue there for drag-and-drop
- Disallow hidden files from being selected in relink dialog
- I think some people on Windows with hidden files shown relink the
.zotero* files that show up when they click Locate, which causes
file sync errors. Which brings us to...
- Fix file sync errors for *.lnk and .zotero* files
- Ignore existing .zotero* attachment files, treating the files as
missing instead to encourage relinking
- Strip leading period in getValidFileName() to prevent added files from
being hidden
- This allows hidden files to be added explicitly; they just won't
stay that way in the storage directory
(These things should have tests, but that will have to happen on the 5.0
branch.)
- Simplified schema
- Tags are now added without reloading entire tag selector
- On my system, adding 400 tags to an item (separately, with the tag
selector updating each time) went from 59 seconds to 42. (Given that
it takes only 13 seconds with the tag selector closed, though,
there's clearly more work to be done.)
- Tag selector now uses HTML flexbox (in identical fashion, for now, but
with the possibility of fancier changes later, and with streamlined
logic thanks to the flexbox 'order' property)
- Various async fixes
- Tests
And use 'version' instead of 'itemVersion' for object version for items
Also add deferred foreign key checking to system.sql so that DROP TABLE
commands don't fail mid-transaction
Show a wizard after items are dragged to My Publications choosing
whether to include files and notes and choosing sharing settings for the
items. Sharing options are Creative Commons licenses, CC0, "All rights
reserved", or keeping the existing Rights field if available.
Also blocks collections, searches, linked file attachments, and
top-level attachments/notes from My Publications at the data layer, but
not yet from the UI in all places (so it can crash if you try).
Todo:
- Block certain UI actions with nice messages
- Show a nice scrollable list of items in the wizard to allow selecting
specific files/notes, instead of just having checkboxes for files and
notes that apply to all dragged items
- Show an explanation of My Publications in the right-hand pane when no
items are selected
- Maybe adjust handling when no attached files/notes, since it might be
a bit alarming at the moment to see sharing options for metadata
entries
Note that this loses conflicting changes to translate_item.js from 849803473a,
so those will need to be reapplied if applicable. /cc @aurimasv, @mtd91429
Adds a "My Publications" source after "My Library", implemented as a
separate library. Top-level items can be dragged in and removed.
(This doesn't currently work without disabling Quick Copy.)
Also:
- Make "Group Libraries" an unselectable header instead of a container,
and don't indent group libraries
- Fix relation purging, which maybe never worked
- Pass only libraryID/key on deletes (which should speed them up)
- Fix async item cloning/copying
- Fix miscellaneous other bugs
To-do:
- Confirmation dialog on drag
- API support
Fixes#426: this allows the localization of the Word Processors Add-in
button and messages displayed in the Prefs, under the "Word Processors"
tab. These strings were previously hardcoded in the integration plugins.
[and delete unused "#zotero-prefpane-keys checkbox" (there used to be a checkbox to "Try to override conflicting shortcuts", which has been definely removed by 5b34dce40f )]
Generated files for a more robust attached-link-dialog and localized strings
* AttachLink.js
* AttachLink.xul
zotero/xpcom/attachments.js
* created function cleanAttachmentURI
* in function linkFromURL, removed the regex constraints and the comment list of valid protocols
* removed outdated function declaration from beginning of script
* Improved automatic title generation mechanism
Promise-based rewrite of most of the codebase, with asynchronous database and file access -- see https://github.com/zotero/zotero/issues/518 for details.
WARNING: This includes backwards-incompatible schema changes.
An incomplete list of other changes:
- Schema overhaul
- Replace main tables with new versions with updated schema
- Enable real foreign key support and remove previous triggers
- Don't use NULLs for local libraryID, which broke the UNIQUE index
preventing object key duplication. All code (Zotero and third-party)
using NULL for the local library will need to be updated to use 0
instead (already done for Zotero code)
- Add 'compatibility' DB version that can be incremented manually to break DB
compatibility with previous versions. 'userdata' upgrades will no longer
automatically break compatibility.
- Demote creators and tags from first-class objects to item properties
- New API syncing properties
- 'synced'/'version' properties to data objects
- 'etag' to groups
- 'version' to libraries
- Create Zotero.DataObject that other objects inherit from
- Consolidate data object loading into Zotero.DataObjects
- Change object reloading so that only the loaded and changed parts of objects are reloaded, instead of reloading all data from the database (with some exceptions, including item primary data)
- Items and collections now have .parentItem and .parentKey properties, replacing item.getSource() and item.getSourceKey()
- New function Zotero.serial(fn), to wrap an async function such that all calls are run serially
- New function Zotero.Utilities.Internal.forEachChunkAsync(arr, chunkSize, func)
- Add tag selector loading message
- Various API and name changes, since everything was breaking anyway
Known broken things:
- Syncing (will be completely rewritten for API syncing)
- Translation architecture (needs promise-based rewrite)
- Duplicates view
- DB integrity check (from schema changes)
- Dragging (may be difficult to fix)
Lots of other big and little things are certainly broken, particularly with the UI, which can be affected by async code in all sorts of subtle ways.
- Always allow "Report Errors...", even when no errors
- Show submitted diagnostic info in report
- Use white background and unitalicized text for report
- Make window larger by default
When opening the advanced search window, the current library is
selected, and a different library can be selected to change the search
scope. If a library is read-only, the saved search button is disabled.
For saved searches, the appropriate library is selected and the
drop-down is disabled.
Also:
- Close the advanced search window after a search is saved
- The default name for saved searches ("Untitled 2", etc.) was based on
collections rather than searches
- Once an initial search has been performed, the drop-downs and
checkboxes now update the results
- More consistent spacing in advanced search window
- (dev) Zotero.DB.getNextName() now takes a libraryID as its first
parameter instead of always using My Library; the old parameters are
deprecated but still work
- Each column in the middle pane can now have its own persistent
secondary sort column, configurable from a new submenu in the column
picker menu (top right of items list). The settings are stored in
extensions.zotero.secondarySort.[primaryField]. The submenu title
includes the current primary field (e.g., "Secondary Sort (Creator)"),
which is pretty weird, and I'm not sure I want to keep it, but it does
convey that the setting is specific to the selected column.
- The fallback sort fields (firstCreator, date, title, dateAdded) are
now configurable via the extensions.zotero.fallbackSort. Setting that
pref to an empty string avoids all fallback sorts, which
allows reverse-order clicking to set the order, as requested by
@aurimasv in #275.
- The previous behavior of sorting based on the exact Creator string
(rather than the actual creators) can now be restored with the
extensions.zotero.sortCreatorAsString pref. (It simply circumvents all
the newer code, so it's pretty safe.) This setting should result in
faster sorting in large libraries that have many items with the same
Creator string.
- Some of the lesser fields in the column picker menu are now in the
More Columns submenu (which is now alphabetical)
- The "Type" column is now the less-ambiguous "Item Type".
- This uses a different method to modify the column picker menu that is
simultaneously less and more hacky. (It no longer has to duplicate
Mozilla code in a custom XBL binding that wouldn't reflect future
upstream changes, and instead it bushwhacks its way through various
boxObject properties to get to the underlying menupopup.)
* Close window on blur after completion on Mac (revert previous change)
* Don't close window when canceling
* Add Esc handler to cancel/close window
* Allow columns to be resized
* Fixes#445
* Fixes#444
- Close Zotero pane before database is closed prior to reload, instead
of waiting until reload is complete
- Show an error message if Zotero Standalone is not accessible when it
should be
Avoid ambiguously implying that Linux users should be running Zotero as
root, and tell them to fix the permissions on the Zotero executable instead.
See https://forums.zotero.org/discussion/31020/
- Catch additional errors on Windows (too-long paths, maybe aliases)
- Fix "Show File" button (which didn't show in some cases, at least on
Windows)
- Clarify error message
- Remove all "Selected" from "Selected Item", because that's how a context menu works
- Change "Remove Selected Item" to "Remove Item from Collection" (fixes#122)
- Change "Delete Selected Item from Library" to "Move Item to Trash")
- Change "Remove" on Duplicate/Unfiled Items to "Hide" (fixes#122)
- Change "Create Parent Item from Selected Item" to "Create Parent Item"
