- Group file sync via Zotero File Storage
- Split file syncing into separate modules for ZFS and WebDAV
- Dragging items between libraries copies child notes, snapshots/files, and links based on checkboxes for each (enabled by default) in the Zotero preferences
- Sync errors now trigger an exclamation/error icon separate from the sync icon, with a popup window displaying the error and an option to report it
- Various errors that could cause perpetual sync icon spinning now stop the sync properly
- Zotero.Utilities.md5(str) is now md5(strOrFile, base64)
- doPost(), doHead(), and retrieveSource() now takes a headers parameter instead of requestContentType
- doHead() can now accept an nsIURI (with login credentials), is a background request, and isn't cached
- When library access or file writing access is denied during sync, display a warning and then reset local group to server version
- Perform additional steps (e.g., removing local groups) when switching sync users to prevent errors
- Compare hash as well as mod time when checking for modified local files
- Don't trigger notifications when removing groups from the client
- Clear relation links to items in removed groups
- Zotero.Item.attachmentHash property to get file MD5
- importFromFile() now takes libraryID as a third parameter
- Zotero.Attachments.getNumFiles() returns the number of files in the attachment directory
- Zotero.Attachments.copyAttachmentToLibrary() copies an attachment item, including files, to another library
- Removed Zotero.File.getFileHash() in favor of updated Zotero.Utilities.md5()
- Zotero.File.copyDirectory(dir, newDir) copies all files from dir into newDir
- Preferences shuffling: OpenURL to Advanced, import/export character set options to Export, "Include URLs of paper articles in references" to Styles
- Other stuff I don't remember
Suffice it to say, this could use testing.
should fix note handling in MacWord and WinWord. still need to figure out how to get the index of the note in OpenOffice without having to loop through every note
Closes#884, final period missing when a citation is first added in note styles
Closes#1298, issues with footnotes and citations in OOo
Closes#1069, Use async HTTP calls for integration requests
Closes#1027, User-customizable integration port number
Closes#698, Migration away from VBA
Closes#1085, Migrate VBA plug-in to new XML-based API
Closes#792, Auto-updating of OO plugins
- dropping Zotero items into a bucket puts them in that IA bucket
- double clicking a bucket takes you to that IA bucket
In order to enable Zotero Commons:
1) Get an access key and secret key at http://www.archive.org/account/s3.php
2) Go to about:config
3) Search "commons" (no quotes)
4) Set "extensions.zotero.commons.enabled" to true
5) Enter your S3 access key into "extensions.zotero.commons.accessKey"
6) Enter your S3 secret key into "extensions.zotero.commons.secretKey"
7) Enter your buckets into "extensions.zotero.commons.buckets" as a comma separated list
Note: Steps 4-7 take effect in new windows
- Restored save when clicking from open text field to another item
- Delay/weirdness when switching items/tabs should be gone
- Tab content updates properly when using Ctrl-arrows to navigate
- Removed some obsolete code
- Remove reference to Word from incompatible version message, which is also used with OpenOffice
- Add lines to Doc Prefs window to clarify the necessary file format (though enforcing this at save time would be best)
- File import now uses a translucent overlay over Zotero pane with a progress meter that doesn't hang
- New method Zotero.wait(timeout) to allow synchronous code to wait for events on main thread to be processed until timeout is reached
- Wait status can be tested with Zotero.waiting property
- Zotero.showZoteroPaneProgressBar(msg, determinate) locks Zotero and creates overlay with progress meter
- Code that might trigger via timers or external UI should check Zotero.lock
Lock checks so far:
- Translator save icon and RIS/Refer import display error if triggered while locked
- Browser content context menu options are now in a Zotero submenu and are disabled when Zotero is locked
- Sync, repository, and DB backup timers check for lock and bail
- If a new window is opened, Zotero pane can't be opened and display an error message until lock is released
Probably need to check lock in word processor integration code and advanced search window
Also:
- New method Zotero.sleep(ms) (currently unused) to allow synchronous code to sleep and allow events on main thread to be processed
Alert doesn't pop up on background sync, and auto-syncs triggered by edits are now considered background syncs (like the auto-sync when opening the Zotero pane), which means they won't pop up the conflict resolution window either.
- For PDFs, stop deleting full-text cache file after indexing, and regenerate when getting content if necessary
- MozMill tests
Other MozMill changes:
- Added Cmd-Shift-M shortcut key to open window
- this.controller now available in Zotero.MozMill if passed in from setupModule() with Zotero.MozMill.setController(module.controller);
- Various new Zotero.MozMill methods, including for creating file attachments
- Use grippy for collapsing tag selector and remove toolbar icon
- Remove redundant Attachments and Notes tabs in metadata pane
- Add all four child attachment options to submenu of item context menu
- Add Zotero.Error(message, error) constructor to create a throwable error object with an error code
- Allow only one automatic client reset between manual syncs
- Fix "Source item for keyed source doesn't exist in Zotero.Item.getSource()" error
- Object produced by item.serialize() now contains .sourceItemKey instead of .sourceItemID
- Better error logging for missing XPCOM files
- Connect tabs to pane
- Allow switching between metadata pane tabs using standard Firefox shortcuts (Ctrl-Tab, Ctrl-Shift-Tab)
- Fixed squished look in Firefox 3.5
- Remove some unnecessary padding, providing a little bit more room in the metadata pane
Needs a little testing
Closes#711, Improve keyboard support
- closes#1513, RTF Scan: {Bib} Placemarker Flexibility
- switches citations from parentheses to brackets, to make it easier to distingush between what was reformatted and what wasn't
- adds a list of example citations to RTF scan dialog
- Add newline and characters not valid in XML (mostly control characters) to filename blacklist
- Strip invalid characters when user is renaming a file manually
- Make note font configurable through extensions.zotero.note.fontFamily hidden pref (for now)
- Additional CSS rules can be added through extensions.zotero.note.css hidden pref
- Give TinyMCE note body #zotero-tinymce-note id for more specific external customization
- Remove status bar position option from visible preferences...
- Double-clicking My Library takes you to online library
- Triple-clicking, rather than double-clicking, now expands/collapses parent items, which no one will notice or care about because I'm apparently the only person in the world who previously double-clicked to expand parent items. I feel so alone.
Closes#905, "ability to set primary attachment for the "view" button in the parent item" -- I think this is as close as we're gonna get.
The tree expand/collapse behavior still happens concurrently, which is a little awkward.
- Double-clicking a group no longer expands/collapses the group while taking you to the group page
- Adjust Zotero.Items.getAll(onlyTopLevel) to take a libraryID in the second parameter and only search local library otherwise -- this has the side effect of no longer exporting group items when using Export Library, which may or may not be what we want
- Add Zotero.Libraries.isEditable(libraryID) (currently unused)
- Addresses #1444, Attachment arrow/count persists after delete, for items (problem probably remains for collections)
- Dragging PDF favicon or link or clicking Create New Item from Current Page when viewing a PDF creates a top-level PDF -- this is a temporary solution that allows "Retrieve PDF Metadata" to be used until it can be run automatically
- Allow PDF child snapshots to be dragged out of parents
- Dragging a link to a collection now creates parent item and child snapshot
- ZoteroPane.newItem(), addItemFromDocument(), addItemFromPage(), addItemFromURL(), and canEdit() now all take an optional row parameter
- Support for group libraries
- General support for multiple libraries of different types
- Streamlined sync support
- Using solely libraryID and key rather than itemID, and removed all itemID-changing code
- Combined two requests for increased performance and decreased server load
- Added warning on user account change
- Provide explicit error message on SSL failure
- Removed snapshot and link toolbar buttons and changed browser context menu options and drags to create parent items + snapshots
- Closes#786, Add numPages field
- Fixes#1063, Duplicate item with tags broken in Sync Preview
- Added better purging of deleted tags
- Added local user key before first sync
- Add clientDateModified to all objects for more flexibility in syncing
- Added new triples-based Relation object type, currently used to store links between items copied between local and group libraries
- Updated zotero.org translator for groups
- Additional trigger-based consistency checks
- Fixed broken URL drag in Firefox 3.5
- Disabled zeroconf menu option (no longer functional)
Developer-specific changes:
- Overhauled data layer
- Data object constructors no longer take arguments (return to 1.0-like API)
- Existing objects can be retrieved by setting id or library/key properties
- id/library/key must be set for new objects before other fields
- New methods:
- ZoteroPane.getSelectedLibraryID()
- ZoteroPane.getSelectedGroup(asID)
- ZoteroPane.addItemFromDocument(doc, itemType, saveSnapshot)
- ZoteroPane.addItemFromURL(url, itemType)
- ZoteroPane.canEdit()
- Zotero.CollectionTreeView.selectLibrary(libraryID)
- New Zotero.URI methods
- Changed methods
- Many data object methods now take a libraryID
- ZoteroPane.addAttachmentFromPage(link, itemID)
- Removed saveItem and saveAttachments parameters from Zotero.Translate constructor
- translate() now takes a libraryID, null for local library, or false to not save items (previously on constructor)
- saveAttachments is now a translate() parameter
- Zotero.flattenArguments() better handles passed objects
- Zotero.File.getFileHash() (not currently used)
(Smith, 2006)
The database is scanned for each citation, and positioning is adjusted automatically for footnotes. Currently, this won't work with names with accents, but I'll get to that.
- makes translator progress window work a little better with translators that call Zotero.done() incorrectly, and complains when item.complete() is called after Zotero.done()
- Fix style saving from repo (updated styles were saved without .csl extension, so updates were ignored)
- Don't update from repository on every load of SVN build with missing symlinked style targets
- Remove deleted "Open Worldcat (Search)" and "Worldcat.org" translators
- fixes NCBI PubMed translator type (should be 12, since it's a search translator as well as a web translator)
- fixes a bug passing multiple translators to translate.setTranslator()
- increases priority of CrossRef translator
- makes sync animated arrow progress indicator into a general CSS class (zotero-small-progress-indicator) so that the same progress indicator can be used for lookup. if this is confusing, we could use something else instead.
When an item has a URL, turn Locate button into a menu and allow lookup via Wayback Machine
Support for additional user-configurable providers is forthcoming
- Only one preference for recognizing proxies for transparent redirection
- Blacklists sites and http://%h/%p
- Fixes to EZProxy
This will clear all existing proxies from Zotero
Addresses #1408, Incorect format for legal case citations in CMS note without biblography style
On the trunk, maps court in case item type to authority, a new CSL variable
SQLite's COLLATE NOCASE doesn't work for Unicode characters, so it won't find existing tag "Äbc" if "äbc" is entered and will allow a duplicate tag to be created
Ben's duplicate detection code, with the integration reworked a bit
Very rough, so currently requires creation of a boolean extensions.zotero.debugShowDuplicates pref to view the Actions menu option
- Should also keep timestamps from going too far into the future when saving many items at once (which they were after my previous fix), though this can still be improved
- Disable unresponsive script warning during items list refresh
- Zotero.UnresponsiveScriptIndicator.disable() now returns true if it disables the indicator and false if it was already disabled
Adds handling for parent/child items in trash -- non-deleted context items will appear in gray, and "Restore to Library" and the delete keystrokes won't be available if any of those are selected
Like other search views, Select All will select only the deleted items
- Still experimental, but committing for testing
- Sync conflicts with deleted items aren't yet supported
Unrelated: deprecated ZoteroPane.deleteSelectedItem() in favor of more accurately named deleteSelectedItems()
- Fixes file syncing after editing a file locally
- Fixes a few storage bugs that could result in eternal spinning, invalid percentages, and other unpleasantries
- Made the attachment display box more flexible, including a filename field that we may or may not want to keep in the main view
This should fix premature closing when saving multiple items. It also means that translators must call Zotero.done() or else the window will stay open.
This change is not being made on the 1.0 branch, since we don't want to have to push dozens of fixed translators.
Values enclosed in double quotes now fail is-numeric test, and the double quotes are stripped on output to CSL -- if any fields actually need enclosing quotes, this could be limited to certain fields such as edition.
- disable bookmarks mode for note-based styles
- display an error when attempting to insert a bookmark into a footnote
- closes#1201, Keyboard focus stays in Word when creating or editing a citation
- All string values are now trimmed going into the DB (with a migration step for existing values) -- this fixes erroneous conflicts due to leading/trailing whitespace in sync XML being ignored
- Case disambiguation fixed on server
- Added basic diff ability to collections and tags so that identical objects won't trigger conflicts
- Fixed various other bugs that could cause erroneous conflicts
- Moved string fields in serialize() objects into a 'fields' object for consistency with Zotero.Item
Upshot of most of the above is that identical pre-upgrade libraries should now merge cleanly
Also reorganized/simplified/modularized parts of the sync code
Also migrates remaining files for past upgrades on the trunk (assuming their ids haven't been changed by syncing--if the item that has a given id already has a directory or if there is no attachment item with that id, the directory will be moved to orphaned-files)
- Fix a scoping issue that could prevent some property files from being purged
- Properly handle systems with millisecond-level file mod time granularity (clearing of storage sync history is required)
- Fix issue that might have erroneously removed property files
- Fix potential security issues with symlinks in ZIP files due to Firefox brokenness
- Zotero.Utilities.HTTP.doGet() can now take a URI instead (and doesn't display the password that way for authenticated requests)
- For now, delete orphaned files immediately when using "Purge Orphaned Storage Files" instead of waiting a day
- Properly remove deleted files from delete log
- Better debugging of various things
- allow deletion of multiple styles simultaneously
- split Zotero.Styles/Zotero.Style and Zotero.CSL into style.js and csl.js respectively
- add Zotero.File.getBinaryContents for binary-safe file reading
- add Zotero.MIMETypeHandler to provide a unified interface for registering observers and capturing MIME types with Zotero
Files are copied from translators.zip and styles.zip (or, for SVN installs, 'translators' and (for now) 'csl' directories) in the installation directory to 'translators' and 'styles' directories in the data directory. A build_zip file is provided for testing translators.zip (which will take precedence over a 'translators' directory) but isn't required.
The timestamp stored in repotime.txt is stored in the database and is sent to the server for updates since that time.
Updating a file in [install-dir]/translators or [install-dir]/styles automatically copies all files in that directory to the data directory.
This will break translation for the dev XPI and on the trunk until Dan commits his part of this (or you copy the translators dir in SVN to the zotero dir in your profile dir)
- Still experimental and incomplete, with no lock support and not much error handling
Also:
- New expiry date for sync functions
- Attachment character set was being dropped during syncing
- Possibly improves sizing issues with preferences window
- Fixes problems with attachment filenames with extended characters
- Fixes some problem with tags that I don't remember
- Makes XMLHTTPRequest calls are now background requests (no auth windows or other prompts)
- Z.U.HTTP.doOptions() now takes an nsIURI instead of a URL spec
- New methods:
- Zotero.Utilities.rand(min, max)
- Zotero.Utilities.probability(x)
- Zotero.Utilities.Base64.encode(str) and decode(str)
- Zotero.getTempDirectory()
- Zotero.Date.dateToISO(date) - convert JS Date object to ISO 8601 UTC date/time
- Zotero.Date.isoToDate(isoDate) - convert an ISO 8601 UTC date/time to a JS Date object
The feature grabs relevant fragments from the document and then searches them on Google Scholar. This will only work with OCRed PDFs, so it doesn't work with JSTOR, but it should work fairly well with everything else.
changes default behavior. transparent redirection is now enabled by default, and a caution dialog appears when proxies are first accessed. when transparent redirection is turned off, no dialog appears, and proxies get saved automatically. when the user switches transparent redirection on, there is a warning that s/he should check that there are only trusted proxies in the list. i'm not sure how well i've worded the messages, so feel free to suggest better alternatives.
also, a bit of jsdoc cleanup on proxy.js
closes#831, transparent EZProxy support
adds a proxy pane to the preferences
asks before saving proxies to the DB (to avoid the potential phishing risk #831 would otherwise pose)
The remote object could legitimately not have been updated locally since the last sync if the server data was cleared and synched from a different client, potentially resulting in a different key condition on sync. The two sides should now be properly merged (at least in theory).
Adds a Python/py-appscript-based plug-in for Word 2008. To get this to work, you'll need to copy the Zotero directory (not its contents) to ~/Microsoft User Data/Word Script Menu Items and install py-appscript (sudo easy_install appscript)
Some caveats:
- Requires Word be installed at /Applications/Microsoft Office 2008/Microsoft Word 2008.app (this is fixable, but I'm still determining the best way to solve it)
- Still need to figure out what to do with items that have been deleted from the DB (right now, we just ignore them)
- Sometimes, Python.app launches with the script, which seems to slow execution time
Also moves orphaned directories into orphaned-files folder in data dir
Safety first: Keeps track of moved files, moving them back if there's an error before the end of the upgrade process (since the keys are generated randomly and would be different if recreated), and creates a zotero.moved-files.36.bak file with a list of id/key pairs
closes#704, EndNote to Zotero style converter (won't actually convert styles due to copyright concerns, but will load them into the DB)
also adds CSL style manager
- Inspired by Dan Chudnov's Python/MODS-based Zeroconf demo at THATcamp
- Enabled by extensions.zotero.zeroconf.enabled (off by default)
- Currently supports only OS X (tested on Leopard, not sure about earlier versions)
- Uses Apple's dns-sd and mDNS command-client clients, but should be able to be extended to other clients, though a native library would be far superior
- Discovery is on-demand for now via Actions menu ("Search for Shared Libraries")
- Includes rudimentary web server (code copied from integration.js) that serves items as sync XML -- no authentication yet!
- Only supports top-level items
- Remote libraries show up in left pane (under remote computer name, for now)
- Items can be dragged into collections (but not the library yet, for some reason)
- On first run, might cause a long pause and the "This file was downloaded from the Internet" message on Leopard -- can't manage to get around the quarantine for the script file that we need to access stdout from Firefox
- Needs a lot of work, and without a real JS (or otherwise Mozilla-native) Zeroconf library we can't do proper discovery without intermittent polling
- But it works, at least for me
Also includes some data/sync-layer changes that I needed along the way (and that we'll need for shared collections of any type)
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
Closes#285, Hitting Esc should close tags popup
Fixes various UI issues and problems adding tags on Minefield
Still getting autocomplete self-destruction
This is a bit uglier, and the large markers are kinda annoying once you know the drop-down is there, but perhaps this will stop the forum posts. Please let them stop.
If anyone's out there, now would be the time to test this.
Missing icons for the new types -- I could try to get some of those now, or we could just launch. Maybe we'll just launch. But really, might want to test this. Just sayin'.
Use automaticSnapshots pref (which defaults to on and is changeable in the prefs window) rather than downloadAssociatedFiles (which defaults to off and is only settable through about:config at the moment) for now in translate.js
downloadAssociatedFiles should eventually be used for PDFs and other large files, whereas automaticSnapshots will be for HTML and the like -- in the meantime, I think it's OK for scrapers to just follow the visible pref for both, since otherwise they'd be totally confused when the NIFP button took a snapshot and the scrapers didn't
Simon, if there's any problem I'm not aware of with switching this for now (other than people getting some large PDFs on JStor), let me know.
closes#165, verify import/export can carry all data for all fields and item types
closes#168, make sure MODS import works with files from external sources
Attach Snapshot of Current Page
Attach Link to Current Page
I also removed "New Item" from the context menu, as it was the only option in the menu that wasn't specific to the selected item, it made the whole menu a lot more confusing, and it doesn't take all that much longer to click the (+) button and choose Book (assuming that's even the item you want).
Addresses #155, Localize strings
- Added "Search subfolders"
- Localized strings in search dialog
Known issue:
#342, "Search subfolders" checkbox should be greyed out until applicable
Including in the DB, which it turns out isn't really all that bad (thanks, among other things, to SQLite's ability to DROP tables within transactions without autocommitting (which MySQL can't do))
At least some scrapers (NYT and WashPo, for sure) should be updated to follow this pref
Addresses #327, Scrapers should either take snapshots or use URL field
Changed creator type <label> to <toolbarbutton> -- hopefully won't have any ill effects with other platforms' default themes (I haven't checked)
Moved some metadata pane CSS from overlay.css to zotero.css
Currently uses an annoying popup window, since the code used in the regular metadata pane has too much baggage to use here -- it really should be abstracted into an XBL binding, but that won't be an easy feat
Also fixes SQL error opening popup notes and JS strict warnings when saving them
- Currently requires user to enter dates in SQL format if they want to change the access date, but at least it doesn't mangle the dates anymore
- Uses new function ScholardammitZotero.Date.dateToSQL(Date date [, Boolean toUTC])
- Utilities.lpad() now forces _string_ to a string so that .length exists
- Unrelated: Item.save() now returns false if the item didn't change
closes#313, Blacklist known ad sites from scraper detection
closes#306, some New York Times ads prevent page from being recognized
closes#308, attachment import bug
currently, the ad site blacklist is located at the top of ingester/browser.js. at some point, we may want to switch this to a database table.
addresses #327, Scrapers should either take snapshots or use URL field
closes#309, Integration server prevents Zotero from loading in multiple instances of Firefox
In other words, show both "Shakespeare" and "Shakespeare, William" in the drop-down, and if the latter is chosen, save both fields
One issue is that since the autocomplete is by default limited to the width of the textbox, longer entries get truncated (though you can see them with a mouseover), and that may not be easy to fix.
Changed "Scholar" to "Zotero", everywhere
Apologies to anyone with working copy changes, but there are probably the fewer at this moment than there will be again.
Hopefully this won't break anything, though existing prefs will be lost. I avoided scholar.google.com--if you know any other legitimate "scholar"s in the code, be sure to fix them once I'm done here.
This is a multi-commit change--there's at least one more coming. *Do not update to this version! It won't work!*
Closes#304, change references to "website" to "web page"
More changes as per discussions with Dan:
- Linked URLs have been given a second chance at life, though they still shouldn't be used for (most, if any) scrapers (which should use snapshots or the URL field instead)
- Renamed the "website" item type to "webpage"
- Removed "web page" from the New Item menu
- Added Save Link To Current Page toolbar button
- Added toolbar separator between New Item buttons and link/attachment/note to differentiate
- Added limited metadata (URL and accessDate) for attachments
- URL for attachments now stored in itemData (itemAttachments.originalPath is no longer used, but I'm probably not gonna worry about it and just wait for SQLite to support dropping columns with ALTER TABLE) -- getURL() removed in favor of getField('url')
- Snapshots now say "View Snapshot"
- Added Show File button to file attachments to show in filesystem
- Added timed note field to attachments for single notes and adjusted Item.updateNote(), etc. to work with attachments
- Fixed bug with manually bound params in fulltext indexer and Item.save() (execute() vs. executeStep()) -- any recently added items probably aren't in the fulltext index because of this
Known bugs/issues:
- Attachment metadata and notes probably aren't properly imported/exported now (and accessDate definitely isn't)
- Scrapers don't save metadata properly
- Attachment title should be editable
- File attachments could probably use some more metadata (#275, more or less, though they won't be getting tabs)
And fixed a bug that caused the text of some search conditions to not display when editing a previously saved search
Also added autocomplete to more fields in metadata pane:
- creator
- publisher
- place
- publicationTitle
- journalAbbreviation
- seriesTitle
- seriesText
It should also do the type and medium fields, but they need to be separated out first--e.g. artworkType, interviewMedium--since they're describing different data
Overhaul of the item drag and drop functionality, allowing dragging notes and attachments into and out of other items and addressing all the related issues that arise when that's possible.
Should also be generally smarter about deciding what can be dragged and dropped where and in what modes.
Let me know if something doesn't work as you expect.
Icons from Ken
Of course, I realized just now that this is really quite similar to the stop sign, and anyone who puts this to the left of their URL bar and doesn't have a Home icon in their toolbar will get the two next to each other... Looks good otherwise, though.
Changes as per my discussions with Dan:
- Separated snapshot functionality into two individual buttons, Create New Item From Current Page and Take Snapshot of Current page
- Updated schema to support primary, secondary and hidden item types (and future user customizations)
- Reorganized New Item menu, moving secondary items into sub-menu
- Removed ability to create link attachments, since it never really made much sense -- will simply use the webpage item type instead. Underlying functionality still exists for the time being, as people have existing links in their libraries--I think we're gonna have to just warn beta testers and delete them in a transition step, as converting nested links really wouldn't be worth the effort.
- Moved file link/add functions into new item menu and removed attachment drop-down
- Large, prominent View and Locate buttons in edit pane for going to an associated URL and looking up in OpenURL, respectively -- buttons gray out as appropriate
- New Item from Page stores the URL and access date (Item.save() checks for the string "CURRENT_TIMESTAMP" for accessDate and doesn't bind it as a string)
- "Website" to "Web Page" (do we prefer "Webpage"? they both look a bit funky in uppercase)
More coming.
Bugs/Known Issues:
- Since snapshots from the toolbar are now top-level in the current collection, there needs to be a way to drag them into items
- The camera icon for adding snapshots, despite being a famfamfam icon, really doesn't read too well (or perhaps just clashes with the rest of our icons). Anybody have a better one? (It also may be able to just be lightened up a bit.)
- Trying the large View/Locate buttons after discussions with Dan, but this approach may not work -- 1) a large View button for the URL makes a lot less sense when you have a parent item with a child snapshot, since people will end up clicking it all the time when they really want to view the snapshot, and 2) the Locate button is awfully big for something that only applies to certain types of items, may not get used very often when it does, and probably won't work when it is
- The access date is stored in UTC and displayed with toLocaleString() like Date Added and Date Modified, but, unlike those two, it's also user-editable. This is clearly a problem. Probably need to parse to Date on blur() with strToDate() and insert as UTC, discarding anything left over.
- Item type itself is still "website" -- should probably change that while we still can
Closes#253, OpenURL arrow should provide visual feedback on mouseover and/or look more button-like
Addresses #304, change references to "website" to "web page"
Addresses #207, openurl arrow functionality
Differentiates between single and double fields for the search, but there's a problem in the current implementation in that only one field is editable at once, so displaying two-field names in a drop-down is a little problematic. While I could display the full names, comma-delimited, and get the discrete parts (which is what Scholar.Utilities.AutoComplete.getResultComment(), included in this commit, is for--the creatorID for the row would be hidden in the autocomplete drop-down comment field), it's a bit unclear what should happen when a user selects a comma-separated name from the drop-down of one of the fields. One option would be to have a row for the last name (in case that's all they want to complete) and other rows for "last, first" matches, and selecting one of the two-part names would replace whatever's in the opposite name field with the appropriate text (and save it to the DB, I'm afraid, unless I change how the creator fields work), keeping the focus in the current textbox for easy tabbing. Not great, but it might work.
Other ideas?
Adds tab handling to tags interface, which also addresses beta tester requests for a quicker way to enter multiple tags
This is getting pretty messy, but it's still probably better than repeating all the itemPane.js in tagsbox.xml.
One known issue is that, since it resorts the list of tags after a change, if you change an existing tag to a name that alters its current position and then tab away, you don't necessarily end up in the field you expect.
- Switched to manually repeated bound parameters in indexWords()
- Switched to the innerHTML regex used elsewhere instead of a more proper but nevertheless misguided DOM traverser to split elements in indexDocument
This may invalidate the fulltext progress indicator ticket
Still need #292, "Delete From Library" context menu option in collections
This may or may not help people understand that deleting from a collection doesn't by design delete from the library, but, regardless, this is the same behavior as iTunes.
Example usage:
var windowWatcher = Components.classes["@mozilla.org/embedcomp/window-watcher;1"].
getService(Components.interfaces.nsIWindowWatcher);
var progress = new Scholar.ProgressWindow(windowWatcher.activeWindow);
progress.changeHeadline('Indexing item...');
progress.addLines(['All About Foo'], ['chrome://scholar/skin/treeitem-book.png']);
progress.addDescription('Bar bar bar bar bar');
progress.show();
progress.fade();
There are currently two types of fulltext searching: an SQL-based word index and a file scanner. They each have their advantages and drawbacks.
The word index is very fast to search and is currently used for the find-as-you-type quicksearch. However, indexing files takes some time, so we should probably offer a preference to turn it off ("Index attachment content for quicksearch" or something). There's also an issue with Chinese characters (which are indexed by character rather than word, since there are no spaces to go by, so a search for a word with common characters could produce erroneous results). The quicksearch doesn't use a left-bound index (since that would probably upset German speakers searching for "musik" in "nachtmusik," though I don't know for sure how they think of words) but still seems pretty fast.
* Note: There will be a potentially long delay when you start Firefox with this revision as it builds a fulltext word index of your existing items. We obviously need a notification/option for this. *
The file scanner, used in the Attachment Content condition of the search dialog, offers phrase searching as well as regex support (both case-sensitive and not, and defaulting to multiline). It doesn't require an index, though it should probably be optimized to use the word index, if available, for narrowing the results when not in regex mode. (It does only scan files that pass all the other search conditions, which speeds it up considerably for multi-condition searches, and skips non-text files unless instructed otherwise, but it's still relatively slow.)
Both convert HTML to text before searching (with the exception of the binary file scanning mode).
There are some issues with which files get indexed and which don't that we can't do much about and that will probably confuse users immensely. Dan C. suggested some sort of indicator (say, a green dot) to show which files are indexed.
Also added (very ugly) charset detection (anybody want to figure out getCharsetFromString(str)?), a setTimeout() replacement in the XPCOM service, an arrayToHash() method, and a new header to timedtextarea.xml, since it's really not copyright CHNM (it's really just a few lines off from the toolkit timed-textbox binding--I tried to change it to extend timed-textbox and just ignore Return keypress events so that we didn't need to duplicate the Mozilla code, but timed-textbox's reliance on html:input instead of html:textarea made things rather difficult).
To do:
- Pref/buttons to disable/clear/rebuild fulltext index
- Hidden prefs to set maximum file size to index/scan
- Don't index words of fewer than 3 non-Asian characters
- MRU cache for saved searches
- Use word index if available to narrow search scope of fulltext scanner
- Cache attachment info methods
- Show content excerpt in search results (at least in advanced search window, when it exists)
- Notification window (a la scraping) to show when indexing
- Indicator of indexed status
- Context menu option to index
- Indicator that a file scanning search is in progress, if possible
- Find other ways to make it index the NYT front page in under 10 seconds
- Probably fix lots of bugs, which you will likely start telling me about...now.
Note that there's no code for user types and fields yet -- just the schema (actually there's a tiny bit of code in the item type manager, since we'll probably use some of the same methods for managing user types, but not much)
Templates for primary item types are currently only used by the item type manager to make creating new types easier and to prevent the removal of fields from an item type that are associated with its template item type -- the fields are all still recorded in itemTypeFields, since they might have different orders or default visibility settings from their templates
This lets us add tables to user.sql without writing migration steps for them yet still have the ability to change existing user tables and migrate data if necessary.
Also added _getDropCommands() to do a regex on the SQL file and create the DROP TABLE|INDEX steps necessary to use the DB_REBUILD flag without DROP commands in the SQL file itself, before I realized that it probably made the most sense to just delete the SQL file and storage directory. Changed _initializeSchema() to do that instead. Leaving _getDropCommands() in, in case there's ever a need for it.
Using Shift-Enter as the save keystroke within the Extra textbox so that people can use Enter to create multiple lines of text. Shift-Enter would normally be the newline command, but that's probably a convention that non-technical users of Zotero wouldn't know... Tab (and other triggers for blur()) also saves, and since Extra is the last field, tabbing away functions the same as hitting Enter does for other fields, so it's probably not that big of a deal.
This should make development much easier, as we can, for example, add 80 item types without having to write transition steps
Pretty sure this won't delete anyone's data. Might want to test that theory, though.
- closes#217, ability to exclude notes/attachments from select items window
- closes#244, ability to quick search from select items window
- fixes a bug with footnotes in Word integration
- fixes a bug in InnoPAC translator where items would sometimes appear twice
Here's a shot at a single/double creator field toggle switch -- let me know what you think
A few issues:
- There's currently no comma between the last name and first name when in two-field mode -- I removed it to greatly simplify the code, hoping to be able to use the CSS :after pseudo-element, but that seems to not work with XUL -- I'll figure out a clean solution and add it back ( refs #288 )
- It's not very smart about switching between single-field mode and two-field mode, as it currently just keeps the last word (even if it's "Jr." or "III") as the last name and puts the rest in the first name field -- not a big deal, but it should at least be a bit smarter about this ( refs #289 )
- There are probably some other bugs
New logic in itemTreeView notify() target:
- Items are only selected on add in the active window -- this fixes a fairly major flaw in the previous system that would cause new items to be selected in all open windows
- If a quicksearch is open in the active window and a new item is added, clear the search
- If quicksearch and active window and item modify, rerun search
- If quicksearch and not active window, rerun search
- If not quicksearch and not active window, update list but retain previous selection
Addresses #260, Add auto-complete to search window
- New XPCOM autocomplete component for Zotero data -- can be used by setting the autocompletesearch attribute of a textbox to 'zotero' and passing a search scope with the autocompletesearchparam attribute. Additional parameters can be passed by appending them to the autocompletesearchparam value with a '/', e.g. 'tag/2732' (to exclude tags that show up in item 2732)
- Tag entry now uses more or less the same interface as metadata -- no more popup window -- note that tab isn't working properly yet, and there's no way to quickly enter multiple tags (though it's now considerably quicker than it was before)
- Autocomplete for tags, excluding any tags already set for the current item
- Standalone note windows now register with the Notifier (since tags needed item modification notifications to work properly), which will help with #282, "Notes opened in separate windows need item notification"
- Tags are now retrieved in alphabetical order
- Scholar.Item.replaceTag(oldTagID, newTag), with a single notify
- Scholar.getAncestorByTagName(elem, tagName) -- walk up the DOM tree from an element until an element with the specified tag name is found (also checks with 'xul:' prefix, for use in XBL), or false if not found -- probably shouldn't be used too widely, since it's doing string comparisons, but better than specifying, say, nine '.parentNode' properties, and makes for more resilient code
A few notes:
- Autocomplete in Minefield seems to self-destruct after using it in the same field a few times, taking down saving of the field with it -- this may or may not be my fault, but it makes Zotero more or less unusable in 3.0 at the moment. Sorry. (I use 3.0 myself for development, so I'll work on it.)
- This would have been much, much easier if having an autocomplete textbox (which uses an XBL-generated popup for the suggestions) within a popup (as it is in the independent note edit panes) didn't introduce all sorts of crazy bugs that had to be defeated with annoying hackery -- one side effect of this is that at the moment you can't close the tags popup with the Escape key
- Independent note windows now need to pull in itemPane.js to function properly, which is a bit messy and not ideal, but less messy and more ideal than duplicating all the dual-state editor and tabindex logic would be
- Hitting tab in a tag field not only doesn't work but also breaks things until the next window refresh.
- There are undoubtedly other bugs.
- import translators no longer fail when trying to import an item with no name
- the T2/BT field becomes the publication title when no JO/JF field is available (fixes newspaper issues)
- Y2 is now treated as part of the date if and only if it is improperly formatted (seriously, why can't Thomson get their own specs straight?)
- work around EndNote's strange behavior of putting article titles into notes for no apparent reason
- RIS export gives dates as per specification
- fixed a bug that could have (potentially) caused problems formatting "January"
- allow translators to access strToDate function
