- Switched to manually repeated bound parameters in indexWords()
- Switched to the innerHTML regex used elsewhere instead of a more proper but nevertheless misguided DOM traverser to split elements in indexDocument
This may invalidate the fulltext progress indicator ticket
Still need #292, "Delete From Library" context menu option in collections
This may or may not help people understand that deleting from a collection doesn't by design delete from the library, but, regardless, this is the same behavior as iTunes.
Example usage:
var windowWatcher = Components.classes["@mozilla.org/embedcomp/window-watcher;1"].
getService(Components.interfaces.nsIWindowWatcher);
var progress = new Scholar.ProgressWindow(windowWatcher.activeWindow);
progress.changeHeadline('Indexing item...');
progress.addLines(['All About Foo'], ['chrome://scholar/skin/treeitem-book.png']);
progress.addDescription('Bar bar bar bar bar');
progress.show();
progress.fade();
There are currently two types of fulltext searching: an SQL-based word index and a file scanner. They each have their advantages and drawbacks.
The word index is very fast to search and is currently used for the find-as-you-type quicksearch. However, indexing files takes some time, so we should probably offer a preference to turn it off ("Index attachment content for quicksearch" or something). There's also an issue with Chinese characters (which are indexed by character rather than word, since there are no spaces to go by, so a search for a word with common characters could produce erroneous results). The quicksearch doesn't use a left-bound index (since that would probably upset German speakers searching for "musik" in "nachtmusik," though I don't know for sure how they think of words) but still seems pretty fast.
* Note: There will be a potentially long delay when you start Firefox with this revision as it builds a fulltext word index of your existing items. We obviously need a notification/option for this. *
The file scanner, used in the Attachment Content condition of the search dialog, offers phrase searching as well as regex support (both case-sensitive and not, and defaulting to multiline). It doesn't require an index, though it should probably be optimized to use the word index, if available, for narrowing the results when not in regex mode. (It does only scan files that pass all the other search conditions, which speeds it up considerably for multi-condition searches, and skips non-text files unless instructed otherwise, but it's still relatively slow.)
Both convert HTML to text before searching (with the exception of the binary file scanning mode).
There are some issues with which files get indexed and which don't that we can't do much about and that will probably confuse users immensely. Dan C. suggested some sort of indicator (say, a green dot) to show which files are indexed.
Also added (very ugly) charset detection (anybody want to figure out getCharsetFromString(str)?), a setTimeout() replacement in the XPCOM service, an arrayToHash() method, and a new header to timedtextarea.xml, since it's really not copyright CHNM (it's really just a few lines off from the toolkit timed-textbox binding--I tried to change it to extend timed-textbox and just ignore Return keypress events so that we didn't need to duplicate the Mozilla code, but timed-textbox's reliance on html:input instead of html:textarea made things rather difficult).
To do:
- Pref/buttons to disable/clear/rebuild fulltext index
- Hidden prefs to set maximum file size to index/scan
- Don't index words of fewer than 3 non-Asian characters
- MRU cache for saved searches
- Use word index if available to narrow search scope of fulltext scanner
- Cache attachment info methods
- Show content excerpt in search results (at least in advanced search window, when it exists)
- Notification window (a la scraping) to show when indexing
- Indicator of indexed status
- Context menu option to index
- Indicator that a file scanning search is in progress, if possible
- Find other ways to make it index the NYT front page in under 10 seconds
- Probably fix lots of bugs, which you will likely start telling me about...now.
Note that there's no code for user types and fields yet -- just the schema (actually there's a tiny bit of code in the item type manager, since we'll probably use some of the same methods for managing user types, but not much)
Templates for primary item types are currently only used by the item type manager to make creating new types easier and to prevent the removal of fields from an item type that are associated with its template item type -- the fields are all still recorded in itemTypeFields, since they might have different orders or default visibility settings from their templates
This lets us add tables to user.sql without writing migration steps for them yet still have the ability to change existing user tables and migrate data if necessary.
Also added _getDropCommands() to do a regex on the SQL file and create the DROP TABLE|INDEX steps necessary to use the DB_REBUILD flag without DROP commands in the SQL file itself, before I realized that it probably made the most sense to just delete the SQL file and storage directory. Changed _initializeSchema() to do that instead. Leaving _getDropCommands() in, in case there's ever a need for it.
Using Shift-Enter as the save keystroke within the Extra textbox so that people can use Enter to create multiple lines of text. Shift-Enter would normally be the newline command, but that's probably a convention that non-technical users of Zotero wouldn't know... Tab (and other triggers for blur()) also saves, and since Extra is the last field, tabbing away functions the same as hitting Enter does for other fields, so it's probably not that big of a deal.
This should make development much easier, as we can, for example, add 80 item types without having to write transition steps
Pretty sure this won't delete anyone's data. Might want to test that theory, though.
- closes#217, ability to exclude notes/attachments from select items window
- closes#244, ability to quick search from select items window
- fixes a bug with footnotes in Word integration
- fixes a bug in InnoPAC translator where items would sometimes appear twice
Here's a shot at a single/double creator field toggle switch -- let me know what you think
A few issues:
- There's currently no comma between the last name and first name when in two-field mode -- I removed it to greatly simplify the code, hoping to be able to use the CSS :after pseudo-element, but that seems to not work with XUL -- I'll figure out a clean solution and add it back ( refs #288 )
- It's not very smart about switching between single-field mode and two-field mode, as it currently just keeps the last word (even if it's "Jr." or "III") as the last name and puts the rest in the first name field -- not a big deal, but it should at least be a bit smarter about this ( refs #289 )
- There are probably some other bugs
New logic in itemTreeView notify() target:
- Items are only selected on add in the active window -- this fixes a fairly major flaw in the previous system that would cause new items to be selected in all open windows
- If a quicksearch is open in the active window and a new item is added, clear the search
- If quicksearch and active window and item modify, rerun search
- If quicksearch and not active window, rerun search
- If not quicksearch and not active window, update list but retain previous selection
Addresses #260, Add auto-complete to search window
- New XPCOM autocomplete component for Zotero data -- can be used by setting the autocompletesearch attribute of a textbox to 'zotero' and passing a search scope with the autocompletesearchparam attribute. Additional parameters can be passed by appending them to the autocompletesearchparam value with a '/', e.g. 'tag/2732' (to exclude tags that show up in item 2732)
- Tag entry now uses more or less the same interface as metadata -- no more popup window -- note that tab isn't working properly yet, and there's no way to quickly enter multiple tags (though it's now considerably quicker than it was before)
- Autocomplete for tags, excluding any tags already set for the current item
- Standalone note windows now register with the Notifier (since tags needed item modification notifications to work properly), which will help with #282, "Notes opened in separate windows need item notification"
- Tags are now retrieved in alphabetical order
- Scholar.Item.replaceTag(oldTagID, newTag), with a single notify
- Scholar.getAncestorByTagName(elem, tagName) -- walk up the DOM tree from an element until an element with the specified tag name is found (also checks with 'xul:' prefix, for use in XBL), or false if not found -- probably shouldn't be used too widely, since it's doing string comparisons, but better than specifying, say, nine '.parentNode' properties, and makes for more resilient code
A few notes:
- Autocomplete in Minefield seems to self-destruct after using it in the same field a few times, taking down saving of the field with it -- this may or may not be my fault, but it makes Zotero more or less unusable in 3.0 at the moment. Sorry. (I use 3.0 myself for development, so I'll work on it.)
- This would have been much, much easier if having an autocomplete textbox (which uses an XBL-generated popup for the suggestions) within a popup (as it is in the independent note edit panes) didn't introduce all sorts of crazy bugs that had to be defeated with annoying hackery -- one side effect of this is that at the moment you can't close the tags popup with the Escape key
- Independent note windows now need to pull in itemPane.js to function properly, which is a bit messy and not ideal, but less messy and more ideal than duplicating all the dual-state editor and tabindex logic would be
- Hitting tab in a tag field not only doesn't work but also breaks things until the next window refresh.
- There are undoubtedly other bugs.
- import translators no longer fail when trying to import an item with no name
- the T2/BT field becomes the publication title when no JO/JF field is available (fixes newspaper issues)
- Y2 is now treated as part of the date if and only if it is improperly formatted (seriously, why can't Thomson get their own specs straight?)
- work around EndNote's strange behavior of putting article titles into notes for no apparent reason
- RIS export gives dates as per specification
- fixed a bug that could have (potentially) caused problems formatting "January"
- allow translators to access strToDate function
MODS uses the encoding as specified in the <?xml tag, or else UTF-8
RIS uses IBM850, since the spec says "IBM Extended Character Set" and it's the only code page Mozilla supports. (should I do this? or just use unicode?)
MARC uses UTF-8, since I don't think there's any way to get full MARC-8 support, and UTF-8 is now the preferred encoding anyway
