closes#160, cache regular expressions
closes#188, rewrite MARC handling functions
MARC-based translators should now produce item types besides "book." right now, artwork, film, and manuscript are available. MARC also has codes for various types of audio (speech, music, etc.) and maps.
the EBSCO translator does not yet produce attachments. i sent them an email because their RIS export is invalid (the URLs come after the "end of record" field) and i'm waiting to see if they'll fix it before i try to fix it myself.
the EBSCO translator is unfortunately a bit slow, because it has to make 5 requests in order to get RIS export. the alternative (scraping individual item pages) would be even slower.
regular expression caching can be turned off by disabling extensions.scholar.cacheTranslatorData in about:config. if you leave it on, you'll have to restart Firefox after updating translators.
On Simon's suggestion, appending .html to all text/html files--I can't reproduce the problem, and I'm not sure what's causing the ExternalHelperAppService to get confused, but this should at least avoid the issue...
import/export of file data should work for all file types _except_ snapshots (in this situation, export is working, but import is not yet complete; see #193)
also, fixes a potential security issue that could have allowed malicious web translators to post local data to remote sites (although, given we maintain the central repository and there's no easy way to install a translator, the risk would have been minimal to begin with).
closes#3, Overflow metadata dumps into "extra" field
add "extra" data where such data is useful and conveniently accessible (not available for XML-based export or MARC formats yet)
add links to permanent URLs
download associated files from full text sources (if extensions.scholar.downloadAssociatedFiles preference is enabled)
fix WorldCat translator
improve InnoPAC translator (it now works on Georgetown search results pages, albeit slowly, because it must first realize the catalog is misconfigured)
tag items from SIRSI and WorldCat
return to putting the full lengths of books into "pages," because some citation styles require it
fix COinS (broken a few revisions ago)
closes#186, stop translators from hanging
when a document loads inside a frameset, we now check whether we can scrape each individual frame.
all functions involving tabs have been vastly simplified, because in the process of figuring this out, i discovered Firefox 2's new tab events.
if a translator throws an exception inside loadDocument(), doGet(), doPost(), or processDocuments(), a translate error message will appear, and the translator will not hang
(Also fixes bug from r379 where changing creator type on a blank unsaved creator would actually insert "(first)" and "(last)")
Localized '(first)' and '(last)'
Refs #179, adding a new creator then clicking an existing creator makes the creator field disappear
closes#180, make all contextual menu export/create bibliography options work right
also:
- add Chicago Note style output
- unregister RDF data sources from cache after import
Fixed a number of creator-editing-related issues (mostly interface-side, with a little help from the data layer):
- New row no longer disappears when clicking "+" and then clicking the existing or new creator (removed onselect="ScholarItemPane.loadPane(this.selectedIndex)" on the <deck> (from r371, with changelog "Individual tabs don't load their content unless selected"), which doesn't seem to be necessary as far as I can tell)
- New row no longer disappears when changing creator before editing names (setting the creator type now triggers an Item.save() with an otherwise blank creator (now allowed by the data layer), which isn't entirely ideal but is probably OK for now)
- Clicking the minus button on an unsaved row (i.e. one just created with the plus button) no longer throws an error (new method Item.hasCreatorAt(pos), and ScholarItemPane.removeCreator() just deletes the label directly, since it doesn't get a notify() event to reload the pane)
- The plus button is disabled on unsaved rows, since allowing the user to create multiple unsaved rows and then edit one in the middle is problematic (and the other alternatives have their own problems); the minus button is also disabled on the default row that shows when there are no creators
- Creator type is no longer reset when editing a name field
- Name field is no longer erased when clicking directly on creator type popup and changing creator type without blurring textbox
- Comma is appended to last name when switching from <textbox> to <label> without saving changes (before it was just appended to labels when the pane was loaded)
references #178, changes to various date fields
- updates CSL to work with the latest schema. we can now (almost) generate completely valid APA style. the only issue is that there's no syntax for specifying short forms for page and creator type labels.
- updates scrapers to use date field rather than year field.
- removes now-unnecessary translation engine code pertaining to year field.
Added isUTC parameter to Scholar.Date.sqlToDate() and now run the two columns through that
Side effect: now using toLocaleString() to format the date strings
Closes#111, minor modifications to field list schema
Changes per above tickets and comments at http://chnm.grouphub.com/projects/310105/msg/cat/2333974/2995041/comments
- date, year and lastModified merged into date
- added date field to journalArticle
- added series, seriesTitle and seriesText to journalArticle
- added seriesNumber (along with series) to book and bookSection
(xmlhttp.status seems to turn into a very large integer when there's a network error--I can't imagine this a fairly reliable test, but, then, neither was the previous one, and at least at the moment we're not actually using the test for anything other than an appropriate debug message)
- Fixed bug in File.hasInternalHandler() (no access to navigator from XPCOM)
- Changed "View Attachment" action to check File.hasInternalHandler() and use window.loadURI() for internally handled files and nsIFile.launch() for external -- this prevents the user from getting a helper app dialog when they try to view external files. I basically had to duplicate most of Mozilla's content detection logic and "guess" whether or not it will be able to handle the file internally, which seems a little silly, but, while I feel there are probably better ways to do various parts of this, what's here seems to do the trick. Let me know if you notice it guessing incorrectly (i.e. you get a helper app dialog rather than having a file just open or it launches a file that should've just been loaded into the window). Also look for text files that should be launched rather than opened, especially XML-based data files, as this is a chance for Scholar to be smarter than Firefox itself--for example, OmniGraffle files, which are actually just XML files, normally open up in Firefox as an XML tree, but Scholar will launch them instead. (I imagine the same will need to be done for OmniOutliner, among other things...)
If you have attachments to the old terminology, feel free to file a complaint.
Changed interface code too, since David is gone (or at the very least has more important things to do with his remaining time)
- 'isInstitution' parameter added to Item.setCreator(), Creators.getID(), Creators.add()
- 'isInstitution' property added to return from Creators.get() and Item.getCreator()
var obj = Scholar.Items.getNewItemByType(1);
obj.setField('title', 'Digital History for Dummies');
obj.setCreator(0, '', 'Center for History and New Media', 1, true); // true == institutional creator
var id = obj.save();
Note: 'firstName' field is ignored when 'isInstitution' is true
Conditions in ANY queries can be made required by passing 'true' as an extra parameter to addCondition() and updateCondition() -- this can be used for limiting ANY queries to particular collections (in place of the removed 'context' condition), but if there was an elegant way to expose it to the user for all ANY queries, it's something users might find very useful.
- Implemented 'collectionID' and 'savedSearchID' conditions (a.k.a. search within a search) and removed special 'context' condition. Per my conversation with Dan, the 'recursive' flag is now a global flag that applies to all specified collectionIDs, which is less than ideal but probably better than the alternatives (e.g. having condition-specific recursive checkboxes). It does mean, however, that a "Search subfolders" checkbox is irrelevant if there are no collectionID conditions and should probably be greyed out until applicable.
Another side effect is that it's no longer possible to do an ANY search and return results only within a specific folder (though it can now be done by putting the ANY conditions in a subsearch). Since ANY searches are always annoying in this regard, what I might do is add a way to mark particular conditions as required even in ANY mode, which would allow for quite a lot of flexibility...
Note also that while 'collectionID' and 'savedSearchID' are standard conditions, they should probably be combined into a single condition on the interface side (like playlists and smart playlists under just 'Playlist' in iTunes).
- Now skips invalid/obsolete saved conditions in load()
- Remaining searchConditionIDs are no longer affected by removeCondition() (i.e. they now act like autoincrements), which should make interface code simpler
- Changed default join mode to ALL
- Fixed loading of saved searches with no search conditions
