closes#100, migrate ingester to Scholar.Translate
closes#88, migrate scrapers away from RDF
closes#9, pull out LC subject heading tags
references #87, add fromArray() and toArray() methods to item objects
API changes:
all translation (import/export/web) now goes through Scholar.Translate
all Scholar-specific functions in scrapers start with "Scholar." rather than the jumbled up piggy bank un-namespaced confusion
scrapers now longer specify items through RDF (the beginning of an item.fromArray()-like function exists in Scholar.Translate.prototype._itemDone())
scrapers can be any combination of import, export, and web (type is the sum of 1/2/4 respectively)
scrapers now contain functions (doImport, doExport, doWeb) rather than loose code
scrapers can call functions in other scrapers or just call the function to translate itself
export accesses items item-by-item, rather than accepting a huge array of items
MARC functions are now in the MARC import translator, and accessed by the web translators
new features:
import now works
rudimentary RDF (unqualified dublin core only), RIS, and MARC import translators are implemented (although they are a little picky with respect to file extensions at the moment)
items appear as they are scraped
MARC import translator pulls out tags, although this seems to slow things down
no icon appears next to a the URL when Scholar hasn't detected metadata, since this seemed somewhat confusing
apologizes for the size of this diff. i figured if i was going to re-write the API, i might as well do it all at once and get everything working right.
- changes scrapers table to translators table; all import/export/web translators now belong in this table
- adds Scholar.Translate to handle translation issues. eventually, Scholar.Ingester.Document will become part of this interface
- adds Scholar_File_Interface (in fileInterface.js) to handle UI for export and eventually import. (David, when you have time, please connect Scholar_File_Interface.exportFile to a button.)
- adds an export translator for MODS. all of our metadata, but not our hierarchy (projects, etc.) translates directly and unambiguously into valid MODS. eventually, we can use RDF or another format to handle hierarchy.
- adds utilities.getVersion() and utilities.inArray() for simplified scraper coding
- fixes minor interface issues with the nifty chrome scraping status window
- Broke schema functions into separate object and got rid of DB_VERSION config constant in favor of a toVersion variable in the _migrateSchema command (which isn't technically necessary either, since the version number at the top of schema.sql is now always compared to the DB version at startup) but will help reduce the chance that someone will update the schema file without adding migration steps)
- Removed Amazon scraper from schema.sql, as it will be loaded with the rest of the scrapers
Scholar.Notifier framework to handle update notifications between data layer and interface
- Notifier.registerColumnTree(ref) and Notifier.registerItemTree(ref) pass back a unique hash that can be used to unregister the tree later with unregister*(hash) (and must be, lest we leak treeViews and probably entire browser windows if the browser window is closed without unregistering the treeView)
- Data layer calls Scholar.Notify.trigger(event, type, id) after various events (only two calls in the data layer at the moment--more coming later)
- Notify.trigger() calls notify(event, type, id) on all registered trees of the appropriate type -- the data layer usually knows what collection the action pertains to, but we decided that it's cleaner to just let the tree decide what it wants to do rather than add all that logic into the data layer)
(Note: Item collection adds appear to be buggy on the interface side, but removes seem to be working)
More details coming on Basecamp: http://chnm.grouphub.com/W161222
*** Important note: after checking out this code, be sure to delete the compreg.dat and xpti.dat files in your FF profile directory or else the extension will not load. They'll be regenerated when you start FF again, and you won't have to do it again unless we add interfaces or other components. Once I set up the XPI packaging system, this step won't be necessary. ***
- Localization properties are now loaded directly via nsIStringBundleService and available via Scholar.getString(name) -- removed stringbundle from XUL
- Updated status line in bottom right to reflect whether Scholar is loaded correctly or not (temporary, obviously)
