153 commits

Author	SHA1	Message	Date
Dan Stillman	393807b152	This isn't quite done (I'm discussing changing the scrapers schema with Simon to better handle scraper updates) but in the interest of getting the scrapers in for testing, I'll commit this now. ... Integrated the scrapers with the schema update mechanism. Changed a bunch of schema methods to handle both schema.sql and scrapers.sql (or others, if need be) and altered the version table to track mu ltiple versions for different files. This theoretically should detect that the version table has changed and force a reinitialization of the DB--let me know if there are problems.	2006-06-07 15:27:21 +00:00
Simon Kornblith	0753d78910	- Add VLTS scraper ... - Fix loadDocument/processDocuments (broken by r145)	2006-06-06 21:35:23 +00:00
Simon Kornblith	152c9bf9e7	- Small changes to MARC record support ... - Implemented loadDocument API, for loading and parsing the DOMs of HTML documents in the background - Added scraper code to SVN repository (now includes 12 scrapers, see Writeboard for details) To update to the latest versions of all scrapers, ensure you have an up-to-date version of sqlite3, then run: sqlite3 ~/Library/Application\ Support/Firefox/Profiles/profileName/scholar.sqlite < scrapers.sql	2006-06-06 18:25:45 +00:00