8 commits

Author	SHA1	Message	Date
Simon Kornblith	77282c3edc	- fixes a bug that could result in scrapers using utilities.processDocuments malfunctioning ... - fixes a bug that could result in the Scrape Progress chrome thingy sticking around forever - makes chrome thingy disappear when URL changes or when tabs are switched	2006-06-29 03:22:10 +00:00
Simon Kornblith	45b9234996	addresses #78, figure out import/export architecture ... - changes scrapers table to translators table; all import/export/web translators now belong in this table - adds Scholar.Translate to handle translation issues. eventually, Scholar.Ingester.Document will become part of this interface - adds Scholar_File_Interface (in fileInterface.js) to handle UI for export and eventually import. (David, when you have time, please connect Scholar_File_Interface.exportFile to a button.) - adds an export translator for MODS. all of our metadata, but not our hierarchy (projects, etc.) translates directly and unambiguously into valid MODS. eventually, we can use RDF or another format to handle hierarchy. - adds utilities.getVersion() and utilities.inArray() for simplified scraper coding - fixes minor interface issues with the nifty chrome scraping status window	2006-06-29 00:56:50 +00:00
Simon Kornblith	257ed8f69b	closes #68, figure out way to have scrapers work for gated resources behind proxies. most institutions use EZProxy for their proxy needs (or a more transparent proxy, which we support natively). this implementation is significantly better than the old one, which refused to work after you'd already logged in once, and is also simpler, because it's stateless. it has to observe every HTTP request, but there's no noticeable speed hit. it also still doesn't work when there's a link from one gated site to another gated site, but as far as i can tell, this only happens on the Gale Group site.	2006-06-27 04:08:21 +00:00
Simon Kornblith	4242c62b1b	- Fix redundancy in utilities.js (I accidentally copied and pasted a much larger block of code than i meant to) ... - Move processDocuments, a function for loading a DOM representation of a document or set of documents, to Scholar.Utilities.HTTP - Add Scholar.Ingester.ingestURL, a simplified function to scrape a URL (closes #33)	2006-06-26 20:02:30 +00:00
Simon Kornblith	4535b220db	Closes #84, make type icon in toolbar match item about to be scraped. It's not perfect, since to get everything right, we'd need to scrape the page as soon as it appears, but it provides a pretty good indication. Multiple items get the folder icon. If there's a better icon out there, it's pretty straightforward to implement.	2006-06-26 18:05:23 +00:00
Simon Kornblith	cb647aa607	remove vestigial code pieces and make usage clearer for Scholar.Utilities.HTTP	2006-06-26 16:32:19 +00:00
Simon Kornblith	ed47e0c84c	Forgot to commit updated utilities...	2006-06-26 16:19:44 +00:00
Simon Kornblith	7148852955	make generic Scholar.Utilities class and HTTP-dependent Scholar.Utilities.Ingester and Scholar.Utilities.HTTP classes in preparation for import/export filters; split off into separate javascript file	2006-06-26 14:46:57 +00:00