mirrors/zotero
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
13500 commits 9 branches 362 tags 278 MiB
Commit graph

13500 commits

This branch
This branch
All branches
Author SHA1 Message Date
Simon Kornblith
19504e6746 - closes #73, use chrome for "Scraping Progress..." indicator 
- multiple and book icons were swapped for Voyager scraper
 2006-06-27 02:03:10 +00:00
Simon Kornblith
f1cc809f76 Add a generic scraper that will scrape any website, although it may not always find very much information. It looks at META tags, both Dublin Core and otherwise. 
When tags are ready, we can pull out META keywords.
 2006-06-26 20:44:45 +00:00
Dan Stillman
031d78eeee Be a bit more helpful when calling getString() on a string with no localization 2006-06-26 20:41:09 +00:00
Simon Kornblith
4242c62b1b - Fix redundancy in utilities.js (I accidentally copied and pasted a much larger block of code than i meant to) 
- Move processDocuments, a function for loading a DOM representation of a document or set of documents, to Scholar.Utilities.HTTP
- Add Scholar.Ingester.ingestURL, a simplified function to scrape a URL (closes #33)
 2006-06-26 20:02:30 +00:00
David Norton
76c118e9e8 Fixes #25, add metadata to top of note 2006-06-26 19:38:56 +00:00
David Norton
77eadc3ea0 Treeview: Fixes problem with added items 2006-06-26 18:41:58 +00:00
Simon Kornblith
4535b220db Closes #84, make type icon in toolbar match item about to be scraped. It's not perfect, since to get everything right, we'd need to scrape the page as soon as it appears, but it provides a pretty good indication. Multiple items get the folder icon. If there's a better icon out there, it's pretty straightforward to implement. 2006-06-26 18:05:23 +00:00
David Norton
a5a27a7400 Fixes #27, collapsable/hierarchical notes in center pane. 
- This required moving the icon to the title field so that the indent would work out right. The type column (which for new installs will be hidden) displays the type in text.
- I expect several small bugs in regard to this.
 2006-06-26 17:51:18 +00:00
Simon Kornblith
cb647aa607 remove vestigial code pieces and make usage clearer for Scholar.Utilities.HTTP 2006-06-26 16:32:19 +00:00
Simon Kornblith
ed47e0c84c Forgot to commit updated utilities... 2006-06-26 16:19:44 +00:00
Simon Kornblith
04730860a6 Move Scholar.HTTP to Scholar.Utilities.HTTP; create Scholar.Utilities.Ingester.HTTPUtilities to handle proxied URLs for Ingester 2006-06-26 16:18:55 +00:00
Simon Kornblith
7148852955 make generic Scholar.Utilities class and HTTP-dependent Scholar.Utilities.Ingester and Scholar.Utilities.HTTP classes in preparation for import/export filters; split off into separate javascript file 2006-06-26 14:46:57 +00:00
David Norton
1096a95f62 Several little interface bugs fixed regarding Notes. 2006-06-26 14:46:21 +00:00
David Norton
b94cf81ba3 Trying this again: 
Fixes #22, #26, #79, #71

Added remove note button, removed some old code.
 2006-06-26 13:54:05 +00:00
David Norton
fb9e803ab5 closes #22, button in note pane for new note 
closes #26, notes list in notes pane
closes #79, add icons for new object types
fixes #71, Metadata pane should be refreshed on a notify() event for the selected item
 2006-06-26 12:58:22 +00:00
Simon Kornblith
a33b119dff grab ISBN from SIRSI 2003+ catalogs 2006-06-26 01:17:29 +00:00
Simon Kornblith
303c6ee68d closes #41, get library call number 2006-06-26 01:08:59 +00:00
Simon Kornblith
d73127b1b3 update modification times 2006-06-25 22:01:04 +00:00
Simon Kornblith
f6b0d9a541 search results scraping for InfoTrac. closes #15 2006-06-25 22:00:20 +00:00
Simon Kornblith
1ec834cef2 Search results scraping for Project MUSE 2006-06-25 21:12:14 +00:00
Dan Stillman
05c8b0e467 Fixes #60, make sure it works well offline 
- Added detection for network failure -- debug message is output and noNetwork property is added to the xmlhttp object

- Removed onStatus callback from HTTP.doGet and HTTP.doPost -- that was copied over from the Piggy Bank API, but the onDone callback has to handle errors anyway, so it can just check the status code if it actually cares to differentiate non-200 status codes from any other error

- Added error handling for empty responseXML to Schema._updateScrapersRemoteCallback

- Renamed SCHOLAR_CONFIG['REPOSITORY_CHECK_RETRY'] to SCHOLAR_CONFIG['REPOSITORY_RETRY_INTERVAL']
 2006-06-25 20:14:11 +00:00
Simon Kornblith
6a627fad0a Search results scraping for LexisNexis 2006-06-25 20:09:27 +00:00
Simon Kornblith
a48ea7dabf Search results scraping for ProQuest 2006-06-25 19:32:49 +00:00
Simon Kornblith
7402577806 Add search results scraping for History Cooperative 2006-06-25 18:34:23 +00:00
Simon Kornblith
a9c79f6110 Search results scraping for JSTOR 2006-06-25 18:17:00 +00:00
Dan Stillman
480f9d56f6 Fixes #14, Add a callNumber field to all item types, and fixes #75, Add an "extra" field to all item types 
I also added accessionNumber to all types (except website), since that's what Endnote does, but we may or may not think that's necessary
 2006-06-25 18:10:27 +00:00
Simon Kornblith
5e73dcdd2e - Search results scraping for WorldCat. 
- Make scraperJavaScript run on reload again, because it makes debugging easier
- There's not actually a memory leak in the proxyMonitor code.
 2006-06-25 16:13:47 +00:00
Dan Stillman
dc8c695855 Fixes #11, Observe user pref to turn off automatic scraper updates 2006-06-25 07:34:03 +00:00
Dan Stillman
691993a6c3 Added Scholar.Prefs, a front-end to the preferences service with simple get(pref) and set(pref, value) methods that will retrieve and set based on the type of the default value, which I believe should generally should fine -- for more fine-grained control, use Scholar.Prefs.prefBranch to access the branch directly 
Scholar.Prefs also registers itself as a preferences observer and can be used to trigger actions when certain prefs are changed by editing the switch statement in the observe() method

Updated preferences.js to use Scholar.Prefs
 2006-06-25 07:31:01 +00:00
Dan Stillman
b679bc6327 Fixes #62, Cache notes count for items 2006-06-25 05:43:00 +00:00
Dan Stillman
a207bf3817 Fixed error in History.add() call in Item.save() that was breaking new item inserts 2006-06-25 05:26:53 +00:00
Simon Kornblith
9e78d62b13 Better handling of itemTypes, and improved date handling in PubMed scraper. 2006-06-25 05:03:01 +00:00
David Norton
f941bc192c Switched Info and Notes tab back to their places. :-) 2006-06-25 04:57:52 +00:00
Dan Stillman
b2247e1dd2 Fixes #66, Need a function to get typeID given typeName 
- Added methods getID(idOrName) and getName(idOrName) to Scholar.CreatorTypes and Scholar.ItemTypes to take either typeID or typeName

- Removed getTypeName() in each and changed references accordingly

- Streamlined both classes to be as similar as possible
 2006-06-25 04:35:11 +00:00
Simon Kornblith
22eebc6cdf Addresses #68, figure out way to have scrapers work for gated resources behind proxies. We can now access pages through an EZProxy. We need to know what alternatives to EZProxy exist in order to support them. Also, fixes some spacing issues in browser.js. 2006-06-25 04:30:43 +00:00
Dan Stillman
b8ad832e74 Begin to work undo functionality into data layer -- currently just for Item.save() 
History.undo()/redo() now reload the item, though changes won't show in open metadata pane due to #71

refs #67
 2006-06-25 04:11:19 +00:00
Simon Kornblith
f897564f0e Temporary fix to get ingested item types right until #66 is implemented 2006-06-24 21:44:36 +00:00
Simon Kornblith
40fabb888c Addresses #65, back button fools ingester, and fixes bugs loading new tabs in the background. 2006-06-24 21:39:36 +00:00
Simon Kornblith
fd2052e63c Search results scraping for PubMed and Google Books. This marks the end of what I can do with respect to #15 until I'm at home or CHNM, where I'll have access to the gated collections. 2006-06-24 17:33:35 +00:00
Simon Kornblith
260ce80086 - Search results scraping for TLC. This is the last of the library scrapers. 
- Minor fixes to ingester utilities.
 2006-06-24 15:38:53 +00:00
Simon Kornblith
06cf9e7853 Search results scraping for SIRSI (old versions) 2006-06-24 14:35:05 +00:00
Dan Stillman
97940c7470 Replaced all instances of "Firefox Scholar" (not counting the repository URL) with "Scholar for Firefox" for now 2006-06-24 09:08:12 +00:00
Dan Stillman
720960feb9 Addresses #5, Add as many item types as possible 
New item types from Elena
 2006-06-24 08:28:37 +00:00
Dan Stillman
22df47a0d3 A rough interface to add, remove and reorder item type fields -- at the moment just to make my life easier, but eventually could be adapted to allow end-user type/field editing, if we decide that's necessary 2006-06-24 08:23:54 +00:00
Dan Stillman
42578ace59 Fixes #61, creator caching sometimes malfunctions 2006-06-23 22:00:39 +00:00
Simon Kornblith
6f19b215f5 Search result scraping for GEAC catalogs 2006-06-23 21:27:32 +00:00
Simon Kornblith
2b58ead7aa Search results scraping for Dynix 2006-06-23 20:53:29 +00:00
Simon Kornblith
2a74e88416 - Make generalized function for finding search results case insensitive 
- Scrape DRA search results
 2006-06-23 20:09:48 +00:00
Simon Kornblith
8fe72b3e3c Search results scraping for VTLS 2006-06-23 19:22:24 +00:00
Simon Kornblith
641d7054cc - Fixed some bugs in the InnoPAC scraper (search results) 
- Made an Aleph search results scraper that works correctly on most sites, and degrades nicely when it doesn't
 2006-06-23 17:35:57 +00:00