Commit graph

11242 commits

Author SHA1 Message Date
Dan Stillman
3aae8d3f89 Fix Scholar.getItems() to not trigger an SQL query for every item 2006-06-06 19:06:40 +00:00
Simon Kornblith
152c9bf9e7 - Small changes to MARC record support
- Implemented loadDocument API, for loading and parsing the DOMs of HTML documents in the background
- Added scraper code to SVN repository (now includes 12 scrapers, see Writeboard for details)

To update to the latest versions of all scrapers, ensure you have an up-to-date version of sqlite3, then run:
sqlite3 ~/Library/Application\ Support/Firefox/Profiles/profileName/scholar.sqlite < scrapers.sql
2006-06-06 18:25:45 +00:00
Dan Stillman
6c55e63eab Add ' et al.' to firstCreator field when more than one creator 2006-06-06 17:37:52 +00:00
Dan Stillman
60f7c8fccd Scholar.Items.search(text) -- (extremely) simple fulltext search on all fields -- returns an array of ids of matching items
The real search function will be considerably more advanced/flexible, but this should work as a placeholder for the moment. Probably quick enough for FAYT, at least with a ~0.5 second delay to avoid unnecessary calls while people are typing (which is probably a good idea anyway). This search doesn't use indexes at all, so if more speed is needed, one option would be to maintain a manual FULLTEXT-type index (using triggers, ideally) that could be quickly searched, but we'd lose intra-word filtering, which people would probably expect...
2006-06-06 08:02:29 +00:00
Dan Stillman
9675ac9d01 New function Scholar.Collection.changeParent([parentCollectionID]) -- move a collection into another collection in the DB
parentCollectionID is optional and defaults to moving item to root

Returns true on success, false on attempt to move collection into its existing parent, itself or a descendent collection; throws exception on invalid parentCollectionID

Sends a columnTree notify() with previous parent ID, id of collection itself, and new parent ID (unless the previous or new are the root, in which case it's omitted) -- that may or may not make sense for the interface code and can be changed if needed
2006-06-05 21:58:01 +00:00
Dan Stillman
b31be7af7e Put square brackets around ids in notifier debug output if multiple ids passed 2006-06-05 21:51:21 +00:00
David Norton
3796741911 Changed all references of 'folder' to 'collection' in code to correspond to database / dataacess nomenclature. 2006-06-05 18:01:53 +00:00
David Norton
560864ad6b Converted interface strings into localized entities/properties.
Fixed: New Item shows the metadata pane if it is not already shown.
2006-06-05 17:47:36 +00:00
David Norton
0a234c9f70 Implemented Scholar.Date, a group of functions for handling date formatting, etc. 2006-06-05 16:19:11 +00:00
David Norton
5411d4b67f Sorting on a colum: Much, much, much, better. (hope)fully implemented! 2006-06-05 15:49:11 +00:00
David Norton
56b1e37c13 New items are properly displayed on a notify()
New items are automatically selected.

When adding a new item, or editing an item without a creator, display an empty creator box.

Edit button changed from toggling to Save/Cancel buttons.
Simple "Do you want to save changes?" dialog if you select a different item while in edit mode.
2006-06-05 13:56:59 +00:00
David Norton
adfcc5cd73 Collection rename works (presently, using a somewhat awkward "rename collection" button)
Scholar Toolbar split into two (you can't really see much difference) - one for collections, one for items.
Scholar Toolbar moved above lists.
2006-06-05 13:08:16 +00:00
David Norton
c4d2e77b27 Interface: Add Collections (debugged to view properly)
Interface: Delete collection now reselects correctly.
2006-06-05 11:58:16 +00:00
Dan Stillman
d22d77fedf More helpful exceptions in itemFields methods when given an invalid fieldID or a fieldID with no associated itemTypes (neither should actually happen once things are set up right) 2006-06-04 19:37:03 +00:00
Dan Stillman
1d9fa85c15 Missing association of ISSN with journalArticle type -- added to sample data (thanks Simon) 2006-06-04 18:53:22 +00:00
Simon Kornblith
85d8153024 Add library, hooks for scraping MARC records. 2006-06-03 22:26:01 +00:00
Dan Stillman
bd840a4c58 Collection names must be non-empty -- interface side might follow iTunes behavior ('untitled playlist') here 2006-06-03 20:51:48 +00:00
Dan Stillman
e2c9ee0387 - Collection.rename(name) -- renames collection in DB
- Collection.add( name [, parentCollectionID] ) -- saves new collection in DB and returns new Collection object; defaults to root if parent not provided

	Currently some exceptions in folderTreeView.js when notified on collection add

- Added back Item.isCollection() and Collection.isCollection(), as they seem to still be used

- Collection._load() now takes optional ids for loading new collections
2006-06-03 20:41:23 +00:00
Dan Stillman
d01ff389d4 Use the Hash datatype in Notifier to store observers and improve debug output to display how many we have registered when trigger() is called 2006-06-03 20:26:31 +00:00
Dan Stillman
98f42001cc New function Scholar.getRandomID( table, column [, max] ) for getting unique random keys in a DB table, because auto_increments make me feel dirty -- max is optional and defaults to 16383 (which should store in 2 bytes in SQLite), but increases automatically if a unique id can't be found after 10 tries 2006-06-03 20:23:19 +00:00
Dan Stillman
bbbb243315 Accept 'null':true as bound parameter type 2006-06-03 20:06:57 +00:00
Simon Kornblith
93652a137c Fix issues with asynchronous scraping and XMLHttpRequest 2006-06-02 23:53:42 +00:00
David Norton
c42991a5bf Metadata now is much more compact when not being edited.
Close buttons on the Metadata and Notes.

The Collections list no longer throws an exception when the header is clicked.
More minor interface improvements.
2006-06-02 23:05:24 +00:00
Dan Stillman
c561ca129e Reverting r111 change to bound parameters for Item.erase() (minus the exception fix) -- bound parameters/multiple statements/no can do
Item deleting should work properly now
2006-06-02 21:11:14 +00:00
Dan Stillman
338bb54fdc Adjust flattenArguments() to properly handle scalar values -- so now it can be passed pretty much anything and just return a flattened array
Fixed typo in notifier debug() statement
2006-06-02 20:51:34 +00:00
Dan Stillman
013af36855 Fixed some issues with object reloading -- because of the way JS closures work, simply overwriting the old object doesn't work, since the interface code retains a reference to the old object -- in other words, a memory leak and stale data. Existing item objects now reload in place instead. Same change made to collections, though I'll have to test that after I add Collection editing and save().
This behavior of JS has some implications for how interface code handles things like deletes. The following won't do what one might expect/desire (and my apologies if this is obvious or redundant, but I figure it's worth pointing out):

var item = Scholar.Items.get(1);
Scholar.debug(item);
Scholar.DB.query("DELETE FROM items WHERE itemID=1");
Scholar.Items.reloadAll();
Scholar.debug(item);

The last line will still display the old object, even though _items[1] has been deleted inside Scholar.Items. The following does work, however:

var item = Scholar.Items.get(1);
Scholar.debug(item);
Scholar.DB.query("DELETE FROM items WHERE itemID=1");
Scholar.Items.reloadAll();
var item = Scholar.Items.get(1);
Scholar.debug(item);

Now item is properly undefined. Moral of the story: object references need to be deleted manually after receiving delete notifications, for if external code still has references to deleted data objects, the data layer can't do much about it.
2006-06-02 19:48:22 +00:00
Simon Kornblith
bb57e6ba7d Provide visual feedback for scraping 2006-06-02 18:22:34 +00:00
Dan Stillman
8f34487205 Fixed error in new item itemData insert (trying to bind across multiple statements) (yes we need unit tests)
Missing parens in Items._load() causing it to load all items in !all mode

Show type in debug output when binding params
2006-06-02 16:37:17 +00:00
Dan Stillman
60ada26e8d Delete from the right array when unregistered itemTrees 2006-06-02 15:41:11 +00:00
David Norton
f37ab47b39 Revised the way the ItemTreeView handles multiple-selection deletes.
Revised the way the dateAdded and dateModified columns are displayed.
Views are now unregistered on window close.

A few functions in overlay.js were renamed.
Began better commenting of the interface code.
2006-06-02 15:27:52 +00:00
David Norton
2a117168a9 Interface code for delete selected collection. (doesn't the function deleteCollectionSelection just sound great?)
Although I do not yet have interface code for it, collections do not have any sort of setName(), save() functions.
Collections list now implements notify()
2006-06-02 14:11:23 +00:00
David Norton
fdd8245eec Some experimentation with drag and drop (doesn't work yet)
Made some fixes on itemTreeView to allow for notify() on multiple deletes
	(shifts the row identifier up 1 for each previous delete, etc.)
	One more thing -- this might not happen because the tree calls each Item.erase() or Collection.removeItem() individually.
Bug fix on notifier.js so that register*Tree() actually returns the hash. :-)
	Unfortunately, unregister*Tree() does not seem to actually work, there is still an object at _observers['itemTree'][hash]
(temporary): When you select an item Scholar no longer loads a page into the browser.
	The javascript bugs on those HTML pages were frustrating the debugger.
2006-06-02 12:59:58 +00:00
Dan Stillman
146d4811ab Changed hyphen in extension version number to period to fix XPI update mechanism (which considered -r99 to be greater than -r100) 2006-06-02 07:35:02 +00:00
Dan Stillman
bebf0f223a Fix exception when erasing items and switch to bound parameter 2006-06-02 07:11:54 +00:00
Dan Stillman
daa1a6e8a5 Fix debug message in Notifier to properly handle int or array for ids 2006-06-02 07:04:13 +00:00
Dan Stillman
a7d5685da7 Updated Item.save() to use bound parameters for most queries so that values are properly escaped (and for somewhat cleaner code) 2006-06-02 07:03:24 +00:00
Simon Kornblith
639a006efb XPCOM-ize ingester, fix swapped first and last name in ingested info, stop ingesting pages field (this should be for pages of the source used, not the total number of pages, right?) 2006-06-02 03:19:12 +00:00
Dan Stillman
864fa537b5 Last of the update triggers, I think 2006-06-02 01:17:44 +00:00
Dan Stillman
70630a2e70 - Change: Scholar.Notifier.trigger() can be passed an array of ids in some cases rather than a single id -- trees that implement notify() should use Scholar.flattenArguments(ids) to get an array either way
- Added Notifier triggers to Item.erase() (which only runs if not in a nested transaction) and Collections.erase()

- notify() in ItemTreeView updated with example of taking multiple ids, though it doesn't actually work (and notify() implementations may decide just to refresh the whole tree when ids.length>1 rather than dealing with changes individually)

- When deleting collections, use DB tables that actually exist
2006-06-01 22:19:21 +00:00
Dan Stillman
4e0dbb3885 DB.transactionInProgress() -- this generally shouldn't be needed externally, but due to nested transactions there can be cases where it's unavoidable 2006-06-01 22:04:09 +00:00
Dan Stillman
8572aefbcb Forgot DROP TABLE IF EXISTS for scrapers table 2006-06-01 21:58:03 +00:00
Dan Stillman
d323df4a2d window.confirm() replacement for XPCOM 2006-06-01 21:57:12 +00:00
Dan Stillman
38d87463af Item.inCollection(collectionID) -- test if an item belongs to a given collection (currently uses a DB call--I'll optimize that later)
Scholar.Notifier framework to handle update notifications between data layer and interface

 - Notifier.registerColumnTree(ref) and Notifier.registerItemTree(ref) pass back a unique hash that can be used to unregister the tree later with unregister*(hash) (and must be, lest we leak treeViews and probably entire browser windows if the browser window is closed without unregistering the treeView)

 - Data layer calls Scholar.Notify.trigger(event, type, id) after various events (only two calls in the data layer at the moment--more coming later)

 - Notify.trigger() calls notify(event, type, id) on all registered trees of the appropriate type -- the data layer usually knows what collection the action pertains to, but we decided that it's cleaner to just let the tree decide what it wants to do rather than add all that logic into the data layer)

 (Note: Item collection adds appear to be buggy on the interface side, but removes seem to be working)
2006-06-01 20:03:53 +00:00
Dan Stillman
0d27815694 Scholar.randomString( [int len] ) -- generate a random string 2006-06-01 19:46:57 +00:00
David Norton
c2c0f0f614 "New Item..." fixed for new Metadatapane overlay.
Minor interface improvements.

(Revision 100!!!!!!!!!!!!!!!!!!)
2006-06-01 18:50:16 +00:00
Dan Stillman
024e1f78e3 Call varDump automatically in Scholar.debug() on non-strings -- no more nesting varDump() 2006-06-01 18:43:44 +00:00
David Norton
d65bf82e0e ItemTreeView now has support for receiving change events. (needs support on the data layer, and probably bug fixes).
(+ some cosmetic changes)
2006-06-01 17:54:41 +00:00
David Norton
c1241b608a Removed organizeWindow.xul/js (All this functionality is in overlay)
Removed view.xul/js/css (see above)
Enabled deletion (exception when deleting from library)
2006-06-01 16:40:43 +00:00
Dan Stillman
b9a5c7dc38 Fixed creators not getting inserted into itemCreators on new item save() 2006-06-01 07:29:39 +00:00
Simon Kornblith
551582eb7e Still getting the hang of Subversion...the rest of the ingester code 2006-06-01 06:53:39 +00:00