Commit graph

193 commits

Author SHA1 Message Date
Dan Stillman
448fde9ff1 Made the schema update system moderately less convoluted
- Broke schema functions into separate object and got rid of DB_VERSION config constant in favor of a toVersion variable in the _migrateSchema command (which isn't technically necessary either, since the version number at the top of schema.sql is now always compared to the DB version at startup) but will help reduce the chance that someone will update the schema file without adding migration steps)

- Removed Amazon scraper from schema.sql, as it will be loaded with the rest of the scrapers
2006-06-07 01:02:59 +00:00
Dan Stillman
9bcaad5946 Added optional second param to Items.search() to restrict search to a particular collectionID 2006-06-06 20:46:08 +00:00
Dan Stillman
c807d335e9 Changed Items.get() to return a regularly indexed array instead of one indexed by itemID--nothing actually used the keyed array, and this way it can be used for building trees directly (e.g. Scholar.Items.get(Scholar.Items.search('barnes'))) 2006-06-06 20:15:50 +00:00
David Norton
b65a56b4d9 Fairly major metadatapane reworking. It is now nextdoor to the items list.
-  Lots of work to be done. For example, the present way of showing a textbox is sort of a hack - taking a label, assigning certain properties to a textbox, then removing the label and placing the textbox in its place. I will be looking into our options.
   -  Also, I need to figure out adding/editing/deleting creators.
   -  When the textbox loses focus, it updates and saves the item (like iCal).

Date: removed Scholare.Date functions, as they are overkill. We can use the built-in Date.toLocaleString()
2006-06-06 19:53:27 +00:00
Dan Stillman
3aae8d3f89 Fix Scholar.getItems() to not trigger an SQL query for every item 2006-06-06 19:06:40 +00:00
Simon Kornblith
152c9bf9e7 - Small changes to MARC record support
- Implemented loadDocument API, for loading and parsing the DOMs of HTML documents in the background
- Added scraper code to SVN repository (now includes 12 scrapers, see Writeboard for details)

To update to the latest versions of all scrapers, ensure you have an up-to-date version of sqlite3, then run:
sqlite3 ~/Library/Application\ Support/Firefox/Profiles/profileName/scholar.sqlite < scrapers.sql
2006-06-06 18:25:45 +00:00
Dan Stillman
6c55e63eab Add ' et al.' to firstCreator field when more than one creator 2006-06-06 17:37:52 +00:00
Dan Stillman
60f7c8fccd Scholar.Items.search(text) -- (extremely) simple fulltext search on all fields -- returns an array of ids of matching items
The real search function will be considerably more advanced/flexible, but this should work as a placeholder for the moment. Probably quick enough for FAYT, at least with a ~0.5 second delay to avoid unnecessary calls while people are typing (which is probably a good idea anyway). This search doesn't use indexes at all, so if more speed is needed, one option would be to maintain a manual FULLTEXT-type index (using triggers, ideally) that could be quickly searched, but we'd lose intra-word filtering, which people would probably expect...
2006-06-06 08:02:29 +00:00
Dan Stillman
9675ac9d01 New function Scholar.Collection.changeParent([parentCollectionID]) -- move a collection into another collection in the DB
parentCollectionID is optional and defaults to moving item to root

Returns true on success, false on attempt to move collection into its existing parent, itself or a descendent collection; throws exception on invalid parentCollectionID

Sends a columnTree notify() with previous parent ID, id of collection itself, and new parent ID (unless the previous or new are the root, in which case it's omitted) -- that may or may not make sense for the interface code and can be changed if needed
2006-06-05 21:58:01 +00:00
Dan Stillman
b31be7af7e Put square brackets around ids in notifier debug output if multiple ids passed 2006-06-05 21:51:21 +00:00
David Norton
0a234c9f70 Implemented Scholar.Date, a group of functions for handling date formatting, etc. 2006-06-05 16:19:11 +00:00
Dan Stillman
d22d77fedf More helpful exceptions in itemFields methods when given an invalid fieldID or a fieldID with no associated itemTypes (neither should actually happen once things are set up right) 2006-06-04 19:37:03 +00:00
Dan Stillman
1d9fa85c15 Missing association of ISSN with journalArticle type -- added to sample data (thanks Simon) 2006-06-04 18:53:22 +00:00
Simon Kornblith
85d8153024 Add library, hooks for scraping MARC records. 2006-06-03 22:26:01 +00:00
Dan Stillman
bd840a4c58 Collection names must be non-empty -- interface side might follow iTunes behavior ('untitled playlist') here 2006-06-03 20:51:48 +00:00
Dan Stillman
e2c9ee0387 - Collection.rename(name) -- renames collection in DB
- Collection.add( name [, parentCollectionID] ) -- saves new collection in DB and returns new Collection object; defaults to root if parent not provided

	Currently some exceptions in folderTreeView.js when notified on collection add

- Added back Item.isCollection() and Collection.isCollection(), as they seem to still be used

- Collection._load() now takes optional ids for loading new collections
2006-06-03 20:41:23 +00:00
Dan Stillman
d01ff389d4 Use the Hash datatype in Notifier to store observers and improve debug output to display how many we have registered when trigger() is called 2006-06-03 20:26:31 +00:00
Dan Stillman
98f42001cc New function Scholar.getRandomID( table, column [, max] ) for getting unique random keys in a DB table, because auto_increments make me feel dirty -- max is optional and defaults to 16383 (which should store in 2 bytes in SQLite), but increases automatically if a unique id can't be found after 10 tries 2006-06-03 20:23:19 +00:00
Dan Stillman
bbbb243315 Accept 'null':true as bound parameter type 2006-06-03 20:06:57 +00:00
Simon Kornblith
93652a137c Fix issues with asynchronous scraping and XMLHttpRequest 2006-06-02 23:53:42 +00:00
Dan Stillman
c561ca129e Reverting r111 change to bound parameters for Item.erase() (minus the exception fix) -- bound parameters/multiple statements/no can do
Item deleting should work properly now
2006-06-02 21:11:14 +00:00
Dan Stillman
338bb54fdc Adjust flattenArguments() to properly handle scalar values -- so now it can be passed pretty much anything and just return a flattened array
Fixed typo in notifier debug() statement
2006-06-02 20:51:34 +00:00
Dan Stillman
013af36855 Fixed some issues with object reloading -- because of the way JS closures work, simply overwriting the old object doesn't work, since the interface code retains a reference to the old object -- in other words, a memory leak and stale data. Existing item objects now reload in place instead. Same change made to collections, though I'll have to test that after I add Collection editing and save().
This behavior of JS has some implications for how interface code handles things like deletes. The following won't do what one might expect/desire (and my apologies if this is obvious or redundant, but I figure it's worth pointing out):

var item = Scholar.Items.get(1);
Scholar.debug(item);
Scholar.DB.query("DELETE FROM items WHERE itemID=1");
Scholar.Items.reloadAll();
Scholar.debug(item);

The last line will still display the old object, even though _items[1] has been deleted inside Scholar.Items. The following does work, however:

var item = Scholar.Items.get(1);
Scholar.debug(item);
Scholar.DB.query("DELETE FROM items WHERE itemID=1");
Scholar.Items.reloadAll();
var item = Scholar.Items.get(1);
Scholar.debug(item);

Now item is properly undefined. Moral of the story: object references need to be deleted manually after receiving delete notifications, for if external code still has references to deleted data objects, the data layer can't do much about it.
2006-06-02 19:48:22 +00:00
Simon Kornblith
bb57e6ba7d Provide visual feedback for scraping 2006-06-02 18:22:34 +00:00
Dan Stillman
8f34487205 Fixed error in new item itemData insert (trying to bind across multiple statements) (yes we need unit tests)
Missing parens in Items._load() causing it to load all items in !all mode

Show type in debug output when binding params
2006-06-02 16:37:17 +00:00
Dan Stillman
60ada26e8d Delete from the right array when unregistered itemTrees 2006-06-02 15:41:11 +00:00
David Norton
fdd8245eec Some experimentation with drag and drop (doesn't work yet)
Made some fixes on itemTreeView to allow for notify() on multiple deletes
	(shifts the row identifier up 1 for each previous delete, etc.)
	One more thing -- this might not happen because the tree calls each Item.erase() or Collection.removeItem() individually.
Bug fix on notifier.js so that register*Tree() actually returns the hash. :-)
	Unfortunately, unregister*Tree() does not seem to actually work, there is still an object at _observers['itemTree'][hash]
(temporary): When you select an item Scholar no longer loads a page into the browser.
	The javascript bugs on those HTML pages were frustrating the debugger.
2006-06-02 12:59:58 +00:00
Dan Stillman
bebf0f223a Fix exception when erasing items and switch to bound parameter 2006-06-02 07:11:54 +00:00
Dan Stillman
daa1a6e8a5 Fix debug message in Notifier to properly handle int or array for ids 2006-06-02 07:04:13 +00:00
Dan Stillman
a7d5685da7 Updated Item.save() to use bound parameters for most queries so that values are properly escaped (and for somewhat cleaner code) 2006-06-02 07:03:24 +00:00
Simon Kornblith
639a006efb XPCOM-ize ingester, fix swapped first and last name in ingested info, stop ingesting pages field (this should be for pages of the source used, not the total number of pages, right?) 2006-06-02 03:19:12 +00:00
Dan Stillman
864fa537b5 Last of the update triggers, I think 2006-06-02 01:17:44 +00:00
Dan Stillman
70630a2e70 - Change: Scholar.Notifier.trigger() can be passed an array of ids in some cases rather than a single id -- trees that implement notify() should use Scholar.flattenArguments(ids) to get an array either way
- Added Notifier triggers to Item.erase() (which only runs if not in a nested transaction) and Collections.erase()

- notify() in ItemTreeView updated with example of taking multiple ids, though it doesn't actually work (and notify() implementations may decide just to refresh the whole tree when ids.length>1 rather than dealing with changes individually)

- When deleting collections, use DB tables that actually exist
2006-06-01 22:19:21 +00:00
Dan Stillman
4e0dbb3885 DB.transactionInProgress() -- this generally shouldn't be needed externally, but due to nested transactions there can be cases where it's unavoidable 2006-06-01 22:04:09 +00:00
Dan Stillman
38d87463af Item.inCollection(collectionID) -- test if an item belongs to a given collection (currently uses a DB call--I'll optimize that later)
Scholar.Notifier framework to handle update notifications between data layer and interface

 - Notifier.registerColumnTree(ref) and Notifier.registerItemTree(ref) pass back a unique hash that can be used to unregister the tree later with unregister*(hash) (and must be, lest we leak treeViews and probably entire browser windows if the browser window is closed without unregistering the treeView)

 - Data layer calls Scholar.Notify.trigger(event, type, id) after various events (only two calls in the data layer at the moment--more coming later)

 - Notify.trigger() calls notify(event, type, id) on all registered trees of the appropriate type -- the data layer usually knows what collection the action pertains to, but we decided that it's cleaner to just let the tree decide what it wants to do rather than add all that logic into the data layer)

 (Note: Item collection adds appear to be buggy on the interface side, but removes seem to be working)
2006-06-01 20:03:53 +00:00
Dan Stillman
0d27815694 Scholar.randomString( [int len] ) -- generate a random string 2006-06-01 19:46:57 +00:00
Dan Stillman
024e1f78e3 Call varDump automatically in Scholar.debug() on non-strings -- no more nesting varDump() 2006-06-01 18:43:44 +00:00
Dan Stillman
b9a5c7dc38 Fixed creators not getting inserted into itemCreators on new item save() 2006-06-01 07:29:39 +00:00
Dan Stillman
db1b85881c Scrapers table and sample scraper from Simon 2006-06-01 06:35:33 +00:00
Dan Stillman
03637a6c63 Updated schema for new collections ("projects"?) model and updated sample data accordingly; added a couple extra indexes; made 'rights' and 'source' regular metadata fields so that items can be used for things like notes
Overhauled data access layer to support new model:

	- Changed Item.getParent to Item.getCollections() to get ids of parent collections
	- Removed Item.setPosition() -- item positions are set internally when items are added to and deleted from collections, but the orderIndex is not currently used or manipulatable externally
	- Item constructor/Items.getNewItemByType()/Items.add() no longer take folderID and orderIndex as parameters
	- Split getTreeRows() into Scholar.getCollections(parent) and Scholar.getItems(parent), which return root collections or all library items, respectively, if no parent given
	- All references to folders in object/method/property names changed to collections
	- New methods Collection.addItem(itemID), Collection.hasItem(itemID), Collection.removeItem(itemID)
	- Collection.erase() takes optional deleteItems parameter to delete items from DB -- otherwise just removes association and leaves item in library (does, however, delete all descendent collections from DB regardless)

* Note: This will break displaying of items until interface code is updated. *
2006-06-01 00:22:18 +00:00
Dan Stillman
a4a1f9bd2b Looks like the name@domain format is indeed the recommended format for extension ids rather than GUIDs (thanks David). My bad.
Switching to scholar@chnm.gmu.edu -- SVN folks should rename the text file in their FF profile directories. (Used to be scholar@chnm, so you need to make the change even if you hadn't changed to the GUID yet.) Anyone who's tried the XPI yet will need to reinstall just once from the URL (http://chnm.gmu.edu/firefoxscholar/download/scholar-dev.xpi), as the old version won't find updates with its id.
2006-05-30 21:01:34 +00:00
Dan Stillman
b135f92ef9 Updated install.rdf format to current template from Mozilla.org and updated some values (extension is now version 0.1pre1)
Added update.rdf for auto-update mechanism (which will get moved out of the extension root by the build system but should stay with a particular tree)

Assigned a new GUID to extension

*** Important: The extension GUID has changed, so you should rename the scholar@chnm text file in your Firefox profile exensions directory to our new GUID "{e007c81c-eb2a-4e1d-b381-d25a45cd11ed}" and restart Firefox ***
2006-05-30 04:28:38 +00:00
Dan Stillman
bc0463774f Move XPCOM-included files into subdirectory to keep separate from chrome files and rename overlay.js to scholar.js 2006-05-27 00:44:24 +00:00