Closes#111, minor modifications to field list schema
Changes per above tickets and comments at http://chnm.grouphub.com/projects/310105/msg/cat/2333974/2995041/comments
- date, year and lastModified merged into date
- added date field to journalArticle
- added series, seriesTitle and seriesText to journalArticle
- added seriesNumber (along with series) to book and bookSection
If you have attachments to the old terminology, feel free to file a complaint.
Changed interface code too, since David is gone (or at the very least has more important things to do with his remaining time)
- 'isInstitution' parameter added to Item.setCreator(), Creators.getID(), Creators.add()
- 'isInstitution' property added to return from Creators.get() and Item.getCreator()
var obj = Scholar.Items.getNewItemByType(1);
obj.setField('title', 'Digital History for Dummies');
obj.setCreator(0, '', 'Center for History and New Media', 1, true); // true == institutional creator
var id = obj.save();
Note: 'firstName' field is ignored when 'isInstitution' is true
Conditions in ANY queries can be made required by passing 'true' as an extra parameter to addCondition() and updateCondition() -- this can be used for limiting ANY queries to particular collections (in place of the removed 'context' condition), but if there was an elegant way to expose it to the user for all ANY queries, it's something users might find very useful.
Implemented advanced/saved search architecture -- to use, you create a new search with var search = new Scholar.Search(), add conditions to it with addCondition(condition, operator, value), and run it with search(). The standard conditions with their respective operators can be retrieved with Scholar.SearchConditions.getStandardConditions(). Others are for special search flags and can be specified as follows (condition, operator, value):
'context', null, collectionIDToSearchWithin
'recursive', 'true'|'false' (as strings!--defaults to false if not specified, though, so should probably just be removed if not wanted), null
'joinMode', 'any'|'all', null
For standard conditions, currently only 'title' and the itemData fields are supported -- more coming soon.
Localized strings created for the standard search operators
API:
search.setName(name) -- must be called before save() on new searches
search.load(savedSearchID)
search.save() -- saves search to DB and returns a savedSearchID
search.addCondition(condition, operator, value)
search.updateCondition(searchConditionID, condition, operator, value)
search.removeCondition(searchConditionID)
search.getSearchCondition(searchConditionID) -- returns a specific search condition used in the search
search.getSearchConditions() -- returns search conditions used in the search
search.search() -- runs search and returns an array of item ids for results
search.getSQL() -- will be used by Dan for search-within-search
Scholar.Searches.getAll() -- returns an array of saved searches with 'id' and 'name', in alphabetical order
Scholar.Searches.erase(savedSearchID) -- deletes a given saved search from the DB
Scholar.SearchConditions.get(condition) -- get condition data (operators, etc.)
Scholar.SearchConditions.getStandardConditions() -- retrieve conditions for use in drop-down menu (as opposed to special search flags)
Scholar.SearchConditions.hasOperator() -- used by Dan for error-checking
Closes#118, add "translator" creator type
Closes#122, add DOI and abbreviated journal title fields
Addresses #45, reorder item fields -- source/rights moved down to bottom; date fields not yet moved
Not finished, but enough to give David something to work with
No BLOBs -- just linking/importing of files and loaded documents
New Scholar.Item methods:
incrementFileCount() (used internally)
decrementFileCount() (used internally)
isFile()
numFiles()
getFile() -- returns nsILocalFile or false if associated file doesn't exist (note: always returns false for items with LINK_MODE_LINKED_URL, since they have no files -- use getFileURL() instead)
getFileURL() -- returns URL string
getFileLinkMode() -- compare to Scholar.Files.LINK_MODE_* constants: LINKED_FILE, IMPORTED_FILE, LINKED_URL, IMPORTED_URL
getFileMimeType() -- mime type of file (e.g. text/plain)
getFileCharset() -- charsetID of file
getFiles() -- array of file itemIDs this file is a source for
New Scholar.Files methods:
importFromFile(nsIFile file [, int sourceItemID])
linkFromFile(nsIFile file [, int sourceItemID])
importFromDocument(nsIDOMDocument document [, int sourceItemID])
linkFromDocument(nsIDOMDocument document [, int sourceItemID])
New class Scholar.FileTypes -- partially implemented, not yet used
New class Scholar.CharacterSets -- same as other *Types classes:
getID(idOrName)
getName(idOrName)
getTypes() (aliased to getAll(), which I'll probably change the others to as well)
Charsets table with all official character sets (copied from Mozilla source)
Renamed Item.setNoteSource() to setSource() and Item.getNoteSource() to getSource() and adjusted to handle both notes and files
Item.addSeeAlso(itemID)
Item.removeSeeAlso(itemID)
Item.getSeeAlso() -- array of linked items
Relationships are mutual
Closes#82, Notes: see also table
- changes scrapers table to translators table; all import/export/web translators now belong in this table
- adds Scholar.Translate to handle translation issues. eventually, Scholar.Ingester.Document will become part of this interface
- adds Scholar_File_Interface (in fileInterface.js) to handle UI for export and eventually import. (David, when you have time, please connect Scholar_File_Interface.exportFile to a button.)
- adds an export translator for MODS. all of our metadata, but not our hierarchy (projects, etc.) translates directly and unambiguously into valid MODS. eventually, we can use RDF or another format to handle hierarchy.
- adds utilities.getVersion() and utilities.inArray() for simplified scraper coding
- fixes minor interface issues with the nifty chrome scraping status window
New methods:
Item.addTag(tag)
Item.getTags() -- array of tagIDs
Item.removeTag(tagID)
Tags.getName(tagID)
Tags.getID(tag)
Tags.add(text) -- returns tagID of new tag
Tags.purge() -- purge obsolete tags
The last two are for use by Item.addTag() and Item.removeTag(), respectively, and probably don't need to be used elsewhere.
- Added 'note' item type
- Updated API to support independent note creation
Notes are, more or less, just regular items, with an item type of 1. They're created through Scholar.Notes.add(text, sourceItemID), which returns the itemID of the new note. sourceItemID is optional--if left out, an independent note will be created. (There's currently nothing stopping you from doing getNewItemByType(1) yourself, but the note would be contentless and broken, so you shouldn't do that.) Note data could've been stuffed into itemData, but I kept it separate in itemNotes to keep metadata searching faster and to keep things cleaner.
Methods calls that can be called on all items:
isNote() (same as testing for itemTypeID 1)
Method calls that can be called on source items only:
numNotes()
getNotes() (array of note itemIDs for a source)
Method calls that can be called on note items only:
updateNote(text)
setNoteSource(sourceItemID) (for changing source--use empty or false to make independent, which is currently what happens when you delete a note--will get option with #91)
getNote() (note content)
getNoteSource() (sourceItemID of a note)
Calling the above methods on the wrong item types will throw an error.
*** This will break note creation/display until David updates interface code. ***
I didn't increment the schema version, since I doubt anyone actual updated in the last 20 minutes... (If so just delete your DB or use SCHOLAR_CONFIG['DB_REBUILD'])
- Removed localLastUpdated field from scrapers table and renamed centralLastUpdated to lastUpdated; updated scraper queries accordingly
- Added query in scrapers.sql to update version table 'repository' row to prevent immediate downloads of newly installed scrapers
- Get version property from extension manager in Scholar.init() and assign to Scholar.version
Assigned guids to scrapers, replaced INSERT queries with REPLACE queries, and removed table DELETE query at top -- this will allow scrapers to be updated without deleting any others that may exist (e.g. that someone is developing, third-party, etc.)
scholar.properties addition is just to keep metadata panel from breaking and could probably be removed once interface is hard-coded to not display the notes field there
Added creatorTypeID to the PK in itemCreators, because a creator could conceivably have two roles on an item
Throw an error if attempt to save() an item with two identical creator/creatorTypeID combinations, since, assuming this shouldn't be allowed (i.e. we don't have a four-column PK), there's really no way to handle this elegantly on my end -- interface code can use new method Item.creatorExists(firstName, lastName, creatorTypeID, skipIndex), with skipIndex set to the current index, to make sure this isn't done
Integrated the scrapers with the schema update mechanism. Changed a bunch of schema methods to handle both schema.sql and scrapers.sql (or others, if need be) and altered the version table to track mu
ltiple versions for different files. This theoretically should detect that the version table has changed and force a reinitialization of the DB--let me know if there are problems.
- Broke schema functions into separate object and got rid of DB_VERSION config constant in favor of a toVersion variable in the _migrateSchema command (which isn't technically necessary either, since the version number at the top of schema.sql is now always compared to the DB version at startup) but will help reduce the chance that someone will update the schema file without adding migration steps)
- Removed Amazon scraper from schema.sql, as it will be loaded with the rest of the scrapers
Overhauled data access layer to support new model:
- Changed Item.getParent to Item.getCollections() to get ids of parent collections
- Removed Item.setPosition() -- item positions are set internally when items are added to and deleted from collections, but the orderIndex is not currently used or manipulatable externally
- Item constructor/Items.getNewItemByType()/Items.add() no longer take folderID and orderIndex as parameters
- Split getTreeRows() into Scholar.getCollections(parent) and Scholar.getItems(parent), which return root collections or all library items, respectively, if no parent given
- All references to folders in object/method/property names changed to collections
- New methods Collection.addItem(itemID), Collection.hasItem(itemID), Collection.removeItem(itemID)
- Collection.erase() takes optional deleteItems parameter to delete items from DB -- otherwise just removes association and leaves item in library (does, however, delete all descendent collections from DB regardless)
* Note: This will break displaying of items until interface code is updated. *
Added Scholar.CreatorTypes with methods getTypes() (multi-dim array with 'id' and 'name') and getTypeName(creatorTypeID)
Fixed bug in Scholar.Creators.purge() causing SQL error when deleting a non-existent creator
Fixed incorrect field order for itemCreators INSERT queries in save()
Changed setCreator() to take empty creator names (causing DELETE in itemCreators but without shifting up creators below it (or down and above, depending on your perspective) like removeCreators())
Scholar.ItemTypes.getTypes() -- returns associative array of all itemTypes, with 'id' and 'name'
Scholar.ItemTypes.getTypeName(itemTypeID) -- returns pre-l10n type name (e.g. 'book', 'journalArticle')
Removed getLevel()
Updated Object.setPosition() to handle setting the position of new objects without committing, so it can be used within save(). May or may not actually work--my money's on no.
Added Folder.isEmpty()
No Folder.setPosition() yet, but, then, no editing or saving of folders at all--coming soon
Added getLevel() functions to Scholar.Object and Scholar.Folder -- save() and setPosition() not yet updated
Updated sample data to include levels
Added orderIndex (from treeOrder table, now) back into Scholar.Object data
New Scholar.Objects method for treeView: getTreeRows() -- returns mixed array of all Folder and Object objects in the proper order, type testable with instanceof (e.g. if (obj instanceof Scholar.Folder))
Scholar.Objects.get() and getAll() return arrays hashed by objectID again, since they're no longer used directly by treeView
Scholar.Objects.get() now returns the object directly if only one argument and scalar (i.e. one id)
Updated schema and sample data to handle folder ordering
Moved schema file out of chrome and removed XML tags, since it will no longer be accessed by XMLHTTP
Changed schema updater to use XPCOM components rather than XMLHTTP
Converted DB class to a singleton named Scholar_DB
Scholar_DB.query() now returns associative array similar to mysql_fetch_assoc() for SELECT statements rather than a mozIStorageStatementWrapper -- the mozStorage executeDataSet() function doesn't work yet, so we do it this way
Added DB functions:
- beginTransaction(), commitTransaction(), rollbackTransaction()
- columnQuery() -- one column, multiple rows, returned as array indexed by row
- getColumns() and getColumnHash
DB query functions can now handle bind parameters passed as an array of object literals: e.g. [{'int':2},{'string':'foobar'}]
valueQuery() now returns an int for COUNT(*) queries, so the result can be tested without a "0" being true
Changed _initializeSchema to drop existing tables before creating new ones, hacked with try/catch until DROP TABLE IF EXISTS works with the mozStorage extension (it's already in the latest SQLite release)
Added DB_REBUILD config flag to manually trigger schema regeneration
Added debug logging at level 5 for all SQL queries
Updated sample data
2006-03-14 11:34:17 +00:00
Renamed from chrome/chromeFiles/content/scholar/schema.xml (Browse further)
