2015-05-01 16:41:41 +00:00
|
|
|
-- 32
|
2006-09-10 20:08:59 +00:00
|
|
|
|
2009-12-28 09:47:49 +00:00
|
|
|
-- Copyright (c) 2009 Center for History and New Media
|
|
|
|
-- George Mason University, Fairfax, Virginia, USA
|
|
|
|
-- http://zotero.org
|
|
|
|
--
|
|
|
|
-- This file is part of Zotero.
|
|
|
|
--
|
|
|
|
-- Zotero is free software: you can redistribute it and/or modify
|
2011-05-18 18:34:22 +00:00
|
|
|
-- it under the terms of the GNU Affero General Public License as published by
|
2009-12-28 09:47:49 +00:00
|
|
|
-- the Free Software Foundation, either version 3 of the License, or
|
|
|
|
-- (at your option) any later version.
|
|
|
|
--
|
|
|
|
-- Zotero is distributed in the hope that it will be useful,
|
|
|
|
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
2011-05-18 18:34:22 +00:00
|
|
|
-- GNU Affero General Public License for more details.
|
2009-12-28 09:47:49 +00:00
|
|
|
--
|
2011-05-18 18:34:22 +00:00
|
|
|
-- You should have received a copy of the GNU Affero General Public License
|
2009-12-28 09:47:49 +00:00
|
|
|
-- along with Zotero. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
|
|
|
|
2006-09-10 20:08:59 +00:00
|
|
|
-- This file creates system tables that can be safely wiped and reinitialized
|
|
|
|
-- at any time, as long as existing ids are preserved.
|
|
|
|
|
2008-02-07 09:45:35 +00:00
|
|
|
-- Describes various types of fields and their format restrictions,
|
|
|
|
-- and indicates whether data should be stored as strings or integers
|
|
|
|
--
|
|
|
|
-- unused
|
|
|
|
DROP TABLE IF EXISTS fieldFormats;
|
|
|
|
CREATE TABLE fieldFormats (
|
|
|
|
fieldFormatID INTEGER PRIMARY KEY,
|
|
|
|
regex TEXT,
|
|
|
|
isInteger INT
|
|
|
|
);
|
|
|
|
|
2015-06-12 06:20:07 +00:00
|
|
|
DROP TABLE IF EXISTS charsets;
|
|
|
|
CREATE TABLE charsets (
|
|
|
|
charsetID INTEGER PRIMARY KEY,
|
|
|
|
charset TEXT UNIQUE
|
|
|
|
);
|
|
|
|
CREATE INDEX charsets_charset ON charsets(charset);
|
|
|
|
|
2008-02-07 09:45:35 +00:00
|
|
|
DROP TABLE IF EXISTS fileTypes;
|
|
|
|
CREATE TABLE fileTypes (
|
|
|
|
fileTypeID INTEGER PRIMARY KEY,
|
|
|
|
fileType TEXT UNIQUE
|
|
|
|
);
|
|
|
|
CREATE INDEX fileTypes_fileType ON fileTypes(fileType);
|
|
|
|
|
|
|
|
DROP TABLE IF EXISTS fileTypeMimeTypes;
|
|
|
|
CREATE TABLE fileTypeMimeTypes (
|
|
|
|
fileTypeID INT,
|
|
|
|
mimeType TEXT,
|
|
|
|
PRIMARY KEY (fileTypeID, mimeType),
|
|
|
|
FOREIGN KEY (fileTypeID) REFERENCES fileTypes(fileTypeID)
|
|
|
|
);
|
|
|
|
CREATE INDEX fileTypeMimeTypes_mimeType ON fileTypeMimeTypes(mimeType);
|
|
|
|
|
Initial Zotero 1.5 Megacommit
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
2008-05-04 08:32:48 +00:00
|
|
|
DROP TABLE IF EXISTS syncObjectTypes;
|
|
|
|
CREATE TABLE syncObjectTypes (
|
|
|
|
syncObjectTypeID INTEGER PRIMARY KEY,
|
|
|
|
name TEXT
|
|
|
|
);
|
|
|
|
CREATE INDEX syncObjectTypes_name ON syncObjectTypes(name);
|
|
|
|
|
2008-02-07 09:45:35 +00:00
|
|
|
-- unused
|
|
|
|
INSERT INTO "fieldFormats" VALUES(1, '.*', 0);
|
|
|
|
INSERT INTO "fieldFormats" VALUES(2, '[0-9]*', 1);
|
|
|
|
INSERT INTO "fieldFormats" VALUES(3, '[0-9]{4}', 1);
|
2007-10-23 07:11:59 +00:00
|
|
|
|
2022-11-04 05:21:27 +00:00
|
|
|
INSERT INTO "charsets" VALUES (1, 'utf-8');
|
|
|
|
INSERT INTO "charsets" VALUES (2, 'big5');
|
|
|
|
INSERT INTO "charsets" VALUES (3, 'euc-jp');
|
|
|
|
INSERT INTO "charsets" VALUES (4, 'euc-kr');
|
|
|
|
INSERT INTO "charsets" VALUES (5, 'gb18030');
|
|
|
|
INSERT INTO "charsets" VALUES (6, 'gbk');
|
|
|
|
INSERT INTO "charsets" VALUES (7, 'ibm866');
|
|
|
|
INSERT INTO "charsets" VALUES (8, 'iso-2022-jp');
|
|
|
|
INSERT INTO "charsets" VALUES (9, 'iso-8859-2');
|
|
|
|
INSERT INTO "charsets" VALUES (10, 'iso-8859-3');
|
|
|
|
INSERT INTO "charsets" VALUES (11, 'iso-8859-4');
|
|
|
|
INSERT INTO "charsets" VALUES (12, 'iso-8859-5');
|
|
|
|
INSERT INTO "charsets" VALUES (13, 'iso-8859-6');
|
|
|
|
INSERT INTO "charsets" VALUES (14, 'iso-8859-7');
|
|
|
|
INSERT INTO "charsets" VALUES (15, 'iso-8859-8');
|
|
|
|
INSERT INTO "charsets" VALUES (16, 'iso-8859-8-i');
|
|
|
|
INSERT INTO "charsets" VALUES (17, 'iso-8859-10');
|
|
|
|
INSERT INTO "charsets" VALUES (18, 'iso-8859-13');
|
|
|
|
INSERT INTO "charsets" VALUES (19, 'iso-8859-14');
|
|
|
|
INSERT INTO "charsets" VALUES (20, 'iso-8859-15');
|
|
|
|
INSERT INTO "charsets" VALUES (21, 'iso-8859-16');
|
|
|
|
INSERT INTO "charsets" VALUES (22, 'koi8-r');
|
|
|
|
INSERT INTO "charsets" VALUES (23, 'koi8-u');
|
|
|
|
INSERT INTO "charsets" VALUES (24, 'macintosh');
|
|
|
|
INSERT INTO "charsets" VALUES (25, 'replacement');
|
|
|
|
INSERT INTO "charsets" VALUES (26, 'shift_jis');
|
|
|
|
INSERT INTO "charsets" VALUES (27, 'utf-16be');
|
|
|
|
INSERT INTO "charsets" VALUES (28, 'utf-16le');
|
|
|
|
INSERT INTO "charsets" VALUES (29, 'windows-874');
|
|
|
|
INSERT INTO "charsets" VALUES (30, 'windows-1250');
|
|
|
|
INSERT INTO "charsets" VALUES (31, 'windows-1251');
|
|
|
|
INSERT INTO "charsets" VALUES (32, 'windows-1252');
|
|
|
|
INSERT INTO "charsets" VALUES (33, 'windows-1253');
|
|
|
|
INSERT INTO "charsets" VALUES (34, 'windows-1254');
|
|
|
|
INSERT INTO "charsets" VALUES (35, 'windows-1255');
|
|
|
|
INSERT INTO "charsets" VALUES (36, 'windows-1256');
|
|
|
|
INSERT INTO "charsets" VALUES (37, 'windows-1257');
|
|
|
|
INSERT INTO "charsets" VALUES (38, 'windows-1258');
|
|
|
|
INSERT INTO "charsets" VALUES (39, 'x-mac-cyrillic');
|
|
|
|
INSERT INTO "charsets" VALUES (40, 'x-user-defined');
|
2006-08-30 16:02:21 +00:00
|
|
|
|
2008-02-07 09:45:35 +00:00
|
|
|
INSERT INTO "fileTypes" VALUES(1, 'webpage');
|
|
|
|
INSERT INTO "fileTypes" VALUES(2, 'image');
|
|
|
|
INSERT INTO "fileTypes" VALUES(3, 'pdf');
|
|
|
|
INSERT INTO "fileTypes" VALUES(4, 'audio');
|
|
|
|
INSERT INTO "fileTypes" VALUES(5, 'video');
|
|
|
|
INSERT INTO "fileTypes" VALUES(6, 'document');
|
|
|
|
INSERT INTO "fileTypes" VALUES(7, 'presentation');
|
2023-08-24 07:06:33 +00:00
|
|
|
INSERT INTO "fileTypes" VALUES(8, 'ebook');
|
2008-02-07 09:45:35 +00:00
|
|
|
|
|
|
|
-- webpage
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(1, 'text/html');
|
|
|
|
-- image
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(2, 'image/');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(2, 'application/vnd.oasis.opendocument.graphics');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(2, 'application/vnd.oasis.opendocument.image');
|
|
|
|
-- pdf
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(3, 'application/pdf');
|
|
|
|
-- audio
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(4, 'audio/');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(4, 'x-pn-realaudio');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(4, 'application/ogg');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(4, 'application/x-killustrator');
|
|
|
|
-- video
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(5, 'video/');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(5, 'application/x-shockwave-flash');
|
|
|
|
-- document
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'text/plain');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/rtf');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/msword');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'text/xml');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/postscript');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/wordperfect5.1');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/x-latex');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/x-tex');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/x-kword');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/x-kspread');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/x-kchart');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/vnd.oasis.opendocument.chart');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/vnd.oasis.opendocument.database');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/vnd.oasis.opendocument.formula');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/vnd.oasis.opendocument.spreadsheet');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(6, 'application/vnd.oasis.opendocument.text');
|
|
|
|
-- presentation
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(7, 'application/powerpoint');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(7, 'application/vnd.oasis.opendocument.presentation');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(7, 'application/x-kpresenter');
|
2017-07-30 04:04:33 +00:00
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(7, 'application/vnd.ms-powerpoint');
|
2023-08-24 07:06:33 +00:00
|
|
|
-- ebook
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(8, 'application/epub+zip');
|
|
|
|
INSERT INTO "fileTypeMIMETypes" VALUES(8, 'application/epub'); -- incorrect, for compatibility
|
2008-02-07 09:45:35 +00:00
|
|
|
|
Initial Zotero 1.5 Megacommit
Apologies for the massive (and, due to data_access.js splitting, difficult-to-follow) commit. Please note that external code that accesses the data layer may need to be tweaked for compatibility. Here's a comprehensive-as-possible changelog:
- Added server sync functionality (incomplete)
- Overhaul of data layer
- Split data_access.js into separate files (item.js, items.js, creator.js, etc.)
- Made creators and collections first-class objects, similar to items
- Constructors now take id as first parameter, e.g. new Zotero.Item(1234, 'book'), to allow explicit id setting and id changing
- Made various data layer operations (including attachment fields) require a save() rather than making direct DB changes
- Better handling of unsaved objects
- Item.setCreator() now takes creator objects instead of creator ids, and Item.save() will auto-save unsaved creators
- clone() now works on unsaved objects
- Newly created object instances are now disabled after save() to force refetch of globally accessible instance using Zotero.(Items|Creators|etc.).get()
- Added secondary lookup key to data objects
- Deprecated getID() and getItemType() methods in favor of .id and .itemTypeID properties
- toArray() deprecated in favor of serialize(), which has a somewhat modified format
- Added support for multiple creators with identical data -- currently unimplemented in interface and most of data layer
- Added Item.diff() for comparing item metadata
- Database changes
- Added SQLite triggers to enforce foreign key constraints
- Added Zotero.DB.transactionVacuum flag to run a VACUUM after a transaction
- Added Zotero.DB.transactionDate, .transactionDateTime, and transactionTimestamp to retrieve consistent timestamps for entire transaction
- Properly store 64-bit integers
- Set PRAGMA locking_mode=EXCLUSIVE on database
- Set SQLite page size to 4096 on new databases
- Set SQLite page cache to 8MB
- Do some database cleanup and integrity checking on migration from 1.0 branch
- Removed IF NOT EXISTS from userdata.sql CREATE statements -- userdata.sql is now processed only on DB initialization
- Removed itemNoteTitles table and moved titles into itemNotes
- Abstracted metadata edit box and note box into flexible XBL bindings with various modes, including read-only states
- Massive speed-up of item tree view
- Several fixes from 1.0 branch for Fx3 compatibility
- Added Notifier observer to log delete events for syncing
- Zotero.Utilities changes
- New methods getSQLDataType() and md5()
- Removed onError from Zotero.Utilities.HTTP.doGet()
- Don't display more than 1024 characters in doPost() debug output
- Don't display passwords in doPost() debug output
- Added Zotero.Notifier.untrigger() -- currently unused
- Added Zotero.reloadDataObjects() to reset all in-memory objects
- Added |chars| parameter to Zotero.randomString(len, chars)
- Added Zotero.Date.getUnixTimestamp() and Date.toUnixTimestamp(JSDate)
- Adjusted zotero-service.js to simplify file inclusion
Various things (such as tags) are temporarily broken.
2008-05-04 08:32:48 +00:00
|
|
|
INSERT INTO "syncObjectTypes" VALUES(1, 'collection');
|
|
|
|
INSERT INTO "syncObjectTypes" VALUES(2, 'creator');
|
|
|
|
INSERT INTO "syncObjectTypes" VALUES(3, 'item');
|
2008-06-02 09:15:43 +00:00
|
|
|
INSERT INTO "syncObjectTypes" VALUES(4, 'search');
|
2008-06-16 05:46:10 +00:00
|
|
|
INSERT INTO "syncObjectTypes" VALUES(5, 'tag');
|
Duplicate detection:
- Adds a per-library "Duplicate Items" virtual search to the source list -- shows up by default for "My Library" but can be added to and removed from all libraries
- Current matching algorithm is very basic: finds exact title matches (after normalizing case/diacritics/punctuation/spacing) and DOI/ISBN matches (untested)
- In duplicates view, sets are selected automatically; in other views, duplicate items can be selected manually and the merge interface can be brought up with "Merge Items" in the context menu
- Can select a master item and individual fields to merge from other versions
- Word processor integration code will automatically find mapped replacements and update documents with new item keys
Possible future improvements:
- Improved detection algorithms
- UI tweaks
- Currently if any items differ, all available versions will be shown as master item options, even if only one item is different; probably the earliest equivalent item should be shown for each distinct version
- Caching of results for performance
- Confidence scale
- Creator version selection (currently the creators from the chosen master item are kept)
- Merging of matching child items
- Better sorting of duplicates if not clustered together by the selected sort column
- Relation path compression when merging items that are already mapped to previously removed duplicates
Other changes in this commit:
- Don't show Trash in word processor integration windows
- Consider items in trash to be missing in word processor documents
- Selection of special views (Trash, Unfiled, Duplicates) is now restored properly in new windows
- Disabled field transform context menu when item isn't editable
- Left/right arrow now expands/collapses all selected items instead of just the last-selected row
- Relation deletions are now synced
- The same items row is now reselected after item deletion
- (dev) Zotero.Item.getNotes(), Zotero.Item.getAttachments(), and Zotero.Item.getTags() now return empty arrays rather than FALSE if no matches -- tests on those return values in third-party code will need to be changed
- (dev) New function Zotero.Utilities.removeDiacritics(str, lowercaseOnly) -- could be used to generate ASCII BibTeX keys
- (dev) New 'tempTable' search condition can take a table to join against -- useful for implementing virtual source lists
- (dev) Significant UI code cleanup
- (dev) Moved all item pane content into itemPane.xul
- Probably various other things
Needless to say, this needs testing.
2011-07-22 21:24:38 +00:00
|
|
|
INSERT INTO "syncObjectTypes" VALUES(6, 'relation');
|
2013-03-03 08:38:02 +00:00
|
|
|
INSERT INTO "syncObjectTypes" VALUES(7, 'setting');
|