From 56c7afc47ef6123fbbed25ac89fd18bacaed0e15 Mon Sep 17 00:00:00 2001 From: Dan Stillman Date: Fri, 22 Jul 2011 21:24:38 +0000 Subject: [PATCH] Duplicate detection: - Adds a per-library "Duplicate Items" virtual search to the source list -- shows up by default for "My Library" but can be added to and removed from all libraries - Current matching algorithm is very basic: finds exact title matches (after normalizing case/diacritics/punctuation/spacing) and DOI/ISBN matches (untested) - In duplicates view, sets are selected automatically; in other views, duplicate items can be selected manually and the merge interface can be brought up with "Merge Items" in the context menu - Can select a master item and individual fields to merge from other versions - Word processor integration code will automatically find mapped replacements and update documents with new item keys Possible future improvements: - Improved detection algorithms - UI tweaks - Currently if any items differ, all available versions will be shown as master item options, even if only one item is different; probably the earliest equivalent item should be shown for each distinct version - Caching of results for performance - Confidence scale - Creator version selection (currently the creators from the chosen master item are kept) - Merging of matching child items - Better sorting of duplicates if not clustered together by the selected sort column - Relation path compression when merging items that are already mapped to previously removed duplicates Other changes in this commit: - Don't show Trash in word processor integration windows - Consider items in trash to be missing in word processor documents - Selection of special views (Trash, Unfiled, Duplicates) is now restored properly in new windows - Disabled field transform context menu when item isn't editable - Left/right arrow now expands/collapses all selected items instead of just the last-selected row - Relation deletions are now synced - The same items row is now reselected after item deletion - (dev) Zotero.Item.getNotes(), Zotero.Item.getAttachments(), and Zotero.Item.getTags() now return empty arrays rather than FALSE if no matches -- tests on those return values in third-party code will need to be changed - (dev) New function Zotero.Utilities.removeDiacritics(str, lowercaseOnly) -- could be used to generate ASCII BibTeX keys - (dev) New 'tempTable' search condition can take a table to join against -- useful for implementing virtual source lists - (dev) Significant UI code cleanup - (dev) Moved all item pane content into itemPane.xul - Probably various other things Needless to say, this needs testing. --- .../content/zotero-platform/mac/overlay.css | 2 +- chrome/content/zotero/bindings/itembox.xml | 111 ++- .../content/zotero/bindings/tagselector.xml | 14 + chrome/content/zotero/duplicatesMerge.js | 156 +++++ chrome/content/zotero/itemPane.xul | 117 +++- chrome/content/zotero/selectItemsDialog.js | 2 +- .../zotero/xpcom/collectionTreeView.js | 375 +++++----- chrome/content/zotero/xpcom/data/item.js | 175 +++-- chrome/content/zotero/xpcom/data/items.js | 62 ++ chrome/content/zotero/xpcom/data/relation.js | 40 ++ chrome/content/zotero/xpcom/data/relations.js | 70 +- chrome/content/zotero/xpcom/duplicate.js | 87 --- chrome/content/zotero/xpcom/duplicates.js | 286 ++++++++ chrome/content/zotero/xpcom/integration.js | 44 +- chrome/content/zotero/xpcom/itemTreeView.js | 224 ++++-- chrome/content/zotero/xpcom/notifier.js | 6 +- chrome/content/zotero/xpcom/search.js | 17 +- chrome/content/zotero/xpcom/sync.js | 4 + chrome/content/zotero/xpcom/utilities.js | 120 +++- chrome/content/zotero/zoteroPane.js | 655 ++++++++++-------- chrome/content/zotero/zoteroPane.xul | 47 +- chrome/locale/en-US/zotero/zotero.dtd | 2 +- .../skin/default/zotero/bindings/itembox.css | 6 + chrome/skin/default/zotero/itemPane.css | 50 ++ chrome/skin/default/zotero/overlay.css | 29 - .../default/zotero/treesource-duplicates.png | Bin 0 -> 278 bytes .../zotero/treesource-search-virtual.png | Bin 317 -> 0 bytes components/zotero-service.js | 2 +- system.sql | 2 +- 29 files changed, 1856 insertions(+), 849 deletions(-) create mode 100644 chrome/content/zotero/duplicatesMerge.js delete mode 100644 chrome/content/zotero/xpcom/duplicate.js create mode 100644 chrome/content/zotero/xpcom/duplicates.js create mode 100644 chrome/skin/default/zotero/treesource-duplicates.png delete mode 100644 chrome/skin/default/zotero/treesource-search-virtual.png diff --git a/chrome/content/zotero-platform/mac/overlay.css b/chrome/content/zotero-platform/mac/overlay.css index 6ed923252b..a3fa35a425 100644 --- a/chrome/content/zotero-platform/mac/overlay.css +++ b/chrome/content/zotero-platform/mac/overlay.css @@ -111,7 +111,7 @@ background-color: #ffffff; } -#zotero-view-selected-label { +#zotero-item-pane-message { color: #7f7f7f; } diff --git a/chrome/content/zotero/bindings/itembox.xml b/chrome/content/zotero/bindings/itembox.xml index 927d76448f..cdf4c80601 100644 --- a/chrome/content/zotero/bindings/itembox.xml +++ b/chrome/content/zotero/bindings/itembox.xml @@ -79,7 +79,6 @@ break; case 'merge': - //this.hideEmptyFields = true; this.clickByItem = true; break; @@ -92,6 +91,11 @@ this.blurHandler = this.hideEditor; break; + case 'fieldmerge': + this.hideEmptyFields = true; + this._fieldAlternatives = {}; + break; + default: throw ("Invalid mode '" + val + "' in itembox.xml"); } @@ -103,15 +107,22 @@ - + + + .item must be a Zotero.Item"); + } + this._item = val; + this.refresh(); + ]]> + + onset="this.item = val; this.refresh();"> @@ -132,6 +143,22 @@ + + [] + + + .visibleFields'); + } + + this._hiddenFields = val; + ]]> + + + + {} + + + .fieldAlternatives'); + } + + if (this.mode != 'fieldmerge') { + throw ('fieldAlternatives is valid only in fieldmerge mode in .fieldAlternatives'); + } + + this._fieldAlternatives = val; + ]]> + + + + +