Commit graph

4158 commits

Author SHA1 Message Date
colefranz
fe0306e6be issue-2023: move opened event out of selected conversation check
Going through the git history the existing logic of "dont do this if
it's already selected" was just for audio or video QOL enhancements to
not stop playing when the same conversation is selected.
2018-04-04 15:42:13 -04:00
Daniel Gasienica
3ae17528d3
Migrate attachments in background without index (#2208)
- [x] Implement batch migration of attachments without index, i.e. use default
      primary key index on `Message::id`.
- [x] Run attachment in background without index.
- [x] Prepare module for whole database migrations in the future. Once we enable
      that, we have to force (remaining) attachment migration upon startup.
- [x] Run migrations explicitly on startup and remove implicit migrations from
      Backbone models using a placeholder that throws an error.
- [x] `Signal.Debug`: Add support for generating real-world data for
      benchmarking based on contents in `fixtures` folder. Add additional files
      to create a larger variety of test cases, e.g. JPEG, PNG, GIF, MP4, TXT,
      etc. **Test command:**
      ```
      Signal.Debug.createConversation({
        ConversationController,
        WhisperMessage: Whisper.Message,
        numMessages: 100,
      });
      ```
- [x] Minor: Improve error message for `storage.fetch` failures.
- [x] Minor: Use ISO-8601 timestamp for key rotation (helped me debug an issue).
- [x] Update tests to explicitly run migrations.
2018-04-03 17:13:48 -04:00
Daniel Gasienica
8474f3cf7f Use create* prefix to clarify curried functions 2018-04-03 15:25:24 -04:00
Daniel Gasienica
6e6b93d917 Run ESLint on preload.js 2018-04-03 15:03:57 -04:00
Daniel Gasienica
12cdeee7ec Rename count to numMessagesPerBatch 2018-04-03 14:43:17 -04:00
Daniel Gasienica
38ac9972e8 Improve termination condition 2018-04-03 13:16:05 -04:00
Daniel Gasienica
5b1a0c45df Replace getAll with cursor
This reduces the likelihood of IPC message size error.
2018-04-03 13:16:05 -04:00
Daniel Gasienica
ed77378293 Fix bug in termination condition 2018-04-03 12:30:13 -04:00
Daniel Gasienica
9c7afab21b Clear database before message view tests 2018-04-03 11:45:11 -04:00
Daniel Gasienica
9065b647ad Run migrations before clearing database during tests 2018-04-03 11:45:11 -04:00
Daniel Gasienica
01480f5dd2 Improve formatting 2018-04-03 11:45:11 -04:00
Daniel Gasienica
7d0871ee0e Run tests using yarn instead of npm 2018-04-03 11:45:11 -04:00
Daniel Gasienica
052fb841f6 Allow database name override in migrations 2018-04-03 11:45:10 -04:00
Daniel Gasienica
1e04083813 Support database upgrades in Database.open 2018-04-03 11:44:50 -04:00
Daniel Gasienica
f50ac1c99e Reduce excessive logging about database status 2018-04-03 11:06:56 -04:00
Daniel Gasienica
77b1f3c505 Replace startup with attachment background migration
Iterates through `Message::id` index.
2018-04-02 20:07:07 -04:00
Daniel Gasienica
c853d1bccd Remove whole database migrations
We should enable this when we are willing to run attachment migration
on startup.
2018-04-02 19:41:30 -04:00
Daniel Gasienica
057762806e Add support for attachment background migration without index 2018-04-02 19:41:28 -04:00
Daniel Gasienica
32ac99b439 Prefer ISO date formatting
Previously, this was formatted using locale specific settings. This will allow
easier debugging by matching key rotation timestamps against log timestamps.
2018-04-02 18:57:58 -04:00
Daniel Gasienica
2e4893f4c1 Clarify implications of introducing new migrations 2018-04-02 18:54:51 -04:00
Daniel Gasienica
7413b787b6 Make migrations private 2018-04-02 18:54:28 -04:00
Daniel Gasienica
b07c66eaa7 Generate attachments from local files
This ensures our benchmarking is closer to real-world usage, e.g. images, video,
etc. that are not compressible.
2018-04-02 17:58:37 -04:00
Daniel Gasienica
4a664bdab8 Add public domain files for generating test data 2018-04-02 17:42:05 -04:00
Daniel Gasienica
064a40544c Fix error message formatting 2018-04-02 17:42:04 -04:00
Daniel Gasienica
442f5d7295 Use done to match JavaScript iterators
See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators
2018-04-02 17:42:04 -04:00
Daniel Gasienica
a4f0c5242f Document why batch size is one 2018-04-02 17:42:04 -04:00
Daniel Gasienica
d59fdddb43 Improve storage.fetch error message 2018-04-02 17:42:04 -04:00
Daniel Gasienica
805031ade8 Conditionally run post-attachment migrations
Introduce placeholder migrations for Backbone models so they never implicitly
run migrations whenever they are `fetch`ed. We prefer to run our migrations
explicitly upon app startup and then let Backbone models be (slightly) dumb(er)
models, without inadvertently triggering migrations.
2018-04-02 17:42:04 -04:00
Daniel Gasienica
887bd83852
Migrate attachments on startup (#2193)
- [x] Run initial Backbone migrations 12–17 upon startup.
- [x] ~~Run attachment migration (without Backbone references to avoid
      migrations): `MessageDataMigrator.processAll`.~~ Disabled in favor of
      background processing without index.
- [x] ~~Run new migrations that can cover entire database (18+).~~
      Not until we have such migrations.
- [x] Add `runMigrations` module that ensures migrations are only run once.
- [x] Add `settings` (`Signal.Settings`) module to interact with our app
      settings (`items` store) using a domain API, e.g.
      `isAttachmentMigrationComplete(…)` vs
      `storage.get('attachmentMigration_isComplete')`. Required to check
      attachment migration status without using Backbone.
- [x] Add `database` (`Signal.Database`) CommonJS module to provide
      `Promise`-based interface to IndexedDB.
- [x] Add `debug` (`Signal.Debug`) module for generating synthetic data to test
      performance of migration.
- [x] Minor: Add `sleep` module for doing promise based sleeps.
- [x] Minor: Extract `wrapDeferred` as CommonJS module named `deferredToPromise`.
2018-04-02 17:33:32 -04:00
Daniel Gasienica
bfbeedab5c Temporarily disable post-attachment migration migrations 2018-04-02 15:26:24 -04:00
Daniel Gasienica
d9be6a0f94 Destructure Lodash requires 2018-04-02 15:12:04 -04:00
Daniel Gasienica
b7b6195cfc Extract IIFE into separate function 2018-04-02 12:17:36 -04:00
Daniel Gasienica
11f98474ba Capture how many messages we have to process 2018-04-02 12:17:36 -04:00
Daniel Gasienica
0fdc1140dd Add Database.getCount function 2018-04-02 12:17:36 -04:00
Daniel Gasienica
1f8556b049 Remove unused createCollection 2018-04-02 12:17:36 -04:00
Daniel Gasienica
c67c2a858a Remove Backbone references for attachment migration
It has to run without any other migrations interfering.
2018-04-02 12:17:36 -04:00
Daniel Gasienica
77f8f598de Add disk-usage.sh script for testing 2018-04-02 12:17:36 -04:00
Daniel Gasienica
0c40f35623 Document disadvantage of fetching messages without index 2018-04-02 12:17:36 -04:00
Daniel Gasienica
a4ecf1a9d6 Define constant after creating idle detector 2018-04-02 12:17:36 -04:00
Daniel Gasienica
d3c9de4712 Add debug module
Lets us generate large conversations with large attachments.
2018-04-02 12:17:36 -04:00
Daniel Gasienica
30037e5308 Reduce attachment migration batch size to 1
This prevents ‘Maximum IPC message size exceeded’ due to IDB `getAll` operation.

- https://github.com/zincbase/zincdb/issues/17
- https://cs.chromium.org/chromium/src/content/browser/indexed_db/indexed_db_database.cc?q=%22Maximum+IPC+message+size+exceeded%22&sq=package:chromium&l=1160
2018-04-02 12:17:36 -04:00
Daniel Gasienica
ce8fd3d847 Expose Signal.Settings module 2018-04-02 12:17:36 -04:00
Daniel Gasienica
02354ce655 Expose Signal.Database module 2018-04-02 12:17:36 -04:00
Daniel Gasienica
21147a20a0 Add sleep module 2018-04-02 12:17:36 -04:00
Daniel Gasienica
08f73b8420 Remove last processed index after attachment migration 2018-04-02 12:17:36 -04:00
Daniel Gasienica
696a144ab7 Add settings.deleteItem 2018-04-02 12:17:36 -04:00
Daniel Gasienica
a18e462817 Move migrations to Signal.Migrations 2018-04-02 12:17:36 -04:00
Daniel Gasienica
5910f84af4 Remove outdated documentation 2018-04-02 12:17:36 -04:00
Daniel Gasienica
efe3cd67fc Allow attachment migration run on higher database version 2018-04-02 12:17:36 -04:00
Daniel Gasienica
921c3dba7c Skip migrations that have already been applied 2018-04-02 12:17:36 -04:00