Commit graph

3581 commits

Author SHA1 Message Date
Daniel Gasienica
e329c23aa0
Support Attachments On File System During Export + Import (#2212)
- [x] Load attachments into memory before export.
- [x] Save attachments to correct place in user profile upon import.
- [x] Upgrade message schema upon import.
- [x] Support both encrypted imports as well as unencrypted Chrome imports.

**Tests**

- [x] Export / import small database (25 messages) with some attachments already
      on disk (`schemaVersion >= 3`) and some not (`schemaVersion < 3`).
- [x] Import Chrome web app import (found and fixed issue with missing
      `schemaVersion`).
- [x] Export / import of real-world Signal Beta data (before: `IndexedDB`
      ~372MB / after: `IndexedDB` ~5.7MB / `attachments.noindex` ~234MB)
 
**Test commands:**

```javascript
// Export
let key = new Uint8Array([1, 3, 4, 5, 6, 7, 8, 11, 23, 34, 1, 34, 3, 5, 45, 45, 1, 3, 4, 5, 6, 7, 8, 11, 23, 34, 1, 34, 3, 5, 45, 45]);
Signal.Backup.exportToDirectory('/Users/<user>/Desktop/Signal-import-export-test', {key});

// Import
let key = new Uint8Array([1, 3, 4, 5, 6, 7, 8, 11, 23, 34, 1, 34, 3, 5, 45, 45, 1, 3, 4, 5, 6, 7, 8, 11, 23, 34, 1, 34, 3, 5, 45, 45]);
Signal.Backup.importFromDirectory('/Users/<user>/Desktop/Signal-import-export-test', {key});

// Import from Chrome
Signal.Backup.importFromDirectory('/Users/<user>/Desktop/Signal Export 2018 Apr 3 at 11.00.54 pm');
```
2018-04-04 19:25:35 -04:00
Daniel Gasienica
41ac9586eb Add start-backup script
This helps running a Signal Desktop instance from a backup (copy of profile
folder) when testing functionality such as database migrations, import/export,
etc.

Usage:
```
BACKUP=development-backup-2018-04-04 PROFILE=development ./scripts/start-backup
```

WARNING: This deletes the original profile and overwrites it with backup.
2018-04-04 19:11:46 -04:00
Daniel Gasienica
195e9b777c Rename willHaveAttachmentsSavedOnFileSystemDuringUpgrade
New name: `willAttachmentsGoToFileSystemOnUpgrade`
2018-04-04 19:11:46 -04:00
Daniel Gasienica
4c4443390a Rename createImporter to createAttachmentDataWriter 2018-04-04 19:11:46 -04:00
Daniel Gasienica
5ea1a305a6 Clarify error origin 2018-04-04 18:41:50 -04:00
Daniel Gasienica
9dcb64b53c Initialize schema version upon import 2018-04-04 18:41:50 -04:00
Daniel Gasienica
cfad983d08 Consistently use ciphertext instead of encrypted 2018-04-04 18:41:50 -04:00
Daniel Gasienica
a61c905146 Impliclity decrypt data if key is present 2018-04-04 18:41:50 -04:00
Daniel Gasienica
94c75762b5 Add Message.createAttachmentLoader 2018-04-04 18:41:50 -04:00
Daniel Gasienica
43ab320d31 Load attachments before writing them upon export 2018-04-04 18:41:50 -04:00
Daniel Gasienica
7eeabb5b1a Add precondition for writeAttachment 2018-04-04 18:41:50 -04:00
Daniel Gasienica
0faaf52b71 Import and upgrade messages upon save 2018-04-04 18:41:50 -04:00
Daniel Gasienica
cf7355bd2f Rename backupToDirectory to exportToDirectory
For symmetry reasons.
2018-04-04 18:41:50 -04:00
Daniel Gasienica
bf67254cc5 Add Message.createImporter 2018-04-04 18:41:50 -04:00
Daniel Gasienica
5a6668e677 Add Attachments.createWriteForExisting
This function lets us choose where to write attachment to instead of picking
random name.
2018-04-04 18:41:50 -04:00
Daniel Gasienica
efd673083d Rename writeAttachmentData to writeNewAttachmentData 2018-04-04 18:41:50 -04:00
Daniel Gasienica
36771a2dab Prefer double quotes over backticks for identifiers 2018-04-04 18:41:50 -04:00
Daniel Gasienica
18dddfe436
Request Configuration Sync After Relink (#2218)
Users lose their read receipt setting after relinking their app.
See: https://github.com/signalapp/Signal-Android/issues/7535

- [x] Request configuration sync after relink, not just on first run.
- [x] Minor: Add `is` dependency as discussed.
- [x] Unit tests!! Started a pattern where functions return their status that
      can be logged instead of logging themselves. Makes for great tests and
      might also help us debug user logs as the status will always be logged
      (with `reason` field) whether it was done or not. Otherwise, we have to
      look for the presence or absence of a log line. Of course, we have to be
      careful about log noise, but I thought this was a nice fit here.

**Manual Tests**

- Started app and set read receipts on iOS.
- Sent message from another device with read receipts enabled.
- Confirmed messages were marked read on both sides.
- Unlinked primary device.
- Relinked primary device.
- Sent messages between primary and other device.
- Confirmed read receipts are still shown.
- Confirmed logs show configuration sync.
2018-04-04 18:40:30 -04:00
Daniel Gasienica
bcbd526b99 Sync read receipt configuration upon relink
Fixes https://github.com/signalapp/Signal-Android/issues/7535.
2018-04-04 18:12:25 -04:00
Daniel Gasienica
ce8fe0d345 Add Signal.Startup module 2018-04-04 18:10:52 -04:00
Daniel Gasienica
29832c445a Add @sindresorhus/is dependency
This will allow us to do runtime and, once we have TypeScript support,
compile-time type checks.
2018-04-04 17:40:19 -04:00
Daniel Gasienica
a76a4865f5 Fix lack of focus when reselecting the same conversation (#2032)
Closes #2032
Fixes #2023
2018-04-04 15:47:33 -04:00
colefranz
234411cb29 issue-2023: introduce unit tests for inbox view
There were no unit tests for the file at all so I added some simple ones
mostly focused on my changes.
2018-04-04 15:42:13 -04:00
colefranz
fe0306e6be issue-2023: move opened event out of selected conversation check
Going through the git history the existing logic of "dont do this if
it's already selected" was just for audio or video QOL enhancements to
not stop playing when the same conversation is selected.
2018-04-04 15:42:13 -04:00
Daniel Gasienica
3ae17528d3
Migrate attachments in background without index (#2208)
- [x] Implement batch migration of attachments without index, i.e. use default
      primary key index on `Message::id`.
- [x] Run attachment in background without index.
- [x] Prepare module for whole database migrations in the future. Once we enable
      that, we have to force (remaining) attachment migration upon startup.
- [x] Run migrations explicitly on startup and remove implicit migrations from
      Backbone models using a placeholder that throws an error.
- [x] `Signal.Debug`: Add support for generating real-world data for
      benchmarking based on contents in `fixtures` folder. Add additional files
      to create a larger variety of test cases, e.g. JPEG, PNG, GIF, MP4, TXT,
      etc. **Test command:**
      ```
      Signal.Debug.createConversation({
        ConversationController,
        WhisperMessage: Whisper.Message,
        numMessages: 100,
      });
      ```
- [x] Minor: Improve error message for `storage.fetch` failures.
- [x] Minor: Use ISO-8601 timestamp for key rotation (helped me debug an issue).
- [x] Update tests to explicitly run migrations.
2018-04-03 17:13:48 -04:00
Daniel Gasienica
8474f3cf7f Use create* prefix to clarify curried functions 2018-04-03 15:25:24 -04:00
Daniel Gasienica
6e6b93d917 Run ESLint on preload.js 2018-04-03 15:03:57 -04:00
Daniel Gasienica
12cdeee7ec Rename count to numMessagesPerBatch 2018-04-03 14:43:17 -04:00
Daniel Gasienica
38ac9972e8 Improve termination condition 2018-04-03 13:16:05 -04:00
Daniel Gasienica
5b1a0c45df Replace getAll with cursor
This reduces the likelihood of IPC message size error.
2018-04-03 13:16:05 -04:00
Daniel Gasienica
ed77378293 Fix bug in termination condition 2018-04-03 12:30:13 -04:00
Daniel Gasienica
9c7afab21b Clear database before message view tests 2018-04-03 11:45:11 -04:00
Daniel Gasienica
9065b647ad Run migrations before clearing database during tests 2018-04-03 11:45:11 -04:00
Daniel Gasienica
01480f5dd2 Improve formatting 2018-04-03 11:45:11 -04:00
Daniel Gasienica
7d0871ee0e Run tests using yarn instead of npm 2018-04-03 11:45:11 -04:00
Daniel Gasienica
052fb841f6 Allow database name override in migrations 2018-04-03 11:45:10 -04:00
Daniel Gasienica
1e04083813 Support database upgrades in Database.open 2018-04-03 11:44:50 -04:00
Daniel Gasienica
f50ac1c99e Reduce excessive logging about database status 2018-04-03 11:06:56 -04:00
Daniel Gasienica
77b1f3c505 Replace startup with attachment background migration
Iterates through `Message::id` index.
2018-04-02 20:07:07 -04:00
Daniel Gasienica
c853d1bccd Remove whole database migrations
We should enable this when we are willing to run attachment migration
on startup.
2018-04-02 19:41:30 -04:00
Daniel Gasienica
057762806e Add support for attachment background migration without index 2018-04-02 19:41:28 -04:00
Daniel Gasienica
32ac99b439 Prefer ISO date formatting
Previously, this was formatted using locale specific settings. This will allow
easier debugging by matching key rotation timestamps against log timestamps.
2018-04-02 18:57:58 -04:00
Daniel Gasienica
2e4893f4c1 Clarify implications of introducing new migrations 2018-04-02 18:54:51 -04:00
Daniel Gasienica
7413b787b6 Make migrations private 2018-04-02 18:54:28 -04:00
Daniel Gasienica
b07c66eaa7 Generate attachments from local files
This ensures our benchmarking is closer to real-world usage, e.g. images, video,
etc. that are not compressible.
2018-04-02 17:58:37 -04:00
Daniel Gasienica
4a664bdab8 Add public domain files for generating test data 2018-04-02 17:42:05 -04:00
Daniel Gasienica
064a40544c Fix error message formatting 2018-04-02 17:42:04 -04:00
Daniel Gasienica
442f5d7295 Use done to match JavaScript iterators
See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators
2018-04-02 17:42:04 -04:00
Daniel Gasienica
a4f0c5242f Document why batch size is one 2018-04-02 17:42:04 -04:00
Daniel Gasienica
d59fdddb43 Improve storage.fetch error message 2018-04-02 17:42:04 -04:00