Notesnook v3.0.3

Notesnook v3.0.3 fixes 15+ bugs via 40 commits. Replaced force sync with separate force push & force pull buttons, added various fixes to real-time sync, improved editor reliability on mobile & fixed export of locked notes.

Previous releases:

• v3.0.2 fixes 10+ bugs via 24 commits. Attachment manager for logged out users, much improved attachment handling fixing a couple of critical issues, pre-v3 backups no longer throw errors, and many other fixes.
• v3.0.1 fixes 20+ bugs via 65+ commits. This release adds SQL based search, improves editor performance & add support for multiline search replace in editor.

Replaced force sync with force push/pull

The reason is simple: force sync is rarely useful on its own and, more often than not, it causes data loss. However, we can't just remove it because there is sometimes need for a "force" sync. I also think a lot of users have been misusing "force sync" as an alternative to auto sync because of sync issues that were common in v2. That is no longer the case which is why we are replacing "Force sync" with separate buttons for "Force pull" and "Force push".

In v2 (and v3), "Force sync" used to work as follows:

1. Get everything that's on the server
2. Send everything that's on the device

In most cases, one of the above steps is unnecessary because either you want to get everything from the server or send everything from your device to the server. In cases where you need both, it's always better to control the sequence of those 2 steps.

NOTE: As mentioned in the screenshot above, this should never be used except for troubleshooting purposes. For 99.99% of cases, auto sync should work and you should never face a scenario where "force pushing" or "force pulling" is needed.

More reliable mobile editor

We have recently been getting a couple of bug reports from (mostly) mobile users where their edits are not saved. This is a critical issue and in this release we have added various fail safes to prevent critical data loss.

In the mobile app, the editor and the app are basically 2 different processes and they communicate internally through an IPC. The editor runs inside a WebView and any edit you make gets sent to the app for storage and syncing. Sometimes, however, this IPC can fail or stop responding resulting in data loss. To prevent such issues, we have now added fail safes which will notify you if the communication with the app is broken, or if saving a note takes too long. You can then take extra measures such as copying your changes.

This is not a common occurence, by any means, and the app will try to automatically recover your edits (if possible).

Fixed File already exists error on duplicate downloads

Downloading the same file over and over again (i.e. when you have multiple copies of an image in a single note) will no longer fail with the File already exists error.

Fixed database disk image malformed error

This issue was partially fixed in v3.0.1.

By default, SQLite FTS5 indexes everything in the table and there's no way to exclude anything. To avoid the unnecessary indexing of locked notes (which are basically gibberish), and permanently deleted notes (which don't have anything worth indexing), we use SQLite triggers with some conditions. The problem occurs when SQLite tries to deindex something that wasn't indexed in the first place.

This can happen, for example, when syncing (or restoring a backup) that contains locked notes and then permanently deleting those notes from trash. Unfortunately, debugging this is next to impossible due to very little information exposed by SQLite.

This version also rebuilds the whole FTS5 index (so if you have a lot of notes it might take some time to start the app after upgrading). This is necessary because due to incorrect logic, previous versions were indexing blacklisted items that shouldn't be indexed. To properly fix the issue, it was necessary to reindex everything.

Fixed edge case during sync causing some items to be missed out

While the new sync engine handles everything without jumping through a whole lot of hoops, it isn't immune to some edge cases. The problem is simple: what would happen if an item gets updated right when it is being pushed to the server?

If you remember from the Introducing Notesnook v3 blog post, the sync engine basically operates on the value of synced. Now imagine the following scenario:

1. Item A is edited (it's synced property is set to false)
2. Sync engine pushes Item A to the server
3. Sync engine sets synced property of Item A to true

All well and good so far, but what if Item A gets updated between step 2 & 3? Oops. The change will never be pushed to the server because it's synced property will be set to true in step 3.

To circumvent this edge case, we now only toggle the synced property if the item wasn't changed while it was being pushed to the server. Thanks to SQLite, this only requires adding the following condition:

WHERE dateModified <= timestampBeforePushStarted

This fixes the issue but the item still isn't synced until the next sync is run (which can never happen if nothing else changes). That is why, right after the sync completes we check if there are any items that were modified during the current sync cycle, and if so, those items immediately get sent to the server. In this way, your edits are never missed.

Other changes

• Fix tests 919524489
• Fix image filename in picker 152794ccd
• Fix editor content update on realtime sync 2e8b234b6
• Add Audios tab in attachments dialog 055c64769
• Fix real time sync test 3c8b8a7bc
• Saving timeout notice shown every time fde45fb6d
• Fix tab not updating when note is locked/unlocked 4a616a3e6
• Fix dateEdited not updating on saving locked content bd0a338dd
• Fix async iterator on sql collection only returning dateCreated & id 5ab721e3e
• Fix errors cb70bb942
• Show error if saving takes more than 30 seconds 82cadd473
• Fix editor loading 0e8ea3dde
• Handle editor not responding 511acc5fc
• Add editor lock safety 001cf8ee9
• Fix color not updating on note (#5573) a965cd694
• Fix wording of push/pull sync d3c1c1959
• Replace force sync with force push/pull 9a866a552
• Push/pull force sync 837c29682
• Fix locked notes not getting exported 9aed6faed
• Fix crash due to undefined is not an object (evaluating 'a.nodeSize') (#5553) 734f491e8
• Fix editor crash due to position (#5519) 0ea13024e
• Close shared service on window close cb2703924
• Do not use auth access tokens as normal access tokens 714ac2dea
• Fix crash on toggling custom dns (#5538) d605bc29b
• Fix database disk image malformed error (#5525) 080101bc4
• Rerun sync if there are unsynced changes after sync complete 4819e17da
• Fix crash when taking camera picture 68a6d6c31
• Fix camera permission c51ef4d5f
• Remove margin in editor d9c5c1f89
• Fix editor scrolling on update 5a75ca73a
• Use yarn instead of npx for build scripts 626d39337
• Fix legacy backup restoration 68d06ff3f
• Fix File already exists error on duplicate downloads 79f20d100
• Handle case when updating item while a push is ongoing ac96354b4
• Set synced to false on removing content by note id 63f44d6fb
• Set synced to false on unlinking relations f9bfa88c8