BODY/TEXT searches previously matched only bodies that happened to be
decoded in memory, so results were inconsistent. Add a persistent FTS5
index (a virtual table inside the SQLCipher store, encrypted at rest) over
the plain-text body of every message we download.
- store.rs: mail_fts(element_id UNINDEXED, body) with unicode61 +
remove_diacritics; index_body / unindex_body / search_body / fts_count.
Terms become prefix tokens ANDed together (factur -> factur*), built by
fts_match_expr which strips everything but alphanumerics so it is
injection-safe.
- rfc2822.rs: strip_html (drops tags + script/style + entities) and
extract_body_text (decodes the text part of our own .eml) feed the index.
- sync.rs: index inline at prefetch; one-time backfill at boot
(body_fts_indexed_v1) for bodies cached before the index existed;
unindex on delete.
- search.rs: BODY/TEXT resolve through the index — the session collects the
distinct body terms, queries the index once each, and passes the hit sets
to matches() via a SearchContext. A body term only matches messages whose
body has actually been downloaded (full coverage needs sync_limit = 0).
- LocalStore threaded into ImapSession (Option; None in unit tests).
Validated live: backfilled 7,687 cached bodies, then BODY/TEXT/AND/OR/NOT
queries returned coherent subsets — NOT BODY x == total - (BODY x), an
exact complement. 233 unit tests, incl. real FTS5 MATCH against the bundled
SQLCipher (confirms FTS5 is compiled in for the cross-OS release).
The local store was capped at sync_limit, so IMAP only ever listed the
newest N messages — a search in Thunderbird (the only search UI we have)
silently missed everything older. Now the syncer lists the *full* mailbox
metadata for every folder, and sync_limit governs only how many recent
message bodies are pre-warmed offline. Bodies outside that window are
fetched on demand the first time a client opens the message.
A one-time full-metadata sync (marker full_metadata_synced_v1) completes
the mailbox view on first launch after upgrade.
Crucially, an empty body is now stored as rfc2822 = None rather than a
rendered "(No body available)" placeholder: the placeholder looked like a
real body to the IMAP layer and suppressed the on-demand fetch. CachedMail
gains body_loaded to track whether the body (not just the headers) is final.
Validated live on a 19,322-message INBOX (26,965 mails total across
folders): full listing, on-demand body fetch (~0.1-0.4s), in-memory cache
on re-fetch.
* Add complete mailbox backup to .eml files (CLI)
`tutabridge backup <dir>` exports every mail of every folder to a
plaintext `.eml` tree, mirroring the IMAP folder hierarchy.
A backup must be *complete*: it enumerates all mails per folder from
the server (`limit == 0`), not just the `sync_limit`-capped subset the
bridge keeps cached. Dumping only the synced subset would silently drop
mail — live-tested here against an INBOX with 6288 server-side mails vs
1050 cached, all 6288 exported. The encrypted local cache
(`.eml.enc`) is used as a fast path; only never-synced mails trigger a
rate-limited (150ms) server fetch.
Format: one `.eml` per mail in `<output>/<folder path>/<YYYYMMDD-HHMMSS>_<id>.eml`.
EML is the most portable target — native to Thunderbird/Apple Mail/
Outlook, no Maildir `:2,S` colons that break on Windows, and a single
corrupt file never takes down the whole archive. Folder path segments
are sanitised for cross-platform filesystems (Windows-illegal chars +
trailing dot/space stripped); the date prefix makes a directory listing
sort chronologically.
`backup::export_eml` is surface-agnostic (takes a progress callback) so
a GUI button can wrap the same engine later. Per-mail failures are
collected in `BackupStats::errors` rather than aborting the run. The CLI
shares the keychain/password login flow with the bridge via the new
`login_session` helper, and opens the cache without the bridge's
reset-on-key-mismatch (a backup must never destroy the cache).
8 backup unit/integration tests: filename + folder sanitisation,
date stamp, and an end-to-end export over a mock backend asserting the
cache-vs-server split, file tree layout, and verbatim cached bodies.
GUI button is a follow-up (needs the Tauri dialog plugin).
* Make backup resumable / incremental
Skip a mail when its `.eml` is already on disk, before any cache read
or server fetch. The filename is deterministic (stable receivedDate +
element id) and mail content is immutable, so an existing file is never
stale. This turns an interrupted backup into a resume (re-run continues
where it stopped) and a periodic re-backup into an incremental one
(only new mail is fetched — the expensive part). New
`BackupStats::skipped` counter, surfaced in the CLI summary.
Two tests: a re-run skips every already-exported mail (zero server
loads), and an incremental run fetches only the newly-arrived mail.
* Add Backup tab to the GUI
A "Backup" tab wraps the same `backup::export_eml` engine as the CLI:
a native folder picker (tauri-plugin-dialog), a live per-folder
progress bar driven by `bridge://backup-progress` events, and a result
summary (mails written, folders, MB, cache vs server vs skipped).
`BridgeHandle` now keeps the logged-in backend + local cache after
`start` and exposes them via `backend_and_store()`, so the
`export_mails` command reuses the live session instead of opening a
second one — and drops the handle lock before the (minutes-long)
export so status/stats stay responsive. The button is disabled unless
the bridge is running.
`BackupStats` is now `Serialize` so it can cross the Tauri boundary.
* Keep backup state across tab switches
The Backup tab is conditionally rendered, so switching away unmounted
`BackupPanel` mid-export — dropping its progress + result state and the
`bridge://backup-progress` listener while the Rust task kept running.
Coming back showed an idle panel even though the backup was still going.
Lift all backup state (busy / progress / result / error), the
`startBackup` action, and the progress listener into the always-mounted
`useBridge` hook. The listener is now active regardless of which tab is
shown, and `BackupPanel` is purely presentational — switch tabs freely
mid-backup and the progress is intact on return. `startBackup` guards
against a double launch while one is in flight.
The CLI (`src/main.rs`) only spawned the syncer + IMAP + SMTP servers
and never started an `EventBusClient` — so the realtime push the
GUI's `BridgeHandle` ships had no effect when running `cargo run` or
the headless binary. Mails that arrived after a bootstrap sync were
silently missed until the next restart with a forced full re-sync;
that's the irritant that triggered today's WS heartbeat/timeout audit.
Replicate the bridge.rs initialisation directly: build an
`EventBusClient`, hydrate `last_batch_ids` from
`event_bus_state` (with the same 44-day expiration guard), spawn
`bus_client.run` alongside the syncer and an
`event_handler::run_event_handler` to consume the mpsc, and log
WsState transitions at INFO so reconnect storms are visible without
`RUST_LOG=debug`. Shutdown aborts the bus + handler handles in the
same Ctrl-C arm as the syncer.
To avoid duplicating the model-version + client-name plumbing, the
helpers `bridge::sys_model_version`, `bridge::tutanota_model_version`
and `bridge::CLIENT_NAME` are now public, and the root crate gains a
direct `tuta-sdk` dependency (already present transitively through
`tutabridge-core`).
Split the bridge into a tutabridge-core crate, a Tauri v2 desktop app
(src-tauri) and a React/TS UI (ui), keeping the CLI entrypoint at the
workspace root.
Add encrypted local storage (SQLCipher metadata index + encrypted .eml
files) so mail persists across launches and only the delta is fetched.
Wire the bridge to the Tuta Rust SDK via the tuta-repo submodule
(batch loading, MailDetailsBlob reading, interactive 2FA login).
Implement SMTP sending: build the draft and send it through Tuta's
DraftService/SendDraftService, mirroring the web client (body in
compressedBodyText, non-empty sender/recipient names, populated
SendDraftParameters). Add unit tests for the draft/send payload building.
- Add tuta-repo as git submodule pointing to spartanz51/tutanota
branch feat/rust-sdk-blob-read (pending upstream PR)
- Remove inline load_mail_details_blob hack, call through
mail_facade().load_mail_details_blob() instead
- Remove /tuta-repo from .gitignore since it's now a submodule
Local bridge that exposes Tuta encrypted email via standard
IMAP/SMTP protocols for use with Thunderbird and other clients.
Features:
- IMAP server with TLS (STARTTLS self-signed cert)
- SMTP server for sending mail via Tuta
- Session persistence via macOS Keychain
- Mail body decryption including LZ4-compressed blobs
- Blob storage access (BlobAccessTokenService + blob server)
- RFC 2822 message formatting
- Interactive first-run configuration
Anthony