Patch Release with a Critical MIME Recursion DoS Fix, Sender-Identity Preservation for DMARC/SPF, Composite Lua Conditions, and the WebUI Tabulator Migration
re_conditions mechanism of mime expressions: rspamd_config:add_composite() accepts a full definition table with a conditions map from symbol name to function(task, symbol), where symbol arrives in the task:get_symbol() layout. A condition may return true/false or a number used as the atom weight; it is ANDed with the atom's option filters, and a failed condition is treated exactly like a missing symbol, so removal policies are not applied. Since a condition can consult symbols invisible to the expression itself, an optional depends_on list feeds the first/second composite pass placement as if those symbols were expression atoms — letting a condition safely consult, for example, a postfilter symbol (#6126)explicit_enable flag now genuinely require explicit enabling. Previously they were executed whenever any settings were applied to a task, regardless of content; now they run only when listed in symbols_enabled of the applied settings or force-enabled via task:enable_symbol(). To combine the default-allow behaviour with selectively unlocked symbols, the apply block gains a policy option: with policy = "implicit_allow", symbols_enabled no longer switches the settings into whitelist mode — all symbols stay enabled, the listed ones are merely unlocked, and symbols_disabled still subtracts. The policy is honoured for raw settings passed via the Settings header as well, and symbols listed in symbols_enabled are now always tracked as force-enabled, which previously missed the map-driven settings path (#6144)fasttext_embed. Words from rspamd's regular tokenization pipeline are re-tokenized into WordPiece subword tokens (ICU-based Bert normalization, Bert pre-tokenizer, greedy WordPiece) and embedded by mean-pooling rows of a precomputed float32 matrix, mmap-ed and shared between workers — no neural forward pass, no FFI, and no new dependencies. Model loading is strict and fail-fast: unsupported normalizer/pre-tokenizer/model types, non-mean pooling, a non-float32 matrix, or a vocab/matrix size mismatch fail the load instead of degrading silently. The subword tokenizer stays internal to the vectorizer, so Bayes tokens and fuzzy hashes are untouched, and a get_token_vectors() method exposes the per-token embedding sequence for offline consumers such as external trainers (#6118)train.forced_learn_minimal_scan (default on for symbol-independent rules) installs a high-priority prefilter that disables every non-neural symbol on an ANN-Train scan, so a training vector is built without issuing RBL/DNS lookups, fuzzy checks, Bayes classification, or ClickHouse work — with the stored vector and profile key byte-for-byte identical to the live full-scan path, and an automatic fallback to a full scan if any applicable rule needs symbols. train.frozen stops automatic training and auto-storing of live vectors, so a frozen model's training pools cannot drift, while inference keeps serving the current ANN; an explicit ANN-Train request still stores and retrains on demand. A supporting task:disable_all_symbols([skip_mask]) Lua binding is also addedrspamadm now discovers Lua sub-commands beyond the built-in directory — from the rspamadm.d drop-in directory under the configuration root (consistent with local.d), and from every directory listed in the colon-separated RSPAMADM_COMMAND_PATH environment variable. Duplicate command names are skipped with first-wins ordering, so built-in commands can never be shadowed, and externally loaded modules run with the same globals and Lua path as built-in ones. Third-party or premium packages can now ship rspamadm commands without writing into the rspamd-owned lualib treerspamadm operations non-interactively (#6114)rspamadm configdump --symbol-details now reports a flags array per symbol (fine, empty, explicit_disable, explicit_enable, ignore_passthrough, nostat, idempotent, mime, trivial, skip, composite, ghost, coro), making it possible to check — for instance — whether a symbol requires explicit enabling under the new settings policy; structural types remain in the type fieldtask:get_from({'mime', 'orig'}) previously returned both the preserved original addresses and the rewritten ones appended by task:set_from, making the flavour unusable for consumers that need the message as it arrived — a duplicate-From check would misfire on the combined list, and Lua could not tell the entries apart. Now, when originals were preserved by a rewrite, only the original addresses are returned; without a rewrite the behaviour is unchanged. The aliased and original email-address flags are exposed to Lua, and the flavour is reachable from selectors as from('mime', 'orig'). This rework underpins the DMARC/SPF sender-identity fixes below (#6141)OR, - to exclude, quoted exact phrases) that combines with the column filters, persisted column visibility, and theming mapped to Bootstrap CSS variables so light and dark modes flip automatically (#6120)task:set_from call is also fixed: a rewrite of any earlier address (such as an envelope recipient) previously caused the unmodified MIME From to be marked as aliased and duplicated (#6137)p=none while googlemail.com publishes p=quarantine), so substituting one for the other broke DKIM alignment, changed the effective DMARC policy, and misattributed DMARC reports. The gmail-style user-part canonicalization (dots and plus tags) is kept for googlemail.com addresses, but the domain is preserved; the equivalence remains available in lua_util.remove_email_aliases for dedup-only consumers such as email-hash RBL lookups (#6137)task:set_from (for example by the aliases module) no longer affect alignment, policy lookup, or rua report attribution (#6137);-separated recipient lists are pervasive in real mail and most MTAs split them. Previously such a list was parsed as a single malformed address whose local part swallowed the semicolon and the next address, breaking recipient counts and producing false invalid-address signals. ; is now treated exactly like ,, mirroring the comma behaviour in every edge case: semicolons inside quoted strings are preserved, undisclosed-recipients:; yields no addresses, and a group's trailing ; no longer leaks into the last member's domain (#6143, #6139)brand.co.uk over a brand.com link no longer fires PHISHING at full weight. Registrable labels are now compared instead of stripping only the last dot component (the old code handled only single-label suffixes, so brand.co.uk vs brand.com was Levenshtein-compared as full strings); the DMARC-authenticated-sender exclusion now normalises the From domain to its eSLD so it actually matches when DMARC reports a subdomain; and strict_domains is checked regardless of the computed weight, so displaying a strictly protected domain over a same-label, different-suffix target still firest.co entry matched walmart.com and *.bit.ly matched foo.bit.ly.evil.com. Translated patterns are now wrapped in ^(?:...)$ at map load time: bare names match exactly, wildcards match only what they say. This affects all glob maps — multimap glob/glob_multi, url_redirector redirector_hosts_map, dkim_signing/arc signing and key tables, mx_check exclusions, and the rbl glob returncodes matcher. Maps that relied on the accidental substring behaviour must now use explicit wildcards (#6125)from("smtp"):domain:lower was accepted and evaluated as from("smtp"):domain, and trailing garbage after a selector or a ; list element was ignored across every consumer, including the WebUI's selector checker. Parsing now requires the whole input to be consumed and reports the position and the unparsed tail otherwise. Selectors that previously loaded thanks to the silent truncation now fail configuration load with a pointed error; they were never evaluating as written, so failing loudly is the correct behaviour (#6127)time:digest — method syntax used where a transform was meant — aborted the whole selector at scan time with an uncaught "attempt to call a nil value" error. The method lookup is now guarded: the selector yields no value and logs an error that hints at the .name transform syntax when the name matches a known transform; valid method calls such as :lower are unaffectedlua_cryptobox secretbox encrypt and decrypt built a zero-padded 24-byte nonce but then passed the original short nonce pointer to libsodium, reading up to 23 bytes past the buffer and producing ciphertexts that depended on adjacent memory contents. The padded nonce is now passed, so a short nonce produces the same ciphertext as its explicitly zero-padded form, and the two forms decrypt each other (#6121)rspamd_kann.load({filename = ...}) always failed with "missing filename" because the options table was read from the wrong argument index (a copy-paste from save, where argument 1 is self), and save({filename = ...}) returned the filename string instead of true because a trailing pop removed the boolean result it had just pushed. Both forms now work, with a regression test covering the file round-tripcontext_augment function invoked its callback synchronously, the pending-fetches counter dropped to zero inside the callback and triggered the request; control then returned to the main flow, where a now-redundant guard fired a second request — this time without the context snippet. The duplicate path is removed (#6142)get_http_headers (#6117)for-loop control variables read-only, turning assignments to them into compile-time errors. The most visible casualty was the bundled lupa Jinja template engine, which failed to load with "attempt to assign to const variable 'word'" and made rspamd unable to start when configs use Jinja templating. All remaining offenders across the tree — found by parse-checking with luac 5.5 — now rebind the value to a local instead (#6130)Content-Type: message/rfc822 wrappers recursed through the MIME parser's message branch without ever tripping the max_nested (64) limit: unlike the multipart branch, the message branch never incremented the nesting counter actually carried into the recursion, so the entry guard never fired. Recursion was bounded only by message size — about 34 bytes per level — so a deeply nested message could exhaust the worker stack (a remote, unauthenticated denial of service via HTTP submission or SMTP/milter delivery), and the per-level boundary pre-scan made even non-crashing depths quadratic in CPU. The message branch now mirrors the multipart accounting, capping recursion at max_nested, which also caps the number of preprocess passes and eliminates the quadratic-CPU vector; a regression test feeds a 500-level chain and asserts the parsed part count stays capped. Reported by @gronkeThis patch release fixes a critical remote denial-of-service in the MIME parser via deeply nested `message/rfc822` messages and repairs a cluster of sender-authentication bugs where alias rewriting leaked into DMARC alignment, SPF evaluation, and forged-recipient checks — including the long-standing googlemail.com→gmail.com rewrite that silently changed the effective DMARC policy. It also anchors glob map patterns and selector parsing (both may require config fixes where the old lax behaviour was relied upon), adds per-symbol Lua conditions for composites, a settings apply policy for explicit-enable symbols, the static_embed neural provider, and completes the WebUI migration from FooTable to Tabulator. Recommended upgrade for all users due to the MIME parser fix.