Skip to main content
🐛 Bug Fix Release

Rspamd 4.2.3

Patch Release with ARC Trust Fixes, Browser-Accurate HTML Parsing, Merged RBL Queries and force_actions Priorities

🛡️ Security

  • ARC forwarders whose seal failed verification are no longer trusted: A seal from a domain in the ARC whitelist counts as valid even when its verification fails. This keeps a broken ARC implementation from failing the whole chain. However, the trusted signers check looked only at the chain result. Such a seal therefore still produced ARC_ALLOW_TRUSTED, and the ARC-Authentication-Results of its instance became the trusted results used by ARC_DMARC_ADJUSTMENT. Anyone can claim a whitelisted d=, so a seal that did not verify could lower DMARC. A failed seal authenticates neither its d= nor its ARC-Authentication-Results. Rspamd now records the verification result of each seal and trusts only seals that actually verified. The whitelist still keeps the chain valid, so ARC_ALLOW is unchanged
  • Exactly one ARC-Authentication-Results header per ARC instance: An extra ARC-Authentication-Results header with the same instance number is not covered by any seal. It could still be selected as the trusted forwarder results and lower a DMARC reject or quarantine through ARC_DMARC_ADJUSTMENT. Each instance must now carry exactly one such header (#6311)

🔄 Changed

  • phishtank.rspamd.com disabled by default: The zone no longer carries data. With the module defaults, however, every installation still hashed every URL into a query against it. These names never repeat, and they made up most of the query load on the Rspamd RBL nameservers. The phishing module no longer queries the zone by default. The zone has no data, so detection is not affected
  • RBL checks that produce the same DNS query are merged: Several checks of one rbl rule can produce the same name, for example urls, emails and dkim all yielding one domain. Such a name used to be queried once per check. Requests that resolve to the same query are now sent once. The comparison ignores case and a trailing dot, with two exceptions. For hashed values, the trailing dot is part of the hashed data. Base64 hash names and names returned by process_script keep their case, because it may carry data. Checks that share a query also share one symbol insertion, with every origin listed as an option, and the symbol's weight counts once. The same applies to the _FAIL symbol. This changes scoring: a rule that used to score once per check for the same value now scores once per matched value. To keep one insertion per check, set merge_checks = false on a rule or default_merge_checks = false globally. The global option also reaches rules with ignore_defaults, such as the stock URIBL and DBL rules. Related changes:
  • a value whitelisted by several RBL rules is tracked for all of them, not only for the last one processed, and the whitelist is checked against the name that is actually queried;
  • resolve_ip carries the origins of all merged sender-side checks to the second-phase query and to failure reports. An address dropped by process_script is skipped. Previously it failed the DNS callback and lost all second-phase queries;
  • requests are merged only within the same resolution phase, so a HELO of 4.3.2.1 no longer hides the direct listing of client 1.2.3.4;
  • empty lookup values are skipped. Previously they were queried against the bare RBL suffix (#6303)
  • dkim_domainonly under dkim_match_from is no longer inverted: With dkim_match_from, the eSLDs were compared when dkim_domainonly was off, which is the opposite of the intended behaviour. Rules that set dkim_domainonly explicitly now behave as documented. Rules converted from a legacy surbl section relied on the inversion to compare eSLDs. The converter now sets dkim_domainonly = true, so converted rules behave as before (#6330)

✅ Added

  • force_actions rules that depend on composites run as postfilters: Composites are evaluated in their own stage, after all normal filters. A dependency from a normal symbol on a composite therefore failed silently, and a rule whose expression referenced a composite never waited for it. force_actions now detects composite atoms and registers such rules as postfilters, as it already did for require_action and honor_action rules. This works in both the rules format and the legacy actions format. These rules run right after the composites, before greylisting and before the require_action/honor_action rules (#6304)
  • force_actions priorities: The new priority rule setting (low, normal, high, critical, or 0 to 3) decides which result wins when several rules match. Invalid values are reported, and out-of-range values are clamped. Combining a priority with least produces a warning. If a rule has a priority, it is shown as a priority:N option next to the fired FORCE_ACTION_* symbol. Arbitration details:
  • A rule without a priority records nothing if the score has already selected its action, so the verdict stays score-based. A rule with its own priority records its result anyway, with process_all implied, so a later lower-priority rule cannot override it.
  • A main-stage rule with a lower priority than another non-least rule gets process_all implied. Its match therefore cannot suppress the atoms and composites the other rule needs.
  • require_action/honor_action rules run by priority and then by name, and they do not record an action that is already selected. A least rule without a priority still records at low, below the least results of other modules such as DMARC.
  • A higher-priority rule without a message keeps the SMTP reply of the result it takes over.
  • Rules are registered in a stable order sorted by name, in both formats. Previously the configuration's unspecified iteration order was used (#6304)
  • task:has_pre_result() returns the action name and flags: This function used to return the action type, which is custom for every custom action. It now returns the action name, as get_metric_action() does. A fifth return value holds the flags, in the format that set_pre_result() accepts. With a true argument, it returns the pre-result that selects the action and the reply. The first pre-result in the list may be a least result and is not necessarily that one. The returned score is unchanged
  • unix_time in metadata_exporter: The json formatter carried no event time. Consumers of buffered backends could therefore only timestamp an event when they read it, for example with redis_list, redis_stream, or json_raw_tcp feeding a log pipeline. After a backlog, every event got the time it was drained. The general metadata now includes unix_time: the task connect time in GMT, with fractional seconds. ClickHouse stores the same value, and history_redis uses the same name (#6326)

🔧 Fixed

  • ARC results wait for every seal check: ARC results are published only after all seal checks of the chain have completed (#6311)
  • DKIM key requests that cannot be scheduled are reported as failures: dkim.verify returned success even when the DNS request for the key could not be scheduled, for example because of invalid characters in d= or s=. No callback ever fired, so an ARC chain containing such a seal published no result at all. This case is now an immediate failure, and the seal counts as failed. An immediate ARC-Message-Signature failure now also goes through the signature callback. For whitelisted domains, seal verification therefore continues in the same way as after other failures (#6311)
  • Milter session use-after-free on connection close: A milter session could be accessed after it had been freed when its connection closed (#6327)
  • No crash on a regexp map that repeats an invalid regexp: The map helper added a new key to its hash before compiling the regexp. When the regexp did not compile, the key stayed in the hash without a value. A later line with the same key took the duplicate branch and logged the value read from the hash, which was an uninitialised pointer. The main process crashed while preloading such a map. Every worker crashed when the map changed at runtime, and again on each respawn. Even a single invalid regexp made the element count wrong. Any walk of the map also read the missing value, for example the WebUI map view, map:foreach or map:get_stats. Each invalid line is now reported and skipped (#6307)
  • task:get_stat_tokens() without a classifier: The statistics tokenizer is configured from the first classifier, so it stays unset when no classifier is defined. task:get_stat_tokens() dereferenced it anyway and crashed, for example when called by rspamadm mime stat -b. It now returns no tokens
  • Stale hyperscan caches are recompiled: Regexp maps and multipattern databases never recovered from a cached blob that the current library refuses to deserialize. This happens, for example, after switching between hyperscan and vectorscan or after a library upgrade. The compilation queue skipped recompilation as long as a cache entry existed. The stale blob therefore survived every restart, and the affected maps kept running on the PCRE fallback. A blob that fails to deserialize is now dropped and recompiled, as the regexp cache already did. Cached databases are loaded and validated before compilation is skipped. Incompatible blobs are replaced before workers are notified, so workers recover during the same startup (#6308)
  • HTTPS reads no longer stall on TLS records over 8 KiB: When a TLS record was larger than the 8 KiB buffer, OpenSSL held the remaining decrypted bytes, and no new socket event arrived to read them. The request stalled until the client timed out. This affected, for example, clients that send headers and body in one write to the controller or to a worker with TLS. The read handler now drains pending TLS data (#6314)
  • DNS UDP channels reconnect after the kernel drops them: The kernel can dissolve the association of a connected UDP socket with SOCK_DESTROY. ss -K does this, as do eBPF service load balancers such as Cilium when a backend goes away. After that, every send() failed with EDESTADDRREQ, and the channel did not recover until Rspamd restarted. The socket itself is still usable. librdns now marks the channel as disconnected and retries once with sendto(), which connects the channel again (#6298, #6297)
  • HTTP upstreams for hosts without a public suffix: On a configuration reload, the TLD list is already loaded when plugins are initialised. The URL parser therefore rejected endpoints such as llm.internal.lan, and http_upstreams_by_url returned nothing. As a result, the GPT plugin failed on every message with "attempt to index field 'upstreams'". On a cold start, the parser is initialised lazily without TLDs, so the problem appeared only after the first SIGHUP. Configured endpoints are now parsed without TLD restrictions, and IPv6 brackets are kept when upstreams are created. The fix applies to every user of http_upstreams_by_url: GPT, AWS S3, the DKIM vault integration and the HTTP maps backend. The GPT plugin also disables itself at configuration time if it cannot create any upstreams (#6323, #6322)
  • HTML declarations, comments and raw text end where browsers end them: Several tokenizer states kept going past the point where browsers stop. The rest of the part, including tags, links and text, was then hidden from the scan:
  • <![CDATA[, and any <![ with unbalanced brackets, ran until the brackets balanced. In HTML content, browsers treat both as a bogus comment that ends at the first >, and Rspamd now does the same. CDATA in HTML content also sets the new cdata_in_html property.
  • Comments ignored the --!> terminator.
  • Raw text ended at any plausible end tag, but not at its own end tag when that tag had a tab or long attributes. It now ends exactly at its own end tag. title, textarea, xmp, iframe, noembed, noframes and plaintext are raw text, as in browsers, so a comment inside them no longer swallows what follows. noscript is parsed as markup, because mail clients run no scripts.
  • When input ended inside a comment, a declaration or a partial tag, the text after the last child of the open elements was lost.
  • <!-x>, </ x>, </>, <!--> and <?> no longer leak into the text. An out-of-bounds read in the comment start state is also fixed (#6324, #6321)
  • SVG and MathML are tracked as browsers build them: Browsers ignore /> on HTML elements other than void elements, so <span style="display:none"/>text hides the text. Rspamd closed the span and treated the text as visible. The slash now closes only void elements and SVG/MathML elements, and <svg/> is closed again. Membership in SVG or MathML now follows browser rules:
  • HTML-only tags such as <p>, <span> or <font color>, and the end tags </p> and </br>, leave foreign content.
  • Integration points such as foreignObject, desc, title or mi hold HTML.
  • Raw text applies only to HTML elements. In SVG, <title>, <style> and <textarea> hold markup. Treating them as raw text hid everything up to an end tag that never came.
  • <object> is no longer head content, because mail clients run no plugins and show its fallback content (#6324, #6321)
  • SVG CDATA sections and stray slashes: A CDATA section in SVG or MathML now runs to ]]>, and its content is text, as in browsers. Ending it at the first > parsed its text as markup. In <svg><![CDATA[x><p><style>]]></svg><a ...>, the <p> left SVG and the style swallowed the link. A slash also closed a tag when an attribute or a space followed it, or when it ended an unquoted value. <svg/x>, <svg/ > and <svg width=10/> all closed the svg, so a following <style> became HTML raw text and hid the rest. Now only a / directly before > closes a tag, and in an unquoted value the slash is part of the value (#6324, #6321)
  • Character references kept in CDATA and raw text: Browsers decode character references in normal text, title and textarea. They do not decode them in CDATA sections or in raw text elements such as xmp or plaintext. Rspamd decoded entities everywhere, so &amp; in those places became &, which changed the visible text and the URLs found in it (#6324, #6321)
  • Text URLs end at quotes: A URL in text kept its closing double quote, so "http://x/" gave http://x/". In href="http://x/">, the > after the quote made the whole URL invalid, so the URL was lost. Schemeless URLs after a double quote, such as "www.example.com", were never found, because unlike ' the double quote was not a URL delimiter. " is now treated like ', and a URL opened by a quote ends at the closing quote. This also cleans up URLs in raw text elements such as <xmp>, which hold markup as text (#6324, #6321)
  • Bayes no longer counts the body twice for plain and HTML alternatives: For most HTML mail, both the plain text alternative and the HTML alternative were tokenized, so Bayes counted the body twice. Exact comparison of the token input rarely works for real alternatives, for three reasons: URLs are written out only in the plain text, HTML blocks join words, and the flags of the last words often differ. The plain part is now skipped when parts_distance shows it is more than 80% similar to the HTML part. This uses the corrected distance described below. Only the pair the distance is computed for is checked, and only when the distance is exact, which requires at most 8192 words in each part. HTML that is the first part of a multipart/related child without a start parameter is also accepted. This covers the usual layout of mail with inline images, alternative{plain, related{html, images}}. Siblings that are not text/plain, such as the text/calendar part of invitations, are ignored. Both parts are kept when the alternative has another text/plain child (#6316)
  • Parts distance substitution costs: The edit distance between the text parts of a message used inverted substitution costs, so the distance was miscalculated. R_PARTS_DIFFER uses this distance, and so does the Bayes deduplication above (#6317)
  • Statistical words finalized in late-injected parts: Text parts injected after the initial MIME parse did not get their words finalized for statistics. They are now processed like any other part (#6315)
  • Whitespace before a header colon accepted: The obsolete syntax of RFC 5322 allows whitespace between a header name and its colon, as in Subject : text. Such headers are now accepted. Header anomaly scoring and headers-only parsing are unchanged (#6313)
  • RSPAMD_URIBL queries registered domains: uribl.rspamd.com lists registered domains (eSLDs). The rule, however, queried the full SMTP From, MIME From and Message-ID domains, and the full DKIM d=. Because the rule sets ignore_defaults, the default dkim_domainonly never applied, so mail.evil.com never matched a listing of evil.com. Numeric URL hosts were queried as hashes of their reversed octets, which the zone can never contain. They are now skipped (#6330)
  • rbl require_symbols and return_codes patterns work: The require_symbols predicate had no return, so it was always false, and any rule using require_symbols never ran. The return_codes and return_bits aliases were applied after the luapattern matcher had already been auto-detected from returncodes. As a result, patterns given as return_codes fell back to equality matching (#6330)
  • Consistent From and DKIM domain normalisation in rbl: The rbl plugin now normalises From domains and DKIM domains the same way (#6330)
  • Every element of a monitored ipnet list is loaded: When ipnet was given as a list, the whole array was passed instead of each element, so none of the networks were loaded
  • Lua file savers retry short writes: task:store_in_file(), rspamd_text:save_in_file() and the RSA and cryptobox signature savers did not retry short writes, and they reported success for truncated files. They now retry short and interrupted writes, and an incomplete file is removed. Other fixes to these functions:
  • The table form of store_in_file() created files with mode 0000 instead of the documented 0600, because a missing mode key was reset to 0. Files now get 0600 by default.
  • On a write error, save_in_file() closed a descriptor passed in by the caller. The caller's own close then hit a closed and possibly reused descriptor, for example in the antivirus scanners' destructors. It now closes only the files it created itself, and on failure it unlinks only those files (#6318)
  • The table form of task:set_pre_result() defaults to normal priority: A missing priority key was written as 0. force_actions and spamtrap, which use the table form, therefore recorded their results at low instead of normal, unlike the positional form. Flags were matched as an else-if chain, so least,no_smtp_message set only least. All listed flags are now applied
  • rspamd_config:get_symbol_flags() reports composites: The function removed the composite flag before returning the flags to Lua, so Lua code could not tell that a symbol was a composite. The symcache now stores the flag on composite symbols, and the flag lists report it directly. Symbol details no longer list a composite twice, once as the type and once as a flag (#6304)
  • Symbols cache persistence and file descriptor ownership: Fixes how the persisted symbols cache is written and which code owns its file descriptor (#6331)
  • DNS resolver leaks on worker shutdown: Workers no longer leak their DNS resolver when they shut down (#6329)
  • ESMTP argument memory leaks: The containers that hold ESMTP arguments are now freed (#6328)

This patch release closes two ways to lower DMARC through ARC. Whitelisted forwarders whose seal failed verification are no longer trusted, and each ARC instance must now carry exactly one ARC-Authentication-Results header. The HTML tokenizer now ends comments, declarations, CDATA sections and raw text where browsers do, and it tracks SVG and MathML content. Content that browsers show can therefore no longer be hidden from the scan. force_actions gains composite-aware postfilters and rule priorities, and metadata_exporter now exports the event time. Other fixes cover stalled HTTPS reads, DNS channels dropped by the kernel, stale hyperscan caches, GPT failing after a reload, Bayes counting the body twice, and several crashes and memory leaks. Before upgrading, note three incompatible changes. First, rbl now merges checks that produce the same DNS query, so a rule scores once per matched value; set `merge_checks = false` to keep the old behaviour. Second, `dkim_domainonly` under `dkim_match_from` is no longer inverted. Third, phishtank.rspamd.com is disabled by default. Recommended upgrade for all users.