Added

• **PDF text extraction**: Extract text from PDF attachments for content analysis with text quality filtering, FlateDecode and ASCIIHexDecode filter support, UTF-16 detection and conversion, ligature support for Standard and MacRoman encodings, and new `rspamadm mime -r` flag for raw extraction ([#5762](https://github.com/rspamd/rspamd/pull/5762))
• **Task registry for Lua**: New task registry prevents use-after-free bugs in async Lua callbacks using counter-based keys instead of raw pointers for safe validation ([#5803](https://github.com/rspamd/rspamd/pull/5803))
• **ClickHouse extra tables API**: Plugins can dynamically register custom ClickHouse tables via `rspamd_plugins['clickhouse'].register_extra_table()` with per-table schemas, row callbacks, and independent retention settings ([#5810](https://github.com/rspamd/rspamd/pull/5810))
• **WebUI error log**: New error log modal tracking last 50 API errors with unseen error counter badge, copy-to-clipboard support with HTTP fallback, and color-coded error types with responsive column hiding ([#5821](https://github.com/rspamd/rspamd/pull/5821))

Fixed

• **Neural symbol flags**: Include symbols with no flags by default instead of excluding them
• **Symcache FINE propagation**: Make FINE propagation deterministic ([#5825](https://github.com/rspamd/rspamd/issues/5825))
• **URL numeric IP**: Prevent false positives from numeric IP regeneration in mailto URLs ([#5823](https://github.com/rspamd/rspamd/issues/5823))
• **Settings selector**: Allow spaces in selector regexps
• **Thai language**: Prevent R_MIXED_CHARSET false positives by marking Thai as having diacritics ([#5817](https://github.com/rspamd/rspamd/pull/5817), [#5799](https://github.com/rspamd/rspamd/issues/5799))
• **GPT plugin**: Improved output format instructions for more reliable responses ([#5815](https://github.com/rspamd/rspamd/pull/5815), [#5670](https://github.com/rspamd/rspamd/issues/5670))
• **io_uring HTTP**: Handle connection errors properly in HTTP client backend
• **MIME Lua stack**: Restore Lua stack properly in second-pass detection
• **Redis stats**: Add null checks for task in callbacks
• **Lua 5.4 compatibility**: Fixes in ClickHouse and Elastic plugins ([#5774](https://github.com/rspamd/rspamd/pull/5774))
• **Task lifetime crashes**: Fix potential crashes when Redis is unavailable longer than task lifetime ([#5803](https://github.com/rspamd/rspamd/pull/5803))

Improved

• **replxx build**: Fix build with LLVM 21+
• **macOS build**: Add Homebrew LLVM libc++ library path
• **fmt library**: Update to version 12.1.0
• **SDK headers**: Avoid SDK headers in include path when package ROOT is specified

Added

• **Metadata Exporter Formatters**: New formatters `multipart`, `msgpack`, and `json_with_message` for better message content handling including binary support and multipart uploads
• **DMARC RUA Validation**: New `rspamadm dmarc_report --recheck-rua` flag to re-validate cached report addresses against exclusion maps (useful with RBLs)
• **Selective AR Header Removal**: New `remove_ar_from` setting in milter_headers to selectively remove Authentication-Results headers from specific domains

Improved

• **Metadata Exporter Docs**: Added comprehensive example for handling multipart metadata exports with Python/aiohttp

Deprecated

• **Metadata Exporter**: `meta_headers` option is deprecated; use `multipart` or `msgpack` formatters instead

Added

• **Composites optimization**: Inverted index with bloom filter for fast negative symbol lookups, precomputed atom types, and composites statistics with control protocol command ([#5764](https://github.com/rspamd/rspamd/pull/5764))
• **Multimap combinators**: Support for combinator option in multimap selector rules for flexible matching logic ([#5766](https://github.com/rspamd/rspamd/pull/5766))
• **SPF flattening tool**: New rspamadm utility for SPF record flattening with macro preservation ([#5757](https://github.com/rspamd/rspamd/pull/5757))
• **URL deep processing**: Architecture for C-to-Lua URL filter consultation with async HTML URL processing ([#5748](https://github.com/rspamd/rspamd/pull/5748))
• **Obfuscated URL detection**: Enhanced url_suspect plugin with rspamd_util.decode_html_entities for detecting obfuscated URLs ([#5761](https://github.com/rspamd/rspamd/pull/5761))
• **lua_shape validation**: New validation library replacing tableshape with T.callable() type and callable defaults support ([#5754](https://github.com/rspamd/rspamd/pull/5754))
• **Whitelist symbol flags**: Auto-mark whitelist symbols with SYMBOL_TYPE_FINE flag for better scoring granularity

Fixed

• **url_suspect false positives**: Fix plugin causing massive false positives with high URL volume messages ([#5758](https://github.com/rspamd/rspamd/pull/5758))
• **Network chunking loop**: Prevent infinite loop in split_networks_into_chunks() function ([#5763](https://github.com/rspamd/rspamd/pull/5763))
• **Memory leak**: Fix leak in custom tokenizer result handling
• **lua_shape compatibility**: Multiple fixes for transform logic, registry resolution, error safety, and tableshape compatibility ([#5754](https://github.com/rspamd/rspamd/pull/5754))
• **Composites inverted index**: Fix group matchers handling and improve atom polarity detection ([#5764](https://github.com/rspamd/rspamd/pull/5764))
• **Lua composites**: Copy expression string to memory pool to prevent use-after-free ([#5764](https://github.com/rspamd/rspamd/pull/5764))
• **D3 libraries**: Update with bug fixes and validation improvements ([#5765](https://github.com/rspamd/rspamd/pull/5765))
• **Settings processing**: Keep groups_*/symbols_* fields in settings for runtime processing
• **Redirect URL encoding**: Properly encode URLs with unencoded spaces and special characters ([#5748](https://github.com/rspamd/rspamd/pull/5748))
• **WebUI hover**: Restore hover colors for symbols and fix hover behavior outside status tables ([#5741](https://github.com/rspamd/rspamd/pull/5741), [#5742](https://github.com/rspamd/rspamd/pull/5742))
• **External relay**: Fix mixins and confighelp for external_relay module ([#5754](https://github.com/rspamd/rspamd/pull/5754))

Improved

• **url_suspect performance**: Optimize plugin for high URL volume messages with better deduplication ([#5758](https://github.com/rspamd/rspamd/pull/5758))
• **tableshape migration**: Complete migration of all plugins and libraries from tableshape to lua_shape ([#5754](https://github.com/rspamd/rspamd/pull/5754))
• **Configuration simplification**: Remove use_*_map flags and simplify URL processing configuration ([#5748](https://github.com/rspamd/rspamd/pull/5748))
• **Whitelist maps**: Use contemporary API for maps in whitelist module ([#5746](https://github.com/rspamd/rspamd/pull/5746))
• **WebUI symbols**: Improved hover colors and behavior for symbols display ([#5760](https://github.com/rspamd/rspamd/pull/5760))
• **Plugins registry**: Centralized plugins registry with reworked mixins and documentation ([#5754](https://github.com/rspamd/rspamd/pull/5754))

Added

• **HTML fuzzy hashing**: Structural similarity matching for HTML content with per-rule text_hashes toggle and dedicated shingles generation ([#5661](https://github.com/rspamd/rspamd/pull/5661), [#5720](https://github.com/rspamd/rspamd/pull/5720))
• **Fuzzy TCP protocol**: Full TCP support in fuzzy storage and check with auto-switch, connection management, and proper framing ([#5669](https://github.com/rspamd/rspamd/pull/5669))
• **CTA URL extraction**: New task:get_cta_urls() API and dedicated CTA module for proper call-to-action domain extraction ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **Web search context**: LLM plugin integration with search API for domain context enrichment with Redis caching ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **HTML URL rewriting**: Infrastructure for async HTML URL rewriting with Lua bindings and UTF-8 support ([#5676](https://github.com/rspamd/rspamd/pull/5676))
• **Dark mode**: Full dark mode implementation in WebUI with theme toggle and auto detection ([#5725](https://github.com/rspamd/rspamd/pull/5725))
• **Email aliases**: Advanced aliases resolution with loop detection for converging paths and expand_multiple mode ([#5655](https://github.com/rspamd/rspamd/pull/5655))
• **ESMTP arguments**: Milter ESMTP argument parsing with Lua API access for per-recipient metadata ([#5663](https://github.com/rspamd/rspamd/pull/5663))
• **URL hash method**: Exposed url:get_hash() method for efficient deduplication without string conversion overhead ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **Postfix wizard**: Configwizard integration for automated Postfix setup using postconf utility ([#5667](https://github.com/rspamd/rspamd/pull/5667))
• **BSD workflows**: Comprehensive GitHub Actions workflows for FreeBSD, NetBSD, and OpenBSD with Lua version selection ([#5726](https://github.com/rspamd/rspamd/pull/5726), [#5728](https://github.com/rspamd/rspamd/pull/5728), [#5729](https://github.com/rspamd/rspamd/pull/5729))
• **Automated code review**: GitHub Actions workflow for Droid-powered code review with Rspamd-specific guidelines ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **Multimap multisymbol**: Support for symbols with leading numerals in multimap plugin ([#5680](https://github.com/rspamd/rspamd/pull/5680))
• **Remove headers array**: Support array of positions for milter remove_headers operations ([#5673](https://github.com/rspamd/rspamd/pull/5673))
• **Client IP forwarding**: Proxy client IP preservation in message headers through chain ([#5671](https://github.com/rspamd/rspamd/pull/5671))
• **Milter header support**: rspamc --mime support for milter.add_headers object format ([#5684](https://github.com/rspamd/rspamd/pull/5684))
• **Public suffix automation**: Automatic synchronization of public suffix list via GitHub Actions ([#5718](https://github.com/rspamd/rspamd/pull/5718))
• **Fuzzy migration utility**: Redis migration utility for fuzzy storage data ([#5692](https://github.com/rspamd/rspamd/pull/5692))
• **Integration test suite**: Docker-based integration tests with ASAN, leak detection, and real corpus ([#5688](https://github.com/rspamd/rspamd/pull/5688))
• **DMARC report headers**: Auto-Reply-To and Precedence headers to prevent out-of-office replies ([#5686](https://github.com/rspamd/rspamd/pull/5686))
• **NetBSD memory support**: Memory usage tracking for NetBSD platform ([#5726](https://github.com/rspamd/rspamd/pull/5726))

Fixed

• **DNS truncation**: Preserve req->pos during reply validation to prevent packet truncation on UDP-to-TCP retransmits ([#5739](https://github.com/rspamd/rspamd/pull/5739))
• **DNS transaction ID**: Regenerate transaction ID before copying to TCP buffer to avoid collisions ([#5739](https://github.com/rspamd/rspamd/pull/5739))
• **DMARC report batching**: Add batching and forced GC for Redis connections to prevent connection pool exhaustion ([#5737](https://github.com/rspamd/rspamd/pull/5737))
• **Batch size validation**: Validate and normalize batch_size to prevent fractional indexing and loop errors ([#5737](https://github.com/rspamd/rspamd/pull/5737))
• **Allocator mismatches**: Fix jemalloc/system malloc mixing in getline(), hiredis, and libucl to prevent crashes ([#5721](https://github.com/rspamd/rspamd/pull/5721), [#5724](https://github.com/rspamd/rspamd/pull/5724))
• **Hyperscan version**: Use runtime version instead of compile-time for database validation and auto-recreate invalid cache files ([#5724](https://github.com/rspamd/rspamd/pull/5724))
• **DNS round-robin**: Fix nameserver rotation from /etc/resolv.conf using ROUND_ROBIN instead of MASTER_SLAVE ([#5721](https://github.com/rspamd/rspamd/pull/5721))
• **Memory leaks**: Fix leaks in fuzzy storage khash, upstream parsing, address parsing, OpenSSL providers, and UCL objects ([#5709](https://github.com/rspamd/rspamd/pull/5709))
• **Fuzzy TCP bugs**: Fix double-release, timeout handling, buffer overflow, endianness mismatch, and race conditions ([#5669](https://github.com/rspamd/rspamd/pull/5669), [#5716](https://github.com/rspamd/rspamd/pull/5716))
• **Shutdown tracking**: Keep srv events active during shutdown to track auxiliary processes via pipe notifications ([#5728](https://github.com/rspamd/rspamd/pull/5728))
• **ARC signing**: Restore strict header ordering per RFC 8617 and add ed25519 key support ([#5684](https://github.com/rspamd/rspamd/pull/5684))
• **Composite evaluation**: Implement two-phase evaluation for postfilter dependencies and fix symbol lookup ([#5681](https://github.com/rspamd/rspamd/pull/5681))
• **URL extraction DoS**: Refactor to use hash-based deduplication with 50k URL limit to prevent string table exhaustion ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **Bayes autolearn**: Allow skipping local/authenticated mail in default autolearn condition ([#5679](https://github.com/rspamd/rspamd/pull/5679))
• **Bayes Redis discovery**: Improve Redis server discovery for Bayes storage ([#5714](https://github.com/rspamd/rspamd/pull/5714))
• **ESMTP args parsing**: Robust per-recipient parsing in milter with safe cursor advance and refcount management ([#5663](https://github.com/rspamd/rspamd/pull/5663))
• **HTML attribute offsets**: Correct offset calculation for URL rewriting in HTML attributes ([#5676](https://github.com/rspamd/rspamd/pull/5676))
• **OpenBSD support**: Fix kinfo_proc member names and disable Hyperscan on OpenBSD ([#5729](https://github.com/rspamd/rspamd/pull/5729))
• **FreeBSD packages**: Fix zstd package name and add IGNORE_OSVERSION for version mismatches ([#5729](https://github.com/rspamd/rspamd/pull/5729))
• **NetBSD build**: Fix package installation with pkgin and correct dependency names ([#5726](https://github.com/rspamd/rspamd/pull/5726), [#5729](https://github.com/rspamd/rspamd/pull/5729))
• **Bayes metadata leak**: Fix memory leak in stat metadata tokenization ([#5688](https://github.com/rspamd/rspamd/pull/5688))
• **MIME anonymization**: Remove Authentication-Results and anonymize envelope-from in Received headers for LLM processing ([#5687](https://github.com/rspamd/rspamd/pull/5687))
• **Multimap symbols**: Handle symbols with leading numerals in multimap ([#5680](https://github.com/rspamd/rspamd/pull/5680))
• **Aliases validation**: Prevent creation of malformed email addresses in aliases module ([#5655](https://github.com/rspamd/rspamd/pull/5655))
• **HTML fuzzy cache**: Fix cache key collision between text and HTML fuzzy hashes ([#5661](https://github.com/rspamd/rspamd/pull/5661))
• **CSS normalization**: Fix CSS class normalization in HTML fuzzy token generation ([#5661](https://github.com/rspamd/rspamd/pull/5661))

Improved

• **RBL configuration**: Refactored with new from selectors, content_urls checks, and lower_utf8 for hashed domains ([#5733](https://github.com/rspamd/rspamd/pull/5733))
• **URL prioritization**: Prioritize CTA URLs in redirector and Lua helpers with proper phishing bonus preservation ([#5732](https://github.com/rspamd/rspamd/pull/5732))
• **Fuzzy checks structure**: New structured checks configuration with backward-compatible legacy flags support ([#5720](https://github.com/rspamd/rspamd/pull/5720))
• **Hash performance**: Replace GHashTable with khash in fuzzy_check and CTA URL extraction for better performance ([#5720](https://github.com/rspamd/rspamd/pull/5720), [#5732](https://github.com/rspamd/rspamd/pull/5732))
• **WebUI icons**: Replace Glyphicons with FontAwesome SVG icons ([#5702](https://github.com/rspamd/rspamd/pull/5702))
• **WebUI libraries**: Update CodeJar to 4.3.0, Node.js/ESLint, and D3-based visualization libs ([#5684](https://github.com/rspamd/rspamd/pull/5684), [#5717](https://github.com/rspamd/rspamd/pull/5717), [#5738](https://github.com/rspamd/rspamd/pull/5738))
• **Bayes learn guards**: Make learn guards configurable instead of hardcoded ([#5701](https://github.com/rspamd/rspamd/pull/5701))
• **Lua logger**: Add type specifiers support for better formatting ([#5668](https://github.com/rspamd/rspamd/pull/5668))
• **DMARC reporting**: Refactor to use helper functions and async maps for better maintainability ([#5722](https://github.com/rspamd/rspamd/pull/5722))
• **Config refcounting**: Add CFG_REF_* macros with debug logging for better lifecycle tracking ([#5709](https://github.com/rspamd/rspamd/pull/5709))
• **Memory pool destructors**: Smart destructor preallocation based on pool type with specialized allocation strategies ([#5693](https://github.com/rspamd/rspamd/pull/5693))
• **Heap implementation**: Convert to fully intrusive kvec-based implementation eliminating double allocation ([#5693](https://github.com/rspamd/rspamd/pull/5693))
• **OpenSSL lifecycle**: Move providers from global to libs_ctx for clearer ownership ([#5709](https://github.com/rspamd/rspamd/pull/5709))
• **Libucl stack management**: Automatic stack preservation for included files without outer braces ([#5717](https://github.com/rspamd/rspamd/pull/5717))
• **Body rewriting**: Improve body rewriting support in rspamc and proxy ([#5675](https://github.com/rspamd/rspamd/pull/5675))
• **AI assistant config**: Add Claude Code and Cursor AI configuration for development ([#5667](https://github.com/rspamd/rspamd/pull/5667))
• **Fuzzy logging**: Add protocol logging and reduce dumb log verbosity ([#5669](https://github.com/rspamd/rspamd/pull/5669), [#5701](https://github.com/rspamd/rspamd/pull/5701))
• **Docker configuration**: Add explicit console logging and runtime flags ([#5701](https://github.com/rspamd/rspamd/pull/5701))

Added

• **Fuzzy check encryption**: Separate encryption keys for read and write operations in fuzzy_check plugin ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **DKIM ED25519 support**: Full ED25519 support for DKIM signing and verification with OpenSSL version checks ([#5664](https://github.com/rspamd/rspamd/pull/5664))
• **Vault KV v2 support**: HashiCorp Vault KV version 2 support for DKIM key management with backward compatibility ([#5654](https://github.com/rspamd/rspamd/pull/5654))
• **MetaDefender integration**: MetaDefender Cloud Lua module for SHA256 hash lookups as free-tier anti-malware scanning alternative ([#5656](https://github.com/rspamd/rspamd/pull/5656))
• **LLM context support**: User/domain context support for LLM-based classification with Redis-based conversation context ([#5647](https://github.com/rspamd/rspamd/pull/5647))
• **DMARC RUA exclusion**: Configuration option to exclude specific RUA addresses from DMARC report storage ([#5653](https://github.com/rspamd/rspamd/pull/5653))

Fixed

• **DKIM bodyhash calculation**: Fixed relaxed bodyhash calculation for lines with only spaces to comply with RFC 6376 Section 3.4.4 ([#5662](https://github.com/rspamd/rspamd/pull/5662))
• **DKIM key loading**: Fixed ED25519 key loading to prevent memory corruption in union handling ([#5664](https://github.com/rspamd/rspamd/pull/5664))
• **HTTP map intervals**: Enforced server-controlled refresh intervals and prevented aggressive polling behavior ([#5660](https://github.com/rspamd/rspamd/pull/5660))
• **HTTP map overflow**: Prevented time_t overflow in expires header processing ([#5660](https://github.com/rspamd/rspamd/pull/5660))
• **Once received plugin**: Fixed duplicate symbol addition by changing break to return in check_quantity_received ([#5658](https://github.com/rspamd/rspamd/pull/5658))
• **Redis Sentinel**: Properly propagate unused Sentinel options ([#5597](https://github.com/rspamd/rspamd/pull/5597))
• **Fuzzy check decryption**: Fixed reply decryption when using separate read/write keys ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **Fuzzy check fallback**: Added fallback when only one specific encryption key is set ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **Fuzzy check filtering**: Fixed duplicate key filtering in reply decryption ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **Fuzzy ping servers**: Allow read/write servers configuration ([#5665](https://github.com/rspamd/rspamd/pull/5665))

Improved

• **Fuzzy check performance**: Refactored encryption key selection into helper functions for better maintainability ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **Fuzzy check efficiency**: Stop early when found a correct key to improve performance ([#5665](https://github.com/rspamd/rspamd/pull/5665))
• **Development workflow**: Added cursor rules for improved development experience ([#5665](https://github.com/rspamd/rspamd/pull/5665))

Added

• **Archive module**: Full support for encrypted ZIP archives, including both ZipCrypto and AES encryption; both reading and writing of AES-encrypted ZIP archives is supported with updated Lua bindings using libarchive for flexible compatibility with all standard ZIP encryption schemes
• **Encrypted maps**: Support for encrypted maps to enable new map distribution scenarios
• **Redis TLS**: Configurable TLS connections in Redis backend for improved compatibility in secure environments

Improved

• **MIME encoding refactoring**: Major overhauls and multiple fixes for MIME encoding logic, including improved handling and decoding of UTF-8 in MIME headers, resulting in more robust email processing and better compatibility
• **Learning system**: Numerous fixes to learn checks and autolearn flag handling, prevention of duplicate message learning, and extended multiclass learning test coverage
• **Map helpers alignment**: Map helpers now enforced to be aligned to 64 bytes to prevent unaligned memory access errors on certain platforms
• **CLI enhancements**: Enhanced secretbox CLI and additional security test coverage
• **Platform compatibility**: Improved compatibility with Lua versions above 5.1 and better support for 32-bit platforms

Fixed

• **Critical fixes**: Fixed bug when converting zero-length strings to numbers
• **XML parsing**: Fixed XML prolog detection in lua_magic module
• **Build issues**: Fixed build issues on 32-bit platforms
• **Empty input handling**: Addressed issues with empty input handling in lua_magic
• **Test stability**: Improved stability of automated testing with multiple miscellaneous test fixes
• **Compatibility**: Minor compatibility improvements and bugfixes (buffer allocation, missing cmath include, etc.)

Breaking Changes

• Multiclass Bayes classification with named classes (2–20 classes), processed in a single Redis call per message. ([#5547](https://github.com/rspamd/rspamd/pull/5547))
• Backward compatible with `is_spam`; new configuration supports labeled classes and autolearn.
• Lua API and autolearn support multiclass workflows with class‑aware Redis caching.
• CLI: `rspamc learn_class:transactional …`, `rspamc learn_class:newsletter …`.
• Lua API: `task:get_multiclass_result()` returns class probabilities and confidence.

Added

• Neural module overhaul to a provider‑based architecture (symbols, LLM embeddings), versioned and backward compatible. ([#5579](https://github.com/rspamd/rspamd/pull/5579))
• Pluggable feature fusion with trained normalization (unit/zscore/none) applied at train and inference.
• Redis‑backed caching for LLM embeddings to control cost and latency.
• Multimap selectors: SA‑style selectors with dedicated `selector` field; integrated with Hyperscan and regex cache. ([#5615](https://github.com/rspamd/rspamd/pull/5615))
• MIME parser automatically detects part types. ([#5619](https://github.com/rspamd/rspamd/pull/5619), [#5608](https://github.com/rspamd/rspamd/pull/5608))
• HTML parser extracts more features for downstream modules.

Improved

• HTTP timeouts configuration and handling. ([#5614](https://github.com/rspamd/rspamd/pull/5614), [#5603](https://github.com/rspamd/rspamd/pull/5603), [#5601](https://github.com/rspamd/rspamd/pull/5601))
• Upstream reliability with probe mode; reduced need for forced revive.
• DNS nameserver resolution switched to `getaddrinfo`.
• Standardized on C++20; build, test, and CI improvements (including ARM support and modern fallback maps). ([#5592](https://github.com/rspamd/rspamd/pull/5592), [#5598](https://github.com/rspamd/rspamd/pull/5598), [#5580](https://github.com/rspamd/rspamd/pull/5580))
• Regular code cleaning and RPM tweaks.
• WebUI: E2E scan test flows; Bootstrap upgrade; Bayes class management from the web interface. ([#5606](https://github.com/rspamd/rspamd/pull/5606), [#5607](https://github.com/rspamd/rspamd/pull/5607))
• GPT/LLM integrations: improved parameter/prompt handling; initial support for OpenAI GPT‑5 and other models. ([#5612](https://github.com/rspamd/rspamd/pull/5612), [#5572](https://github.com/rspamd/rspamd/pull/5572))

Fixed

• DCC plugin rewritten and optimized. ([#5602](https://github.com/rspamd/rspamd/pull/5602))
• DKIM relaxed body canonicalization; improved reliability in multimap, WebUI, and configuration. ([#5593](https://github.com/rspamd/rspamd/pull/5593))
• Numerous minor bug fixes and CI/build improvements.

Added

• **Bayes Classifiers Endpoint**: Add `/bayes/classifiers` HTTP endpoint for better classifier management
• **Enhanced Scheduling**: Further improvements in scheduling next checks for better performance
• **AEAD Testing**: Add comprehensive tests for AEAD (Authenticated Encryption with Associated Data) used in Rspamd
• **Improved Maps Consistency**: Better consistency in cached flag for maps and logging when dealing with Expires

Improved

• **Maps Locking**: Eliminate maps locking for better performance and reduced contention
• **Proxy Headers**: Fix proxy headers duplication and improve header handling
• **Milter Logic**: Do not add log tag header in milter logic for cleaner output
• **Connection Handling**: Do not explicitly add Connection header if it's already present
• **Attachment Processing**: Treat *.library-ms and *.search-ms attachments as harmful (with proper application)
• **Maps Concurrent Loading**: Fix race conditions in maps concurrent loading
• **Write Operations**: Try to avoid incomplete writes for better data integrity

Fixed

• **Maps Locking Issues**: Eliminate maps locking to prevent deadlocks and improve performance
• **Proxy Headers Duplication**: Fix proxy headers duplication issue
• **Milter Header Logic**: Fix logic error in milter_headers.lua: skip_wanted() function
• **Connection Header**: Prevent duplicate Connection headers in proxy
• **Maps Concurrent Access**: Fix race conditions in maps concurrent loading
• **Incomplete Writes**: Prevent incomplete writes that could cause data corruption
• **Attachment Security**: Properly apply security rules for *.library-ms and *.search-ms attachments
• **Logging Consistency**: Improve consistency in logging when dealing with Expires and cached flags

Changed

• **Maps Architecture**: Eliminate maps locking mechanism for better performance
• **Header Processing**: Improve header processing to avoid duplicates
• **Attachment Handling**: Enhanced security for Windows library and search attachments
• **Scheduling Logic**: Improved scheduling algorithm for next checks
• **Logging Format**: Better consistency in logging output and cached flag handling

Security

• **Attachment Security**: Enhanced protection against potentially harmful Windows library and search attachments
• **Header Validation**: Improved header validation to prevent duplication and injection
• **Write Protection**: Better protection against incomplete writes that could lead to data corruption

Performance

• **Maps Performance**: Eliminate maps locking for better concurrent access performance
• **Scheduling**: Improved scheduling algorithm for better resource utilization
• **Header Processing**: Optimized header processing to reduce overhead

Testing

• **AEAD Testing**: Comprehensive tests for AEAD encryption used throughout Rspamd
• **Maps Testing**: Improved testing for maps concurrent loading scenarios
• **Header Testing**: Better testing for header processing and duplication scenarios

Added

• **GPT Module Enhancements**: Add Ollama support and token usage logging for OpenAI-compatible APIs
• **Fuzzy Storage Improvements**: Add support for separate read and write servers in fuzzy check
• **CDB Maps Support**: Allow CDB files as external maps for improved performance
• **Contextal Integration**: New plugin to integrate with Contextal platform for advanced threat detection
• **Enhanced Logging**: Allow to specify max log tag length for all log messages and log tag in proxy
• **Proxy Improvements**: Add keep-alive support, HTTPS backend connections, and extra headers specification
• **Redis Version Support**: Allow to specify Redis version for compatibility
• **HEIC File Support**: Add HEIC files patterns for better image analysis
• **Enhanced Maps Management**: Show all maps status and output content for all maps
• **File Upload in WebUI**: Add file upload functionality to Test Selectors
• **Convenience Methods**: Add various convenience methods for Lua development
• **Enhanced Fuzzy Operations**: Add deletion of specific fuzzy hashes in WebUI
• **Improved Error Handling**: Better error messages for multimap invalid types

Improved

• **Security Enhancements**: Critical fix to disable macros and file variables by default in lua-ucl
• **Maps Loading**: Fix race condition in maps loading by unlocking backend on switch
• **Greylisting**: Improve body hash calculations for better accuracy
• **Replies Module**: Rework to consider all recipients and use SMTP ones
• **WebUI Updates**: Update Bootstrap, D3 and PrismJS libraries to latest versions
• **Lua Logging**: Major improvements to logging output format and complex table key support
• **Fuzzy Storage**: Better handling of fuzzy lua callback when there are no shingles
• **Milter Headers**: Fix logic error in skip_wanted() function
• **Known Senders**: Improved recipients test logic and consistency with replies module
• **Maps Management**: Grey out not loaded maps in the Maps table for better visibility
• **Redis Integration**: Improve Redis script loading and caching framework usage
• **Configuration**: Use safe parsers everywhere except configuration for security
• **Build System**: Modernize cmake and improve sanitizers support
• **Documentation**: Update API docs for multiple HTTP headers and various other improvements

Fixed

• **Critical Security**: Disable macros and file variables by default in lua-ucl parser
• **Race Conditions**: Fix race condition in maps loading by unlocking backend on switch
• **Memory Issues**: Fix lua-bit stack buffer overflow vulnerability
• **Proxy Crashes**: Prevent crashes when accessing upstream address in self-scan mode
• **Maps Management**: Fix maps IDs and static maps description passing
• **Fuzzy Storage**: Filter invalid domains in fuzzy extra data
• **Redis Integration**: Fix various Redis-related issues and improve script loading
• **WebUI Issues**: Fix Fuzzy hashes card close button alignment and map editor modal handling
• **Build Issues**: Fix Debian package build and RPM log directory attributes
• **Compiler Warnings**: Fix various compile warnings and compatibility issues
• **Test Framework**: Fix various test issues and improve test configuration
• **Documentation**: Fix writing rules tutorial link and various documentation issues
• **URL Lists**: Update default URL for openphish and fix various URL-related issues
• **Headers Processing**: Fix header processing in various modules
• **Lua Integration**: Fix various Lua-related issues and improve error handling

Changed

• **Configuration**: Use safe parsers everywhere except configuration for enhanced security
• **Maps Architecture**: Rework to use locks/loaded per backend for all maps
• **Replies Logic**: Consider all recipients and use SMTP ones instead of just From header
• **Fuzzy Storage**: Remove servers completely and use `read_servers` for all compatibility
• **Logging Format**: Improve format string processing and logging output
• **Build System**: Modernize cmake and rework OSDep for better maintainability
• **WebUI**: Rework file upload JS implementation and improve user interface
• **Documentation**: Update various documentation files and improve API documentation
• **Error Handling**: Improve error messages and handling throughout the codebase
• **Performance**: Various performance improvements in maps, Redis, and fuzzy operations

Removed

• **Deprecated Features**: Remove various deprecated and unused code paths
• **Unused Helpers**: Remove unused cleanTextUpload helper and other obsolete code
• **Legacy Support**: Remove compatibility code that is no longer needed

Security

• **Critical Fix**: Disable macros and file variables by default in lua-ucl parser
• **Memory Safety**: Fix lua-bit stack buffer overflow vulnerability
• **Input Validation**: Improve input validation and error handling throughout
• **Configuration Security**: Use safe parsers everywhere except configuration

Performance

• **Maps Loading**: Optimize maps loading with better caching and race condition fixes
• **Redis Operations**: Improve Redis script loading and caching framework usage
• **Fuzzy Storage**: Better handling of fuzzy operations and storage management
• **Memory Usage**: Various memory optimization improvements

Documentation

• **API Documentation**: Update API docs for multiple HTTP headers and various modules
• **Tutorial Links**: Fix writing rules tutorial link and other documentation references
• **Examples**: Add more examples for Lua HTTP and other modules
• **Configuration**: Improve configuration documentation and examples

Testing

• **Test Framework**: Fix various test issues and improve test configuration
• **Fuzzy Tests**: Add comprehensive tests for split, read-only, and write-only server modes
• **CDB Maps**: Add tests for CDB maps functionality
• **HEIC Recognition**: Add tests for HEIC file recognition

Added

• GPT: Add ollama support
• Allow to hash any Lua types
• Allow to use LLM for anonymize
• Add ability to not send response_format in gpt plugin to support gpt4all
• Allow to store shingles as opaque Lua data
• Add 'noop' redis backend for scripts running
• Allow multiple lua scripts for fuzzy storage
• Support LLM models consensus
• GPT: Support reason adding
• Add Redis caching framework
• Add ability to create timers from Lua

Improved

• Expand Detection of Fake Reply Subjects Across Multiple Languages
• Add another acceptable mime type for icon
• Respect ipv4 and ipv6 configurations for rbl resolve_ip
• Set RBL checks to bool true
• Rules regexp url separated and fix no subdomain cases for Google urls
• WebUI: Reset dropdown when clearing filters
• Pass both the multimap and the rules descriptions for combined multimap on create
• Some small fixes to statistics_dump
• More features to GPT plugin
• Better support for maps and IP-related fixes/improvements in settings
• Use caching framework in gpt module
• Try to check maps earlier if their expires is too long

Fixed

• Fix transposed results in `rspamadm fuzzy_ping`
• connIp is not correctly added to request
• Fix Thunderbird for Android marked with FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN
• Fix issue with synchronous Redis calls
• Fix some broken links
• rbl check_types was missing images
• RBL: fix use of `content_urls` and `images` inside `checks`
• Use sub_utf8 to strip headers value to not break utf8 strings
• Properly close multipart/related boundary when adding text footer
• Verify key type to match DKIM signature type
• Avoid collision hacks in mempool variables hash
• Add expiration for neural ham and spam sets
• Properly expire neural ham and spam sets

Changed

• Log queue id with cloudmark analysis string
• Allow to disable RBLs via map
• Prevent option duplicates in rspamd_stats.pl
• Regenerate manpages with recent Pandoc version
• Fix spelling errors in libserver
• Update JavaScript linters

Removed

• Remove nixspam

Breaking Changes

• **Elasticsearch/OpenSearch Plugin**: Major rework with breaking changes
• Added support for Elasticsearch 8 & OpenSearch 2
• Added index policy with logs retention
• Updated configuration format

Added

• Added LRU cache for last filled ratelimit buckets
• Added utilities to manage ratelimit buckets
• Added include/exclude logic for headers processing
• Added `rspamadm mime strip` command for attachments removal
• Added new message anonymization capabilities
• Added more ways to extend Rspamd configuration, including `lua.local.d` folder

Improved

• Improved address rotation algorithm for upstream selection
• Replaced fastutf with simdutf for better architecture support and performance
• Reworked symbol description display on hover in WebUI
• Improved keyboard accessibility in WebUI
• Enhanced symbol rendering in WebUI
• Improved handling of DNS limits in SPF module
• Improved GPT module JSON parsing
• Multiple performance optimizations

Fixed

• Fixed ARC-Seal signing issues
• Fixed RFC 2047 header encoding
• Fixed issues with dynamic keys in fuzzy storage
• Fixed TCP connection handling with cumulative timeouts
• Fixed multiple phishing detection false positives
• Fixed DMARC structured headers encoding

Changed

• Skip extra RBL checks when Received IP matches From IP
• Multimap now uses only distinct text parts for content matching
• Various configuration and logging improvements