Artur Asadullin, Head of Infrastructure,
An agent that runs on every node of every cluster has to adapt to its host. Kernels are older than the documentation pretends. OpenSSL ships a minor release every six months: the public API is stable across minor releases, but the internal layout of opaque connection objects can change between any two, and several have. Application servers recycle their workers on a schedule, and the agent finds out about it through events that may or may not arrive in order. The shape of this release is not "what new things does the agent see" so much as "what kinds of host conditions does the agent now handle in stride."
Five changes are worth a paragraph each: a sampling control for tracing on high-volume tiers; a new way of decrypting OpenSSL traffic that does not need to keep up with libssl's internal reorganisations; a graceful-degradation path for kernels that do not support the agent's full kernel-level instrumentation; a self-heal in PHP-FPM monitoring across worker recycles; and a cleaner startup log so the interesting messages are not buried.
The agent now exposes a configurable trace sampling rate, --traces-sampling, with a default of 1.0 — every L7 span is emitted. The control matters on the largest web tiers, where a single host can emit tens of thousands of spans per second from request-level instrumentation alone, more than the downstream pipeline cares to keep. Operators can dial sampling down to a fraction without disabling tracing, and metrics and logs are not affected by the rate: only the trace channel narrows.
Some processes never want trace spans at all — batch jobs, internal chatter between sidecars, health-check probers that exist only to poke. A per-process environment-variable opt-out, set in the application's pod spec or systemd unit, removes that one process from tracing while leaving its metrics and logs intact. The opt-out is read once per process at first contact and cached for the process's lifetime, so it costs nothing in steady state.
To capture decrypted traffic on a TLS connection, the agent attaches to the entry points of the OpenSSL library — SSL_read and SSL_write, the userspace functions an application calls to send and receive plaintext through the TLS pipe. The catch is finding the file descriptor those entry points operate on. Their signature does not carry one — it carries a pointer to OpenSSL's own connection object, which holds the socket somewhere inside.
The previous design read that socket directly out of the OpenSSL object's memory layout. That worked, but the layout is private to OpenSSL. Maintaining the path meant carrying separate descriptors of the layout across the live 3.0, 3.2, 3.3, 3.4, 3.5, and 3.6 series, plus the still-deployed-but-EOL 1.1.1, with another descriptor due whenever a new minor lands. The friction was not in any one version; it was in the steady cost of chasing a moving target.
This release replaces that approach with one that does not look inside OpenSSL at all. The plaintext entry points still capture their buffer and length, but the file descriptor is filled in by correlating with the underlying read or write syscall that OpenSSL itself eventually performs to move the bytes onto the socket. The kernel knows which descriptor that syscall used; the agent reads it from there. The OpenSSL object's layout becomes irrelevant — every libssl version the agent attaches to, from 1.1.1 through the current 3.x series, is covered by one mechanism, and future minor releases do not require a new descriptor.
The honest part of this story is that the change is large for one feature: the kernel-side capture is a different mechanism than the userspace dereference it replaces, and the rollout had to keep up with the existing inode-level deduplication that already amortises one attachment across all processes sharing the same libssl. The reward is that every OpenSSL minor release stops being a release-blocking event for the agent — and the same is true for the next one.
The agent makes heavy use of kernel-level instrumentation through eBPF, which is the cheapest way to observe network and L7 traffic without slowing the host down. A few of the agent's BPF programs require features that landed in Linux kernel 5.6 — on older kernels the verifier rejects them at load time.
On those hosts, the agent now starts in a documented degraded mode instead of failing closed. It logs which kernel features were unavailable and continues with the file-system-based collectors: process discovery, systemd-service classification, host metrics, syslog ingestion, and Kubernetes event capture all stay on. L7 capture and TLS decryption are skipped, because they fundamentally need the kernel-level path, and the startup log says so plainly.
A second change in the same spirit applies to the standalone agent binary. It is now compiled against glibc 2.31 (the Debian 11 toolchain), so it installs cleanly on Ubuntu 20.04 (glibc 2.31) and Debian 11 (2.31) — among the older enterprise baselines still in broad production — without manual workarounds.
PHP-FPM is structured around a pool of worker processes that handle requests concurrently. When an operator sets pm.max_requests — typically a value in the hundreds or low thousands, since the default of 0 leaves workers in place indefinitely — the pool retires a worker after it has served that many requests and starts a replacement. On a busy host, a worker is born, serves its quota, and is retired roughly every few minutes. The agent's PHP-FPM request metrics need to register every one of those new workers as it appears.
On busy hosts, workers can recycle faster than process-start events can always be delivered through the kernel's bounded event buffer. The agent now reconciles a worker's bookkeeping on the spot the first time it sees that worker's network traffic, querying the kernel directly for the process's accounting information and installing the missing record. A second, slower sweep runs on a timer and walks the process table for anything that has slipped through. Both paths are bounded so they cannot become a load source of their own.
The result is that PHP-FPM request rates stay accurate across long-running hosts, regardless of how often workers cycle underneath them.
The startup log is one of the few places where an agent has the operator's attention. A startup log that buries the interesting line in eight pages of routine output trains people to scroll past it. Recent releases have been trimming this back: log lines from optional integrations no longer report as errors when the optional thing is simply absent, and the messages that survive are the ones a reader might act on.
This release adds one more thing to that pattern. At the end of the agent's initial discovery window — the few seconds where it walks the host and figures out what is running on it — the agent now prints a one-line summary of what it found, grouped by container type, with a handful of example identifiers per group. The dashboard already shows this information later, but having it in the agent's own log makes the discovery phase legible without scrolling, and makes it obvious whether the agent has seen what an operator expected it to see.
This release is about resilience. The agent now operates across a wider range of host conditions — high-volume tiers that need a sampling control, OpenSSL releases that arrive every six months, kernels older than the agent's preferred path, application servers that recycle workers faster than events arrive, startup logs that benefit from a one-line summary instead of pages of routine output. Each of these is a place where the host environment varies more than a single fixed implementation can absorb, and the agent now meets that variation directly: with a knob the operator can turn, a mechanism that does not depend on private library internals, a graceful path through reduced kernel capability, a self-reconciling worker record, and a clearer record of what was discovered at startup.
Most of what an observability agent does is invisible until the day someone needs the data. The work in this release widens the range of conditions under which that data stays accurate and the agent stays out of the operator's way — so the data is there, ready, when the day comes.
Share it:
A pattern for tool-using AI agents — make findings the unit of output, gate the top-tier severity claim on objective evidence at record time, and require cross-source corroboration for any elevated claim at synthesis time. The agent's report becomes something a reviewer can rank.
How to predict what PostgreSQL would do with a query without running it — the statistics the planner reads, where they live, and how an offline analyzer reconstructs the plan from collected pg_class, pg_indexes, and pg_stats data, plus the honest boundary between structural prediction and the live cost model.
Notes from an April 2026 release: the logs page redrawn with a collapsible filter sidebar and resizable columns, Lighthouse split onto its own tab in URL monitoring, an AI dashboard agent that reads its own rendered output and revises itself, and an observability agent that now records packet drops, DNS query shape, TCP round-trip time, and syslog alongside the application-protocol view.
