I have written that exact block of code in at least four projects. Every time it felt wrong, because it is wrong. PEM is a 30-year-old format with a published spec, and the standard library made me parse it by hand with String.replace.

JDK 27 finally fixes this. JEP 538 ships a real PEM API in java.security: PEMEncoder and PEMDecoder, which read and write keys, certificates, and CRLs without a single replace call. This post covers the old pain in detail, the new two-line replacement, how to encrypt private keys correctly, how to handle files with multiple PEM blocks, and exactly what changed between the preview and the final release. If you have ever managed TLS material in a Java service, like the certificate handling I touched on in my full-stack social identity guide, this is the API you have wanted for a long time.

Why has reading PEM files in Java always been painful?

Reading PEM files in Java was painful because the platform never had a parser for the format, so every key load became manual string surgery. PEM is just Base64-encoded DER wrapped in -----BEGIN X----- and -----END X----- markers, but the JDK gave you no direct way to go from that text to a PrivateKey.

Here is the code I have copy-pasted across projects to load an RSA private key:

// The old way: read a PEM private key by hand
String pem = Files.readString(Path.of("private-key.pem"));
String base64 = pem
    .replace("-----BEGIN PRIVATE KEY-----", "")
    .replace("-----END PRIVATE KEY-----", "")
    .replaceAll("\\s", "");
 
byte[] der = Base64.getDecoder().decode(base64);
PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(der);
KeyFactory factory = KeyFactory.getInstance("RSA");
PrivateKey key = factory.generatePrivate(spec);

Look at everything that can go wrong there. The header text has to match exactly, including whether it says PRIVATE KEY or RSA PRIVATE KEY. You have to know the key is PKCS#8 and not PKCS#1. You have to hardcode "RSA" even though the file itself describes the algorithm. And if the key is encrypted, none of this works at all.

Certificates were slightly less awful, because CertificateFactory does accept PEM-wrapped X.509 input:

// Certificates were tolerable, keys were not
CertificateFactory cf = CertificateFactory.getInstance("X.509");
X509Certificate cert = (X509Certificate) cf.generateCertificate(
    new ByteArrayInputStream(Files.readAllBytes(Path.of("cert.pem"))));

But there was no single, uniform API. Certificates went through CertificateFactory, keys went through KeyFactory plus manual stripping, CRLs went through yet another path, and encrypted keys meant pulling in Bouncy Castle. That fragmentation is the actual problem JEP 538 solves.

What is the PEM API in JDK 27, and what does it replace?

The PEM API is a pair of classes, PEMEncoder and PEMDecoder in java.security, that convert between PEM text and Java security objects through one consistent interface. JEP 538 finalizes it for JDK 27 after two preview rounds, and it replaces every hand-rolled Base64-stripping routine you have ever written.

The design is small on purpose. Both classes are immutable and you get an instance with a static factory:

PEMDecoder decoder = PEMDecoder.of();
PEMEncoder encoder = PEMEncoder.of();

Everything that can be encoded implements a marker interface called BinaryEncodable. The types that implement it are the ones you actually work with:

• PrivateKey, PublicKey, and KeyPair
• X509Certificate and X509CRL
• EncryptedPrivateKeyInfo
• PKCS8EncodedKeySpec and X509EncodedKeySpec
• PEM, a value type that holds a raw block the decoder did not recognize

That single interface is why the API stays tiny. You do not need a different entry point per object type. You decode to the type you expect, or you decode to the interface and pattern-match on what you got.

How do you read a PEM file in Java with PEMDecoder?

You read a PEM file by calling decode with the class you expect back. The typed overload, decode(String, Class<S>), parses the text, Base64-decodes the body, and returns a fully built object of that type.

The four-step ritual from earlier collapses to this:

String pem = Files.readString(Path.of("private-key.pem"));
PrivateKey key = PEMDecoder.of().decode(pem, PrivateKey.class);

Two lines, and the decoder figured out the algorithm from the encoding. No KeyFactory, no hardcoded "RSA", no PKCS8EncodedKeySpec.

Certificates and public keys work the same way. You change the Class argument and that is it:

PEMDecoder decoder = PEMDecoder.of();
 
X509Certificate cert = decoder.decode(certPem, X509Certificate.class);
PublicKey pub = decoder.decode(pubPem, PublicKey.class);
X509CRL crl = decoder.decode(crlPem, X509CRL.class);

The difference between the old way and the new way is stark when you put them next to each other.

There is also a decode(InputStream) overload, which matters because you often read this material straight off the classpath or a socket rather than from a String. If the type does not match what the PEM actually contains, the decoder throws rather than handing you a silently wrong object, which is exactly the behavior you want for security material.

How do you write objects to PEM with PEMEncoder?

You write a key or certificate to PEM by passing it to encodeToString, which returns the full PEM text including the header and footer lines. PEMEncoder is the mirror image of the decoder and handles every BinaryEncodable type.

PEMEncoder encoder = PEMEncoder.of();
 
String certPem = encoder.encodeToString(cert);
String keyPem = encoder.encodeToString(privateKey);
String pubPem = encoder.encodeToString(publicKey);

If you need raw bytes instead of a String, for example to write directly to a file or a network buffer, use encode:

byte[] pemBytes = PEMEncoder.of().encode(cert);
Files.write(Path.of("cert.pem"), pemBytes);

The encoder picks the correct header label for you. A PrivateKey becomes a PRIVATE KEY block, an X509Certificate becomes a CERTIFICATE block, and so on. You are no longer responsible for getting the boundary text right, which was a real source of bugs when other tools refused to parse a key because the label was slightly off.

How do you encrypt a private key in PEM format?

You encrypt a private key by chaining withEncryption(password) before you encode. The encoder wraps the key as an encrypted PKCS#8 structure, so the PEM you get out is an ENCRYPTED PRIVATE KEY block that is safe to store next to your code.

char[] password = "correct-horse-battery-staple".toCharArray();
 
String encryptedPem = PEMEncoder.of()
    .withEncryption(password)
    .encodeToString(privateKey);

The default password-based encryption algorithm is PBEWithHmacSHA256AndAES_128. If your security policy needs a different one, set it through the jdk.epkcs8.defaultAlgorithm security property rather than passing it per call. That keeps the choice in one place instead of scattered through your code.

Reading it back is the symmetric operation. You hand the password to the decoder with withDecryption:

PrivateKey key = PEMDecoder.of()
    .withDecryption(password)
    .decode(encryptedPem, PrivateKey.class);

For more control, you can work with EncryptedPrivateKeyInfo directly. The encoder accepts it as input, and when you decode an encrypted block without a password, you get one back so you can inspect the algorithm before deciding how to handle it:

BinaryEncodable obj = PEMDecoder.of().decode(encryptedPem);
if (obj instanceof EncryptedPrivateKeyInfo info) {
    PrivateKey key = info.getKey(password);
}

This is a real upgrade. Before JDK 27, encrypted PKCS#8 keys were the single biggest reason teams reached for Bouncy Castle. Now it is in the platform.

How do you handle a file with multiple PEM blocks?

You handle multi-block files by decoding to the BinaryEncodable interface instead of a concrete class, then pattern-matching on each result. A chain file or a bundle often holds a private key followed by a certificate followed by a CA chain, and the untyped decode lets you take them one at a time.

BinaryEncodable obj = PEMDecoder.of().decode(pemBlock);
 
switch (obj) {
    case PrivateKey key -> loadKey(key);
    case X509Certificate cert -> addToChain(cert);
    case X509CRL crl -> registerRevocations(crl);
    case PEM pemBlockData -> log.warn("Unrecognized block: {}", pemBlockData.type());
    default -> throw new IllegalStateException("Unexpected PEM content");
}

The PEM case is the interesting one. When the decoder hits a block it does not have a dedicated type for, it does not throw. It gives you a PEM object that exposes the raw pieces: the type() (the label between the dashes), the Base64 content(), and any leadingData() that appeared before the block. That last part matters for real-world files, which frequently carry human-readable comments above the actual PEM.

So you can round-trip even content the JDK does not understand natively, which means the API does not lock you out of custom or newer PEM labels. That is a thoughtful piece of design for a security API that has to live for decades.

What changed between the preview and the final PEM API?

The biggest changes are three renames that landed when JEP 538 finalized the API, so preview code from JDK 25 and 26 needs small edits. If you tried this during preview, the concepts are identical but a few names moved.

The renames:

• The encodable interface was called DEREncodable in the previews. It is now BinaryEncodable.
• The catch-all type was a record named PEMRecord. It is now an ordinary class named PEM, which let the team add constructors that take Base64 byte arrays with proper defensive copying.
• PEMDecoder.withFactory(Provider) became withFactoriesOf(Provider).

The release path explains the polish. The feature previewed as JEP 470 in JDK 25, came back as a second preview (JEP 524) in JDK 26, and is final as JEP 538 in JDK 27. As of late May 2026 it sits at Proposed to Target for 27, with the finalization review closing.

If you are on JDK 25 or 26 and want to try it today, you compile and run with preview features enabled:

javac --release 26 --enable-preview PemDemo.java
java --enable-preview PemDemo

On JDK 27 you drop both flags, because the API is a permanent part of the platform. That is the whole point of the two-preview cadence: the API got two rounds of real feedback before it became something you cannot change.

Should you adopt the Java PEM API right away?

Yes, the moment you are on JDK 27, because the manual approach is not just ugly, it is a security liability. Every hand-rolled parser is a place where you can mismatch an algorithm, mishandle an encrypted key, or accept malformed input you should have rejected. Moving that logic into the platform means it gets the same scrutiny as the rest of java.security.

The thing I keep coming back to is how long we tolerated the old way. PEM parsing was a rite of passage, a snippet everyone carried in their head, and that is exactly the kind of code that should never have been ours to write. If JDK 27 is in your future, delete your PEM helper class the day you upgrade. You will not miss it.

For the full specification and reference, see the JEP 538: PEM Encodings of Cryptographic Objects, the JDK 25 security enhancements writeup by the JDK security lead, and the InfoQ Java News Roundup tracking the finalization.

Keep Reading

• Java 26 HTTP/3 in the HttpClient: the other half of modern Java networking, where the keys and certificates you load with the PEM API actually get used for TLS.
• Full-Stack Social Identity with Spring Security and Next.js: a real auth flow where certificate and key handling shows up in production.
• Modern Java Development with Java 21+ and Spring Boot: more recent language and platform features worth folding into your services.

I hit this on a React dashboard last year. The initial load scored a clean LCP of 1.4s. Real users on the reports page were staring at spinners for 3 seconds, and our RUM tool showed nothing wrong, because in a single-page app the browser only ever fires one navigation. Every route change after that is invisible to the performance timeline.

Chrome's Soft Navigations API fixes this. It teaches the browser to recognize a client-side route change as a real page view, then attributes LCP, INP, and CLS to that specific route. This post walks through what a soft navigation is, how to turn the API on, how to wire it into PerformanceObserver and the web-vitals library, and how to read the per-route numbers. If you have ever wondered why your SPA metrics felt too good, this is the missing piece. It pairs well with the work I did on moving data fetching off useEffect, since slow route data is the usual reason a soft navigation scores badly.

Why are Core Web Vitals invisible in a single-page application?

Core Web Vitals are invisible on SPA route changes because the browser ties every metric to a single navigation event, and a client-side route change never fires one. When your router swaps the view with history.pushState and a DOM update, the browser sees the same document it loaded minutes ago. LCP was already finalized on first paint. CLS keeps summing layout shifts across the whole session. INP is the only one that survives, and even it has no idea which route the slow interaction belonged to.

So you end up with a metric that describes the first thing a user saw and ignores everything after. For a content site with full page loads, that is fine. For a SPA where users spend 90% of their time on routes they reached by clicking, it is close to useless.

The traditional workarounds were all bad. Some teams manually called performance.mark() on every route change and computed their own timings, which misses the actual paint and interaction signals the browser tracks internally. Others gave up and reported only the initial load, accepting the blind spot. Neither gives you the real LCP element or the real interaction latency for a route.

What you actually want is for the browser to reset its Core Web Vitals accounting at each route change, the same way it would on a multi-page site. That is exactly what soft navigations provide.

What is a soft navigation, and how does Chrome detect one?

A soft navigation is a client-side route change that Chrome recognizes as a new logical page view based on a heuristic. The browser cannot read your router's mind, so it watches for a specific pattern of three things happening together.

All three conditions must hold:

1. A user action initiates the navigation. A click or a keypress, not a background timer or a fetch that resolves on its own.
2. The URL changes visibly. Through history.pushState, history.replaceState, or a direct History API call that updates what the user sees in the address bar.
3. The interaction produces a visible paint. New content has to actually render, so a no-op route that changes the URL but paints nothing does not count.

That heuristic is deliberately conservative. Chrome would rather miss a borderline navigation than wrongly split one page view into two and corrupt your numbers. The replaceState trigger was added to the final origin trial after developer feedback, because plenty of routers use replaceState for things like filter changes that users perceive as navigations.

The important detail for instrumentation: once Chrome flags a soft navigation, it stamps a unique navigationId onto the performance entries that follow. That id is the thread you pull on to group LCP, CLS, and interaction entries by route.

How do you turn on the Soft Navigations API in Chrome?

You enable the Soft Navigations API one of two ways depending on whether you want local testing or real field data. For local work, flip a flag. For production measurement, join the origin trial.

For local testing, enable the Chrome flag:

chrome://flags/#soft-navigation-heuristics

Or launch Chrome from the command line with the feature turned on:

# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --enable-features=SoftNavigationHeuristics
 
# Linux
google-chrome --enable-features=SoftNavigationHeuristics

For field data from real users, register your origin in the Soft Navigations origin trial. It runs from Chrome 147 through Chrome 149, with a stable launch expected later in 2026. Once you have a token, add it as a meta tag in your document head:

<meta http-equiv="origin-trial" content="YOUR_ORIGIN_TRIAL_TOKEN" />

Or send it as an HTTP response header, which is the better choice for a SPA because the token applies before your JavaScript runs:

Origin-Trial: YOUR_ORIGIN_TRIAL_TOKEN

Either way, the API surface is identical. The flag is for your own Chrome during development. The token is what lets the feature run for visitors who have not enabled any flags, so your real-user monitoring actually collects data.

How do you observe soft navigations with PerformanceObserver?

You observe soft navigations by registering a PerformanceObserver for the soft-navigation entry type. Each entry that arrives represents one detected route change, carrying the new URL and a navigationId you use to correlate other metrics.

Always feature-detect first so you do not throw on browsers without the API:

function supportsSoftNavigations() {
  return (
    typeof PerformanceObserver !== 'undefined' &&
    PerformanceObserver.supportedEntryTypes.includes('soft-navigation')
  );
}
 
if (supportsSoftNavigations()) {
  const observer = new PerformanceObserver((list) => {
    for (const entry of list.getEntries()) {
      console.log('Soft navigation to', entry.name);
      console.log('navigationId', entry.navigationId);
      console.log('started at', entry.startTime);
    }
  });
 
  observer.observe({ type: 'soft-navigation', buffered: true });
}

The buffered: true option matters. It replays entries that fired before your observer registered, which is common in a SPA where your analytics code loads after the first interaction. Without it you would miss early route changes.

Each soft-navigation entry gives you a few useful fields:

• name is the new URL the navigation resolved to.
• navigationId is the unique key for grouping all metrics from this route.
• startTime is when the initiating interaction happened.
• largestInteractionContentfulPaint points at the largest paint that resulted from the navigation, the soft-nav equivalent of LCP.

That navigationId is the whole game. Every Core Web Vital entry that follows a soft navigation gets the same id, so you can finally answer "how slow was the reports route specifically" instead of "how slow was the session."

How do you attribute LCP, INP, and CLS to each route?

You attribute each metric to a route by reading the navigationId that Chrome now attaches to standard performance entries after a soft navigation. The largest-contentful-paint, layout-shift, and event entries all gain that field, so grouping by it gives you per-route vitals.

Here is a single observer that buckets metrics by navigation:

const metricsByNavigation = new Map();
 
function getBucket(navigationId) {
  if (!metricsByNavigation.has(navigationId)) {
    metricsByNavigation.set(navigationId, {
      url: null,
      lcp: 0,
      cls: 0,
    });
  }
  return metricsByNavigation.get(navigationId);
}
 
const observer = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    const id = entry.navigationId;
    if (id === undefined) continue;
    const bucket = getBucket(id);
 
    if (entry.entryType === 'soft-navigation') {
      bucket.url = entry.name;
    }
    if (entry.entryType === 'largest-contentful-paint') {
      bucket.lcp = entry.startTime;
    }
    if (entry.entryType === 'layout-shift' && !entry.hadRecentInput) {
      bucket.cls += entry.value;
    }
  }
});
 
observer.observe({ type: 'soft-navigation', buffered: true });
observer.observe({ type: 'largest-contentful-paint', buffered: true });
observer.observe({ type: 'layout-shift', buffered: true });

The difference this makes is stark once you compare it to the old single-load view.

INP needs one extra note. Interaction entries map to a soft navigation through the interactionId, and the spec guidance is to use interactionId rather than navigationId when you correlate interaction-contentful-paint entries. For most reporting you will let the web-vitals library handle that wiring, which is the next section.

How do you report per-route vitals with the web-vitals library?

You report per-route vitals by using the experimental soft-navigation build of Google's web-vitals library, which exposes a reportSoftNavs option on each metric function. That option tells the library to emit a fresh metric object for every soft navigation instead of one per page load.

Import the soft-navs build and opt in per metric:

import {
  onLCP,
  onINP,
  onCLS,
} from 'https://unpkg.com/web-vitals@soft-navs/dist/web-vitals.js?module';
 
function sendToAnalytics(metric) {
  // metric.navigationId ties this value to a specific route
  const body = JSON.stringify({
    name: metric.name,
    value: metric.value,
    rating: metric.rating,
    navigationId: metric.navigationId,
  });
  navigator.sendBeacon('/analytics', body);
}
 
onLCP(sendToAnalytics, { reportSoftNavs: true });
onINP(sendToAnalytics, { reportSoftNavs: true });
onCLS(sendToAnalytics, { reportSoftNavs: true });

With reportSoftNavs: true, the callback fires once for the initial load and again for each detected route change, and every metric object carries the navigationId so your backend can group by route. The standard web-vitals v5 build still works for the hard load, but only the soft-navs build resets the metrics per soft navigation.

A few practical notes from wiring this into a real RUM pipeline:

• Use navigator.sendBeacon or a fetch with keepalive: true so the report survives the user leaving the route.
• Send the resolved route pattern, not the raw URL, if your routes have ids in them. Reporting /orders/:id instead of /orders/8412 keeps your dashboard groupable.
• Keep the hard-load report too. The first load is still your most important view and you want both in the same dataset.

If you already report Core Web Vitals from a framework like Next.js, you are swapping the import path and adding the reportSoftNavs flag. The shape of your analytics payload barely changes.

How do you read soft navigation data in Chrome DevTools?

You read soft navigation data directly in the Performance panel, which has shown soft-nav markers since Chrome 145. Record a trace, click through a few client-side routes, and DevTools draws a marker at each detected soft navigation so you can see exactly where the browser reset its metrics.

Source: Chrome for Developers

The markers are the fastest way to confirm the heuristic is firing on your routes before you invest in a full RUM integration. If you click a link and no marker appears, one of the three conditions failed. Usually it is the visible-paint requirement, which trips when a route renders from cache so fast that Chrome does not register a contentful paint, or when the URL changed without a user interaction the browser could attribute.

Record the trace, watch for the markers, and line them up against the LCP and layout-shift entries in the same timeline. That visual check has saved me from shipping instrumentation that silently recorded nothing on half my routes.

What are the limits of the Soft Navigations API?

The biggest limit is that this is a heuristic running only in Chromium during an origin trial, so it is a sample and not a complete picture. You need to design your reporting around that from day one rather than treating soft-nav data as ground truth for every user.

The constraints worth planning around:

• Chromium only. Firefox and Safari have no equivalent. Your soft-nav numbers describe Chrome users. Keep your hard-load metrics as the cross-browser baseline.
• It is a heuristic. Routes that paint from cache instantly, or navigations not tied to a clear user interaction, can be missed. The detection favors precision over recall, so expect some false negatives rather than false positives.
• Origin trial status. The API ran as an origin trial through Chrome 149 with a stable launch targeted for later in 2026. Field tokens expire, and the surface could still shift slightly before it ships, though the team froze most of it for this final trial.
• CrUX is undecided. Google has been explicit that this trial is about evaluating the API, not about how the data feeds the Chrome User Experience Report. So do not assume soft-nav vitals will show up in your CrUX dashboard or affect search signals yet.

None of that makes the API less worth adopting. A Chromium-only sample of your real per-route performance is infinitely more than the zero data you have today. Just label it honestly in your dashboards so nobody mistakes a Chrome sample for the whole population.

Is the Soft Navigations API worth adopting now?

Yes, even as an origin trial, because a Chromium-only sample of real per-route performance beats the zero data you have today. The reason this matters goes beyond a nicer dashboard. For years, SPA teams optimized the one route the browser could measure and flew blind on everything else, which quietly trained a generation of apps to be fast on the homepage and slow everywhere that mattered. Per-route Core Web Vitals change the incentive. Once you can see that the settings page has a 4-second LCP, you fix the settings page.

My advice: turn on the flag this week and record a Performance trace of your own app clicking through its three most-used routes. The markers will tell you in 30 seconds whether your performance story is as good as your current metrics claim. I would bet it is not, and that is exactly the point.

For the full specification and reference, see the Soft Navigations API documentation, the final origin trial announcement, and the web-vitals library on GitHub.

Keep Reading

• Why I Replaced useEffect Data Fetching with Server Actions: slow route-change data is the most common reason a soft navigation scores a bad LCP, so this is the upstream fix.
• Building a Modern Documentation Generator with Next.js 16: a practical look at the kind of client-side routing that produces soft navigations in the first place.
• How to Upgrade Next.js for the May 2026 Security Release: keeping your framework current is what gives you access to the newest performance APIs like this one.

I ran the upgrade on a Micronaut 4.7 service last week. The BOM bump is one line. Everything after that is real work: Java 25, Jackson 3, JSpecify imports, a renamed security processor, RxJava 2 gone, MicroStream replaced, and a new programmatic retry API that the old annotation flow quietly redirects to. This guide walks the path I took, with the exact Gradle and Maven snippets that compiled cleanly on my service.

What changed between Micronaut 4 and Micronaut 5?

Micronaut 5 is a platform-wide refresh across 70+ modules, not just a framework bump. The headline changes that affect real services are: JDK 25 baseline (was JDK 17), Kotlin 2.3, Groovy 5, a refactored IoC container with tighter qualifier semantics, JSpecify nullability replacing the older annotation soup, Jackson 3 in micronaut-jackson-databind, HTTP/3 promoted to stable on the Netty stack, and a new programmatic retry and circuit breaker API alongside the existing annotation model.

On the deprecation side, Bootstrap Configuration is marked for removal in Micronaut 6, RxJava 2 is fully dropped (you go to RxJava 3), and MicroStream support is replaced by EclipseStore. If you used micronaut-eclipsestore-annotations, that artifact is renamed too.

The piece I underestimated was the annotation processor rename. Security in particular silently broke my build until I swapped the artifact ID. More on that below.

How do you install JDK 25 for the migration?

Use SDKMAN. It is the cleanest way to flip JDK versions per project without touching system PATH.

sdk install java 25-tem
sdk default java 25-tem
 
# Verify
java -version
# openjdk version "25" 2025-09-16

Add a .sdkmanrc to your project root so the JDK pins automatically when you cd into it:

java=25-tem

If you run on a Docker base image, switch your runtime stage to eclipse-temurin:25-jre or bellsoft/liberica-openjdk-alpine:25. I ran a quick perf check after the JDK bump alone, before touching Micronaut: startup dropped roughly 8% on my service from JDK 21 to JDK 25, before any framework gains. Your numbers will vary, but it is a real improvement.

For more on what is coming after JDK 25, see my deep-dive on Java 26 Structured Concurrency.

How do you update Gradle builds for Micronaut 5?

Bump three coordinates: the Micronaut platform BOM, the Micronaut Gradle plugin, and Gradle itself. Then add the Kotlin and Shadow plugin updates if you use them.

Before (Micronaut 4.7.x):

plugins {
  id("io.micronaut.application") version "4.4.4"
  id("com.gradleup.shadow") version "8.3.5"
  id("org.jetbrains.kotlin.jvm") version "2.0.21"
  id("org.jetbrains.kotlin.plugin.allopen") version "2.0.21"
  id("com.google.devtools.ksp") version "2.0.21-1.0.27"
}
 
micronaut {
  version("4.7.6")
  runtime("netty")
  testRuntime("junit5")
}
 
java {
  sourceCompatibility = JavaVersion.VERSION_17
  targetCompatibility = JavaVersion.VERSION_17
}

After (Micronaut 5.0.0):

plugins {
  id("io.micronaut.application") version "5.0.0"
  id("com.gradleup.shadow") version "9.4.1"
  id("org.jetbrains.kotlin.jvm") version "2.3.21"
  id("org.jetbrains.kotlin.plugin.allopen") version "2.3.21"
  id("com.google.devtools.ksp") version "2.3.7"
}
 
micronaut {
  version("5.0.0")
  runtime("netty")
  testRuntime("junit5")
}
 
java {
  sourceCompatibility = JavaVersion.VERSION_25
  targetCompatibility = JavaVersion.VERSION_25
}

Then run ./gradlew wrapper --gradle-version=9.5.0 --distribution-type=bin once to upgrade the wrapper. Commit the changed wrapper files in the same PR as the Micronaut bump so CI runs against the matching Gradle.

If you read the version catalog instead of inline versions, you only update gradle/libs.versions.toml:

[versions]
micronaut = "5.0.0"
micronaut-plugin = "5.0.0"
kotlin = "2.3.21"
ksp = "2.3.7"
shadow = "9.4.1"

How do you update Maven builds for Micronaut 5?

For Maven, swap the micronaut-parent coordinate and set the Java release properties to 25. Nothing else in the POM should change for a clean Micronaut 4 → 5 upgrade.

Before:

<parent>
  <groupId>io.micronaut.platform</groupId>
  <artifactId>micronaut-parent</artifactId>
  <version>4.7.6</version>
</parent>
 
<properties>
  <jdk.version>17</jdk.version>
  <release.version>17</release.version>
</properties>

After:

<parent>
  <groupId>io.micronaut.platform</groupId>
  <artifactId>micronaut-parent</artifactId>
  <version>5.0.0</version>
</parent>
 
<properties>
  <jdk.version>25</jdk.version>
  <release.version>25</release.version>
</properties>

Make sure your Maven version is 3.9 or later. If you use the Micronaut Maven plugin for native-image builds, it pulls the matching version from the parent BOM, so you do not need to pin it separately.

How do you adopt JSpecify nullability annotations?

Micronaut 5 standardizes on JSpecify for nullability. If your codebase mixes org.jetbrains.annotations.Nullable, javax.annotation.Nullable, and io.micronaut.core.annotation.Nullable, this is the cleanup moment.

The replacement is consistent: any package-level, class-level, or method-level nullability comes from org.jspecify.annotations.

// Before (mixed across the codebase)
import org.jetbrains.annotations.Nullable;
import io.micronaut.core.annotation.NonNull;
 
public Optional<User> findUser(@NonNull String id, @Nullable String tenant) { ... }
 
// After
import org.jspecify.annotations.NonNull;
import org.jspecify.annotations.Nullable;
 
public Optional<User> findUser(@NonNull String id, @Nullable String tenant) { ... }

For broader scope, mark whole packages as non-null by default with a package-info.java:

@NullMarked
package com.example.users;
 
import org.jspecify.annotations.NullMarked;

That single annotation says every reference in the package is non-null unless explicitly marked @Nullable. It cuts annotation noise and gives Kotlin call sites cleaner platform types. Add JSpecify as a direct dependency only if you compile against it outside Micronaut classes; Micronaut transitively brings it in.

IntelliJ has a built-in inspection called "Migrate nullability annotations". Set the target to JSpecify and run it on your src/main tree. Review every change, then commit. Do not let the inspection touch generated code.

Why does Jackson 3 break existing serialization?

micronaut-jackson-databind now uses Jackson 3 exclusively, and Jackson 3 has real wire-compatible differences from Jackson 2. The two ones that bit me:

First, the package root changed. Custom JsonSerializer and JsonDeserializer classes that import com.fasterxml.jackson.databind.* need to move to tools.jackson.databind.*. The class names are the same, the package is different.

// Before (Jackson 2)
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
 
// After (Jackson 3)
import tools.jackson.databind.JsonSerializer;
import tools.jackson.databind.SerializerProvider;

Second, Jackson 3 fails fast on unknown properties by default. If you relied on lenient deserialization in production, set the right deserialization feature on your ObjectMapper at startup, or annotate the DTO:

@JsonIgnoreProperties(ignoreUnknown = true)
public record OrderEvent(String id, String status, Instant createdAt) { }

I had three DTO classes that were silently dropping unknown fields in Micronaut 4 and started throwing UnrecognizedPropertyException in Micronaut 5. Run integration tests against real payloads before merging.

If you prefer to stay on the Micronaut serialization stack (micronaut-serde-jackson), this section does not apply. Serde is unchanged across the upgrade.

What changed in the security annotation processor?

The annotation processor artifact ID was renamed for several modules in Micronaut 5. The most important one is micronaut-security-annotations, which became micronaut-security-processor.

The compiler error is misleading. You get a stack of "cannot find symbol" or "annotation processor failed to load" messages that point at your own code, not at the missing processor.

<!-- Before (Maven, annotationProcessorPaths) -->
<path>
  <groupId>io.micronaut.security</groupId>
  <artifactId>micronaut-security-annotations</artifactId>
</path>
 
<!-- After -->
<path>
  <groupId>io.micronaut.security</groupId>
  <artifactId>micronaut-security-processor</artifactId>
</path>

// Before (Gradle)
annotationProcessor("io.micronaut.security:micronaut-security-annotations")
 
// After
annotationProcessor("io.micronaut.security:micronaut-security-processor")

The same rename pattern applies to micronaut-eclipsestore-annotations, which becomes micronaut-eclipsestore-processor. If you use Views Turbo, the dependency moved out of the main views module: add micronaut-views-turbo, and rename @TurboView to @TurboStreamView at every usage.

How do the new programmatic retry and circuit breaker APIs work?

Micronaut 4 only let you opt in to retry and circuit breakers through annotations like @Retryable and @CircuitBreaker. Micronaut 5 adds a programmatic, typed API in io.micronaut.retry that you can call without an annotation, which is what you want when the retry policy depends on runtime config.

import io.micronaut.retry.RetryPolicy;
import io.micronaut.retry.RetryState;
import jakarta.inject.Singleton;
 
import java.time.Duration;
 
@Singleton
public class PaymentClient {
 
  private final RetryPolicy retryPolicy = RetryPolicy.builder()
    .maxAttempts(5)
    .delay(Duration.ofMillis(200))
    .multiplier(2.0)
    .includes(IOException.class)
    .build();
 
  public PaymentResult charge(ChargeRequest req) {
    return retryPolicy.execute(() -> gateway.charge(req));
  }
}

The circuit breaker variant uses the same builder shape with circuitBreaker(true), an open-state duration, and a half-open probe count. It lets you wire policy from application.yml:

@ConfigurationProperties("payments.retry")
public record PaymentsRetryConfig(int maxAttempts, Duration delay, double multiplier) { }

I switched two services from annotation-only to programmatic retry during the upgrade. The win is testability: you inject a stub RetryPolicy in unit tests instead of waiting on real backoff timers.

The old annotations still work. You only need the programmatic API where the policy is dynamic. Mixing both in the same service is fine.

What other breaking changes should you plan for?

A few smaller items can stall the build if you do not catch them up front.

• RxJava 2 is dropped. Add micronaut-rxjava3 and migrate io.reactivex.* imports to io.reactivex.rxjava3.*. If you only used RxJava 2 for HTTP client return types, switching to Mono and Flux via Reactor is the cleaner long-term move.
• MicroStream is replaced by EclipseStore. Same APIs in most places, but the artifact and package roots moved to org.eclipse.store. If you use the Micronaut integration, depend on the eclipsestore modules.
• Testcontainers wiring changed. Use org.testcontainers:testcontainers-junit-jupiter for JUnit 5 integration, not the bare junit-jupiter artifact. The old coordinate compiles but produces a NoClassDefFoundError at test time.
• Data embedded fields need an annotation. Annotate embedded fields with @MappedProperty so the column names stay stable. If you want the old behavior, set micronaut.data.embedded.naming.strategy=LEGACY in application.yml.
• Bootstrap Configuration deprecated. Still works in Micronaut 5, scheduled for removal in Micronaut 6. If you load critical secrets through bootstrap, plan the migration to PropertySourceImporter SPI.

After fixing these, a clean rebuild should pass. If your test suite still fails on dependency injection, regenerate the IDE project files. IntelliJ caches the old annotation-processor outputs and will mislead you for an hour before you give up and reimport.

What should you do after the Micronaut 5 upgrade is live?

Once the service compiles and tests pass, do four things before merging:

1. Run an integration test pass against real upstream payloads, especially anything that hits Jackson. The strict unknown-property default will catch dirty data you forgot you were ignoring.
2. Re-baseline startup time and memory in production. The IoC refactor and JDK 25 combined cut my startup by ~14% on a six-controller service. You want a number you can show to your platform team.
3. Audit your @Replaces, @Requires, and qualifier annotations. The compile-time semantics tightened, and a couple of ambiguous-bean errors that Micronaut 4 silently resolved by ordering now throw at startup. The error message names both candidates, which is the fix.
4. Update your CI base images to JDK 25. I forgot this for one repo and got a green local build with a broken mvn deploy job. Pin the image, not just the toolchain.

The platform refresh is the biggest Micronaut release since 4.0 in 2023, and it sets the floor for what a Java microservice baseline looks like for the next two years.

For more on the Micronaut 5 release, see the official Micronaut 5.0.0 announcement, the Update to Micronaut 5 guide, and the Java News Roundup May 18, 2026 on InfoQ.

Keep Reading

• Modern Java + Spring Boot. Compare the Micronaut upgrade path with how the same JDK 25 baseline reshapes a Spring Boot service.
• Java 26 Structured Concurrency: What Changed in the Sixth Preview. The next JDK feature your Micronaut 5 service can adopt once the upgrade is in.
• Java Libraries Beyond Lombok. Useful JVM libraries that pair well with a JDK 25 + Micronaut 5 stack.

This post walks through the full setup in a Spring Boot 4 project: how to override the Hibernate version to get 7.4, how to wire the @Changelog entity that supplies changeset ids, how to annotate an entity to start auditing, and how to query history with the new AuditLog interface. The feature is incubating, meaning the API can shift before the final release, but the surface is small enough that the migration cost is bounded if it does.

If you already use Envers, the migration path is gradual. You can keep auditing existing entities with Envers and adopt the new annotation only on new ones until the dust settles.

What is the @Audited annotation in Hibernate 7.4?

The new @Audited annotation marks an entity class (or a collection field) as audited, meaning Hibernate keeps a historical record of every change to it. It lives in org.hibernate.annotations.Audited and was added in Hibernate 7.4 as an @Incubating feature, shipped first in 7.4.0.CR1 on May 7, 2026.

When you annotate an entity with @Audited, Hibernate creates two tables instead of one. The primary table holds the current state, exactly like before. A separate audit log table holds a row for every change. Each audit row contains:

• The full state of the entity at the moment of the change (minus any fields you mark with @Excluded).
• A changesetId column that groups changes into atomic batches.
• A modificationType column encoded as 0 for creation, 1 for modification, and 2 for deletion.

The changesetId is supplied by one of three mechanisms, in order of preference: a @Changelog entity in your domain model (the recommended path), a custom ChangesetIdentifierSupplier registered via configuration, or, as a last resort, Instant.now(). The Hibernate javadoc itself flags relying on the Instant.now() fallback as not recommended.

You query history three ways. You open a session with SessionBuilder.atChangeset(id) to transparently read entity state as of that changeset (regular HQL queries just work). You use the AuditLog interface for programmatic access to revision history and cross-entity queries. Or you open a session with AuditLog.ALL_CHANGESETS and write custom HQL using the new changesetId() and modificationType() functions.

This is the entire model. No revision-info entity. No revision listener. No conditional auditing strategy. Just an annotation, a changelog entity, and a query interface.

How does Hibernate 7.4 @Audited differ from Envers?

The Envers module (introduced in Hibernate 3.6 over a decade ago) lives in org.hibernate.envers and ships as a separate jar. It uses a revision-tracking model with RevisionEntity and @RevisionNumber, writes audit tables prefixed with _AUD, and queries through AuditReader. It works and is mature, but it is also crusty: configuration is XML-shaped, the API surface is wide, and getting current-user-in-audit-trail right always involves a custom listener.

The new annotation rebuilds the same idea on a smaller surface. Key differences:

• Package: org.hibernate.annotations.Audited (new core annotation) versus org.hibernate.envers.Audited (legacy Envers annotation). Both annotations are spelled @Audited, so the import you choose decides which engine handles the entity.
• Module: the new annotation is in hibernate-core. No extra dependency needed if you already have Hibernate 7.4 on your classpath. Envers ships in hibernate-envers and must be added separately.
• Changeset metadata: Envers uses a RevisionEntity with @RevisionNumber and @RevisionTimestamp. The new annotation uses a @Changelog entity with @ChangesetId and @Timestamp. The shape is similar but the field annotations are first-class on the new side.
• Audit table layout: Envers writes a _AUD suffixed table per entity with revision linkage. The new model writes a parallel audit table with changesetId plus modificationType columns directly. Different layout, different query SQL.
• Querying: Envers exposes AuditReader retrievable from the EntityManager. The new model exposes AuditLog through AuditLogFactory.create() and supports transparent point-in-time reads via SessionBuilder.atChangeset().
• Maturity: Envers is GA, battle-tested, used in production for a decade. The new annotation is incubating, so the API can shift between 7.4 and 7.5.

The Hibernate team has stated the 7.4 release is backward compatible with Envers, meaning your existing Envers-audited entities keep working unchanged when you upgrade. You can layer the new @Audited annotation on new entities while leaving the old ones on Envers, then migrate over time.

How do you set up the audit feature in a Spring Boot project?

You override the Hibernate version managed by Spring Boot, because Spring Boot 4.0 ships Hibernate 7.0 and Spring Boot 4.1 has not been released yet. The override is a single property in your build file.

For Maven (pom.xml):

<properties>
  <java.version>24</java.version>
  <hibernate.version>7.4.0.CR1</hibernate.version>
</properties>

For Gradle (build.gradle.kts):

ext["hibernate.version"] = "7.4.0.CR1"
 
dependencies {
    implementation("org.springframework.boot:spring-boot-starter-data-jpa")
}

Spring Boot picks up the override and Hibernate Core, Hibernate JPA, and Hibernate Annotations all align on 7.4.0.CR1.

If you already had Envers on the classpath, leave it. The new annotation is in core. You can keep both jars side by side and import explicitly. If you intentionally do not want Envers anymore, exclude it once you have migrated every entity.

Verify the version on application startup. Hibernate logs it at INFO level on the bootstrap line:

INFO  o.h.Version - HHH000412: Hibernate ORM core version 7.4.0.CR1

There is no extra configuration required for the audit feature to activate. The presence of a @Changelog entity and any @Audited entity is enough.

How do you define a @Changelog entity for changeset ids?

Add a single class to your domain model annotated with @Changelog and @Entity. It must declare an @Id field with @ChangesetId and a timestamp field with @Timestamp. Hibernate handles the rest, persisting one instance per transaction and using its primary key as the changeset id for every audited change in that transaction.

A minimal version:

package com.example.audit;
 
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import org.hibernate.annotations.Changelog;
import org.hibernate.annotations.ChangesetId;
import org.hibernate.annotations.CreationTimestamp;
import org.hibernate.annotations.Timestamp;
 
@Entity
@Changelog
@Table(name = "changeset")
public class Changeset {
 
    @Id
    @ChangesetId
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
 
    @Timestamp
    @CreationTimestamp
    private Instant createdAt;
 
    public Long getId() { return id; }
    public Instant getCreatedAt() { return createdAt; }
}

@CreationTimestamp populates createdAt automatically on insert, which the @Changelog Javadoc explicitly cites as the recommended way to initialize the timestamp field. The Instant type maps cleanly to a PostgreSQL timestamp column.

Once this class is on the classpath, Hibernate auto-registers it as the changeset id supplier. You do not need to set org.hibernate.cfg.StateManagementSettings.CHANGESET_ID_SUPPLIER manually. Only one entity per application may be annotated with @Changelog, so this is your single source of truth for changeset metadata.

To enrich the audit trail with the current user and an optional comment, add fields and a @ChangesetListener. The listener fires on every changeset insertion and gives you a hook to populate the user from your security context:

package com.example.audit;
 
import org.hibernate.audit.ChangesetListener;
import org.springframework.security.core.context.SecurityContextHolder;
 
public class CurrentUserChangesetListener implements ChangesetListener {
 
    @Override
    public void prePersist(Object changeset) {
        if (changeset instanceof Changeset c) {
            var auth = SecurityContextHolder.getContext().getAuthentication();
            c.setCreatedBy(auth != null ? auth.getName() : "system");
        }
    }
}

You register the listener once during SessionFactory build via the Hibernate Integrator SPI, the same way you would have wired any other Hibernate listener.

How do you annotate an entity to start auditing it?

Annotate the entity class with @Audited. That is the entire setup. Hibernate generates the audit log table on schema export, populates it on every change, and you do not have to write a single line of audit-specific code anywhere else.

A complete example for a Customer aggregate:

package com.example.customer;
 
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import org.hibernate.annotations.Audited;
import org.hibernate.annotations.Excluded;
 
@Entity
@Audited
@Table(name = "customer")
public class Customer {
 
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
 
    @Column(nullable = false)
    private String name;
 
    @Column(nullable = false, unique = true)
    private String email;
 
    @Excluded
    @Column(nullable = false)
    private String passwordHash;
 
    // getters and setters
}

The @Excluded annotation on passwordHash keeps that column out of the audit log, which is what you want for password hashes, PII you do not need to track historically, or large binary blobs. Everything else is recorded on every change.

When Hibernate exports the schema, it creates two tables: the regular customer table and a parallel audit log table. Hibernate names the audit table conventionally, similar to customer_aud by default, and the @Audited annotation accepts custom table name overrides if you need them. The audit table contains the same state columns minus excluded ones, plus a changeset_id column and a modification_type column.

A typical row sequence after a create-update-delete cycle on a single customer:

SELECT * FROM customer_aud WHERE id = 42 ORDER BY changeset_id;
 
 id | name      | email          | changeset_id | modification_type
----+-----------+----------------+--------------+-------------------
 42 | Alice Doe | alice@ex.com   |            1 | 0
 42 | Alice Doe | alice@xyz.com  |            2 | 1
 42 | Alice Doe | alice@xyz.com  |            3 | 2

The 0/1/2 encoding is from the source javadoc directly. You can decode it in queries using the new modificationType() HQL function.

How do you query audit history with the AuditLog interface?

Obtain an AuditLog instance from AuditLogFactory.create(), run your history queries, and close it when done. The AuditLog interface is AutoCloseable and manages its own internal session, so a try-with-resources block is the cleanest pattern.

A repository method that returns the full revision history of a customer:

package com.example.customer;
 
import java.util.List;
import org.hibernate.audit.AuditLog;
import org.hibernate.audit.AuditLogFactory;
import org.springframework.stereotype.Repository;
 
@Repository
public class CustomerAuditRepository {
 
    private final AuditLogFactory factory;
 
    public CustomerAuditRepository(AuditLogFactory factory) {
        this.factory = factory;
    }
 
    public List<Customer> getHistory(Long customerId) {
        try (AuditLog auditLog = factory.create()) {
            return auditLog.getHistory(Customer.class, customerId);
        }
    }
}

The getHistory(Class, Object) method returns every revision of the entity in chronological order, including the final deletion row if the entity has been deleted. Each returned instance is a snapshot of the entity at that changeset, not a live managed object.

For more specific queries (e.g., who deleted what last quarter), open a session with the AuditLog.ALL_CHANGESETS magic value and write custom HQL using the new functions:

try (var session = sessionFactory.withOptions()
        .atChangeset(AuditLog.ALL_CHANGESETS).open()) {
 
    var deletions = session.createSelectionQuery("""
            select c.id, c.email, changesetId(c), changeset.createdBy
            from Customer c
            join Changeset changeset on changesetId(c) = changeset.id
            where modificationType(c) = 2
              and changeset.createdAt > :since
            """, Object[].class)
        .setParameter("since", lastQuarterStart)
        .getResultList();
}

changesetId(c) returns the changeset of the audit row. modificationType(c) returns the 0/1/2 code. Joining against the Changeset entity lets you correlate with whatever metadata you put on your changelog (user, comment, request id).

How do you read entity state at a past changeset?

Open a session with SessionBuilder.atChangeset(changesetId) and run regular HQL queries. Every entity load, every query, every association traversal returns the state as of that changeset, not the current state.

public Customer customerAtChangeset(Long customerId, Long changesetId) {
    try (var session = sessionFactory.withOptions()
            .atChangeset(changesetId).open()) {
 
        return session.createSelectionQuery(
                "from Customer where id = :id",
                Customer.class)
            .setParameter("id", customerId)
            .uniqueResult();
    }
}

This is the killer feature versus Envers, which required different APIs to query past state. Here the same HQL works whether the session is rooted at the current state or a historical one, because the session itself carries the temporal anchor.

Use cases I have already mapped to this:

• Audit trail UI: render a customer detail page as it looked on a specific date by passing the changeset id from a date picker.
• Debug session reconstruction: when a customer reports a bug from yesterday, open a session at yesterday's last changeset and run the user's flow against the historical data.
• Compliance exports: for GDPR Article 30 records of processing, you can prove exactly what data existed when, without parsing audit log tables yourself.

One important caveat from the source: atChangeset() sessions are read-only. Attempting a flush or commit raises an exception. This is by design (you cannot retroactively edit history), but it means you cannot use the same session for both historical reads and current writes.

What should you do this week?

I would not migrate production audit code to 7.4 yet. It is @Incubating, the API can shift before final release, and Envers is still production-grade. But I would:

1. Spin up a 7.4 sandbox project and try the annotation on one entity. The full setup above takes about thirty minutes.
2. File feedback on anything that feels off (the Hibernate team is active on the issue tracker and incubating-annotation feedback is the whole point of the CR phase).
3. Plan the migration path for after the 7.4 final release. The Envers-to-new-annotation walk is straightforward because both can coexist.

If you start a new Spring Boot 4 project from scratch right now and want auditing, the choice is harder. Envers is mature but verbose. The new annotation is clean but unstable. I would still pick Envers today for production, the new annotation for greenfield experiments.

Either way, the days of writing your own audit-trail listeners and updating-by-hand audit columns should be behind you. Hibernate has caught up to what every team eventually builds by hand.

For more on the new feature, see the Hibernate 7.4 What's New page, the Hibernate Audited.java source on GitHub, and the official Envers documentation for the legacy module.

Keep Reading

• Spring Boot 4 AOT Data Repositories: The Underrated Feature. Companion piece on Spring Boot 4 data layer improvements that pair well with the new audit setup.
• PostgreSQL 18 Temporal Foreign Keys with Spring Boot JPA. Database-side temporal data. Complements the application-side audit log with row-level effective dating.
• Hibernate Lazy Initialization Guide. Same kind of Hibernate-runtime detail post if you want more of this depth.

I want to be upfront about where I sit. I work at Amazon and I have contributed to Kiro Crew. Everything below comes from the public release: the announcement post, the product page, and the Apache 2.0 repository. No internal detail, no roadmap talk. Just the setup path and the parts I think are worth slowing down on.

This is a Kiro Crew setup guide, not a review. By the end you will have a gateway running, a service that survives a reboot, and enough understanding of the sandbox to decide what you are comfortable letting it do while you sleep.

What is Kiro Crew and how is it different from a coding agent?

Kiro Crew is a persistent workspace that coordinates agents across sessions, where a normal coding agent lives and dies inside one conversation. That difference sounds small and it is not. A single-session agent forgets everything when the process exits, which means anything longer than one sitting gets restarted by hand. Crew keeps memory, schedules, approvals, and in-flight task state in a local gateway, so a job started Tuesday afternoon can still be making progress Wednesday morning.

It started inside Amazon as a side project called MeshClaw, built by three engineers who wanted to kick off work and walk away. AWS says it reached more than 39,000 Amazon builders and roughly 500 contributors in under six months of internal use before the public release.

The architecture is three pieces:

Two open standards do the heavy lifting. Agent Client Protocol (ACP) is how the gateway drives and observes the agent, and Model Context Protocol (MCP) is how tools get attached. If you have read my take on why the MCP spec went stateless, the shape here will feel familiar.

What do you need before running Kiro Crew setup?

You need three things: Python 3.10 or higher, an authenticated kiro-cli, and hardware you are willing to leave running. Node.js 20 or 22+ only matters if you build the dashboard from source, so if you install a prebuilt wheel or the desktop app you can skip it entirely.

The one people miss is kiro-cli. It is the agent backend, not an optional extra, and Crew cannot do anything without it:

kiro-cli login

Do this before anything else. Skipping it produces an AcpTimeoutError: ACP prompt timed out later, which reads like a network problem and is not.

Memory search needs no setup. On first start, the in-process llama-cpp-python runtime pulls the Qwen3-Embedding-0.6B model, about 610 MB. Budget the disk and the first-run wait. Search comes back empty until that download lands, which is expected rather than broken.

One billing note that matters more for unattended work than interactive work. The software is free under Apache 2.0 and runs on your own machine, but agent inference still needs a Kiro account, and model requests from scheduled or background tasks count against your Kiro plan exactly like IDE usage. Steps that only run scripts, with no model call, cost nothing. A cron job that wakes an agent every fifteen minutes is a billing decision, not just a config line.

How do you install Kiro Crew?

Pick the install method that matches how long you intend to keep it running. There are five, and they are not interchangeable.

One-line install. Fastest path, and the right default:

curl -fsSL https://download.crew.kiro.dev/cli.sh | sh

That script pins to the stable channel. You can target others:

curl -fsSL https://download.crew.kiro.dev/cli.sh | sh -s -- --channel insider
curl -fsSL https://download.crew.kiro.dev/cli.sh | sh -s -- --version 0.1.0

Worth saying plainly: piping a remote script into a shell executes whatever that URL serves. It is the documented install path and the domain is AWS-operated, but if you are on a machine where that tradeoff is not acceptable, use the wheel or source path instead.

Docker, for a server you do not want to babysit:

docker run -d --name kirocrew \
  -p 127.0.0.1:5476:5476 \
  -v kirocrew-home:/home/kirocrew \
  ghcr.io/kirodotdev/kirocrew:stable

Note the 127.0.0.1: prefix on the port binding. Dropping it exposes the dashboard on every interface, which you do not want.

From source, if you plan to contribute or read the code:

git clone https://github.com/kirodotdev/KiroCrew.git
cd KiroCrew
make build
source .venv/bin/activate

Self-contained wheel, when you want a clean install without the Node toolchain on the target box:

make wheel
pip install dist/*.whl

Desktop bundle, which embeds a Python interpreter so end users need neither Python nor npm:

make desktop

That drops a signed .dmg on macOS or an .AppImage on Linux into website/electron/dist/.

How do you verify the setup and start the gateway?

Run the wizard, then doctor, then the gateway, in that order:

kirocrew setup
kirocrew doctor
kirocrew gateway

kirocrew setup is interactive and covers agent config, workspace directory, optional Slack credentials, slash-command name, timezone, dashboard URL, the Playwright MCP server, and the macOS desktop app. Answering "n" to Slack turns off that integration and leaves the dashboard fully working, so there is no reason to wire up chat on the first pass.

Three flags narrow what it touches:

• --agent-only installs agent config and skips every credential prompt
• --clean rebuilds agent config from defaults and ignores what is already there
• --electron-only installs just the macOS desktop app

The --agent-only --clean combination is the fix for a stale MCP config. If the agent starts timing out after you have changed tools around, rebuild before you start debugging anything else.

kirocrew doctor is the command to reach for whenever something is off. It reports platform details, data home, agent config, config values, MCP servers, runtime environment, vector memory status, speech-to-text, Slack, and connectivity. Three failures cover most of what you will hit:

kirocrew gateway starts the server on http://localhost:5476. It binds to loopback unless you deliberately configure otherwise, which is the correct default and worth leaving alone.

Configuration lives at ~/.kiro/crew/config.json:

{
  "agent": {
    "provider": "acp",
    "approval_mode": "interactive",
    "sandbox": "auto"
  },
  "session": {
    "timeout_secs": 1800,
    "pool_size": 2
  },
  "dashboard": {
    "bot_name": "Kiro Crew"
  }
}

Edit it through kirocrew config get, set, and edit rather than by hand, so validation runs. A few environment variables override the basics: KIROCREW_HOME for the data directory, KIROCREW_PORT for the dashboard port, and KIROCREW_EMBED_MODEL_URL if you need to point the embedding download at a mirror.

How do you keep Kiro Crew running when you close your laptop?

Install it as a system service, because a foreground kirocrew gateway dies with your terminal and takes the point of the tool with it:

kirocrew service install
kirocrew service status

On Linux this writes /etc/systemd/system/kirocrew.service and prompts for sudo. The gateway itself still runs as your user, never as root, which is the behavior you want. On macOS it writes a launchd plist instead.

Linux users on Ubuntu 23.10 or newer should expect one specific failure. Crew isolates the agent from your credentials using user and mount namespaces, and recent Ubuntu restricts unprivileged user namespaces through AppArmor. The error looks like this:

sandbox: unshare(NEWNS) failed: errno 1

kirocrew service install adds the required kirocrew-userns AppArmor profile automatically. If you are running outside systemd, invoke it yourself:

aa-exec -p kirocrew-userns -- kirocrew gateway

The installer skips the profile on distributions where it does not apply, so Debian, Arch, RHEL, and Amazon Linux users will not see this at all.

One thing that surprised me the first time: uninstalling does not remove ~/.kiro/crew. Config, credentials, memory, and sessions all survive, and no uninstall path deletes them. That is a sensible default, but it means a "clean reinstall" is not clean unless you move the directory yourself. Back it up before you touch it.

What does the security model actually block?

Kiro Crew ships seven layers of defense, and the ones that matter for unattended work are the sandbox, the deny patterns, and the audit trail. The full set is OS sandboxing, denied command patterns, bash blocking, input and output validation, sensitive path protection, credential redaction, and signed audit logs.

Concretely, here is what you get:

• The dashboard binds to 127.0.0.1 unless you configure otherwise
• 137 deny patterns block destructive and exfiltration-shaped commands
• Sensitive path guards keep the agent out of protected directories
• Credential redaction strips secret-shaped environment variables and patterns from output
• Security events append to ~/.kiro/crew/security_events.jsonl
• Linux uses namespaces, macOS uses Seatbelt, with standard, strict, and off modes

Optional policy files compose tightest-wins, and enterprise settings enforce caps that runtime config cannot loosen. That ordering is the right way round, and it is the detail I would check first if you are deploying this anywhere shared.

Read security_events.jsonl after your first unattended run. Not because you expect trouble, but because it is the fastest way to learn what your agent actually reached for when nobody was watching. The gap between what I assumed a task would touch and what it tried to touch was the most useful thing I learned in my first week.

The Windows situation deserves a direct warning rather than a footnote.

On Windows, Kiro Crew has no OS-level sandboxing. Because of that it refuses to spawn subprocesses by default, and the only way to change that is opting in through sandbox_allow_unsandboxed_exec. Turning that on means an unattended agent runs commands on your machine with no OS isolation between it and everything else. There is also no Windows desktop app yet, so you install from source. If you want unattended runs, put this on Linux or macOS. Use Windows for interactive work where you are approving each step.

What breaks when you walk away?

The honest answer is that autonomy shifts your work from writing code to reviewing it, and the failure modes move with it. Crew handles the mechanical parts well: checkpoints and retries carry long tasks through transient failures, heartbeat monitoring watches PRs and deploys until state changes, and corrections you make become durable lessons stored as editable Markdown.

What it cannot do is notice that the task was wrong. A well-specified job runs beautifully unattended. A vague one produces a large, confident, wrong diff that costs more to review than it would have cost to write. Scope discipline matters more here than in interactive work, because there is no midpoint where you glance at the screen and course-correct.

Two settings shape this directly. approval_mode set to interactive means you gate tool calls, which defeats overnight runs but is the right place to start. session.timeout_secs, 1800 by default, bounds how long a session can sit before it gives up. Loosen both deliberately, one at a time, once you trust a particular workflow.

If you want a gentler on-ramp, the launch Apps are scoped jobs rather than open-ended autonomy. Issue Radar triages issues and PRs, Task Runner handles bounded long-running work, DevFleets manages worktrees, and Code Review Sage weights findings by blast radius. Starting there teaches you the review rhythm without betting a repository on it. The same instinct applies to scheduling agents in CI: narrow scope first, widen only after you trust the output.

Should you set up Kiro Crew today?

If you already work with agents and keep hitting the session boundary, yes. That is the specific problem Crew solves, and it solves it in a way you can audit, since the whole thing is Apache 2.0 and runs on your hardware.

If you are new to agentic tooling, install it but keep approval_mode on interactive for a couple of weeks. Watch what it reaches for. The persistence is the feature, and persistence without understanding is how you end up reviewing a hundred commits you did not ask for.

The thing I did not expect is how much the review rhythm changes. Interactive agent work is a conversation. Unattended agent work is closer to managing a very fast, very literal contractor who never asks clarifying questions. That is a different skill, and it is worth building deliberately before you scale it up.

For more, see the Kiro Crew announcement, the Apache 2.0 source on GitHub, the install guide, and the Agent Client Protocol that Crew uses to drive agents.

Keep Reading

• Google Antigravity: Why I Think It Changes Everything. A different take on the agentic IDE idea, useful for comparing design choices.
• Claude Code Routines: Async CI Automation Just Became Real. The same scheduled-agent problem approached from the CI side.
• How to Build a Self-Hosted Slack AI Bot with any CLI Agent. Relevant if you plan to drive Crew from Slack.

If your team ran npm install between roughly 01:39 and 02:06 UTC on May 19, your CI runner or developer machine probably executed the payload. The malware was a preinstall script that harvested over 20 credential types and shipped them to attacker-controlled GitHub repos before the install even finished resolving.

This post is not a postmortem. It is a recovery playbook. If you shipped anything during the bad window, work through these steps in order. The first three are about containment. The last three are about hardening so the next wave does not land.

What happened in the May 2026 @antv npm attack?

On May 19, 2026, an attacker (tracked as TeamPCP by Socket, attributed by Microsoft as part of the Mini Shai-Hulud campaign) compromised the npm maintainer account atool. Over a 22-minute window, the attacker published 639 malicious versions spanning 323 packages, including the full @antv data-visualization suite, echarts-for-react, timeago.js, size-sensor, and canvas-nest.js.

The payload was a preinstall script with the entry "bun run index.js". On install, it ran a credential harvester that scraped at least 20 credential types: AWS access keys, GCP service accounts, Azure secrets, GitHub Personal Access Tokens, npm tokens, SSH private keys, kubeconfig files, and Vault tokens. It then created public GitHub repositories on stolen accounts (with names like sayyadina-stillsuit-852) and committed the harvested credentials there as a fallback exfiltration channel in case the primary C2 at t[.]m-kosche[.]com was blocked.

GitHub eventually invalidated 61,274 npm tokens with write access and 2FA bypass. The malicious package versions were unpublished. But anything those tokens already touched is suspect, and anything those exfiltrated credentials can still log into is fair game for an attacker. Cleanup is on you.

The earlier TanStack attack on May 11 used a different primitive (a GitHub Actions cache-poisoning bug that produced valid SLSA Build Level 3 attestations). This attack reverted to the classic playbook: own a maintainer account, push malicious versions, wait for npm install. Different mechanism, same outcome.

How do you know if your app pulled a poisoned echarts-for-react version?

You know if your package-lock.json or pnpm-lock.yaml or yarn.lock resolved echarts-for-react or any @antv/* package to a version published between May 19, 2026 01:39 UTC and 02:06 UTC. There is no ambiguity. Any install in that window pulled a poisoned tarball.

Run this from your project root:

# npm
jq '.packages | to_entries[] | select(.key | test("echarts-for-react|@antv/|size-sensor|timeago\\.js|canvas-nest\\.js")) | {key, version: .value.version, resolved: .value.resolved}' package-lock.json
 
# pnpm
grep -E "echarts-for-react|@antv/|size-sensor|timeago\.js|canvas-nest\.js" pnpm-lock.yaml | head -50
 
# yarn (v1)
grep -E "^(echarts-for-react|@antv/|size-sensor|timeago\.js|canvas-nest\.js)" yarn.lock -A 2

Then check each resolved version against the npm registry publish timestamp:

npm view echarts-for-react@<version-from-lockfile> time --json

If the timestamp falls in the bad window, you are hit. If not, you are clean for that package. Repeat for every match in the lockfile.

If the lockfile lookup is too slow because you have dozens of @antv/* deps, run a faster fleet check. Grep your CI runs for preinstall activity on May 19:

# Search CI logs for the preinstall payload signature
grep -r "bun run index.js" .github/ ci-logs/ 2>/dev/null
 
# Or query GitHub Actions runs (gh CLI)
gh run list --workflow=ci.yml --created '2026-05-19' --json conclusion,databaseId,headBranch

Any green run from that day that ran npm ci against a lockfile resolving to bad versions exfiltrated credentials. Treat the runner as compromised.

Which echarts-for-react and @antv versions are safe to pin?

Any version published before May 19, 2026 01:39 UTC is safe. The compromised window is bounded. Versions published after the unpublish event on May 20 are also safe, because the maintainer account has been recovered and the malicious versions are gone from the registry.

The fastest way to find the right pin is to list all versions with timestamps:

npm view echarts-for-react versions --json | jq -r '.[]' | tail -20
npm view echarts-for-react time --json | jq 'to_entries | sort_by(.value) | .[-20:]'

Pin to the last entry whose time value is before 2026-05-19T01:39:00Z. Same for every @antv/* package in your lockfile.

To enforce the pin across the workspace, use overrides. For npm:

{
  "overrides": {
    "echarts-for-react": "3.0.4",
    "@antv/g2": "5.2.20",
    "@antv/x6": "2.18.1",
    "@antv/graphin": "3.0.4"
  }
}

For pnpm:

# pnpm-workspace.yaml or package.json
pnpm:
  overrides:
    echarts-for-react: 3.0.4
    "@antv/g2": 5.2.20

The version numbers above are placeholders. Look up the actual safe versions yourself with npm view, because new safe versions ship continuously and a static post will go stale. Overrides force the resolver to ignore the transitive dependency graph and pin everywhere, including inside nested deps.

How do you clean the lockfile and reinstall safely?

Delete node_modules and the lockfile, then reinstall with --ignore-scripts so any residual malicious preinstall hook in your tree does not run. This is the single most important step in the whole playbook. Skipping --ignore-scripts on a poisoned lockfile is what trips most teams trying to clean up.

# Stop the dev server, kill any watchers
rm -rf node_modules
rm -f package-lock.json
# (or pnpm-lock.yaml / yarn.lock)
 
npm install --ignore-scripts

If you also have leftover untracked artefacts in your working tree from the poisoned install (random dot-files, generated test fixtures), use git clean with the -n dry run first so you do not nuke a stash or unstaged work.

Now audit the regenerated lockfile against the same grep from earlier. The versions should all be either pre-attack or post-recovery. If anything still resolves to a bad version, your overrides did not bite and you need to track down which transitive dep is pulling it in:

npm ls echarts-for-react
npm ls @antv/g2

npm ls shows the dependency chain. Whatever package brought in the bad version needs to be updated or temporarily replaced.

Once the lockfile is clean, run the install one more time without --ignore-scripts so any legitimate postinstall hooks (like patches or native compiles) run as expected:

npm install
npm run build
npm test

If the test suite passes and the build is green, your application code is back. Now you can deal with the credentials.

How do you rotate GitHub Actions secrets and npm tokens?

Rotate everything the payload could have seen. The payload ran inside npm install, which means it had access to whatever the runner had access to. On a typical GitHub Actions runner, that includes the entire GITHUB_TOKEN, every repository secret, every organization secret pulled into the job, the npm token in .npmrc if present, and any cloud credentials injected as env vars.

Walk this list in order:

1. npm tokens. Visit npmjs.com settings, Access tokens. Revoke every automation token used in the affected workflows. Re-create new ones, or better, migrate to npm trusted publishing with OIDC so you do not need long-lived tokens at all.
2. GitHub repository secrets. In each affected repo, Settings then Secrets and variables then Actions. Rotate every secret used in workflows that ran on May 19 or that share runners with affected workflows. Treat shared organization secrets the same way.
3. GitHub Personal Access Tokens. Go to your account Settings then Developer settings then Personal access tokens. Revoke every classic PAT and fine-grained PAT older than May 19. Re-issue with minimum required scopes.
4. Cloud credentials. AWS access keys, GCP service-account keys, Azure client secrets used by the affected workflows must all be rotated. Use the cloud console to identify keys last used by the affected runner.
5. SSH keys deployed to runners. If your runners had SSH keys to deploy to staging or production, rotate them. The payload reads ~/.ssh/ directly.
6. Vault tokens, Kubernetes configs, Datadog API keys, anything else you mounted into the job env. All of these were enumerated and exfiltrated. Rotate every one.

After rotation, audit the actual usage. AWS CloudTrail, GCP Audit Logs, and GitHub's audit log can show whether any of the rotated keys made an unusual API call between May 19 and the moment you revoked them. Anything unusual gets escalated to a full incident.

How do you scan git history for leaked credentials?

Run a fresh credential scan against the affected repository because the attacker may have pushed commits with exfiltrated data back into your repo using your own write tokens, or the rotation step might surface previously committed secrets that you never noticed. Both paths happen in real incidents.

Two tools that work well for this:

# trufflehog: live scan against the full git history
docker run --rm -v "$PWD:/repo" trufflesecurity/trufflehog \
  git file:///repo --only-verified
 
# gitleaks: faster, config-driven, can run in CI
docker run --rm -v "$PWD:/repo" zricethezav/gitleaks \
  detect --source /repo --verbose --no-banner

--only-verified on TruffleHog means it actually tests each finding against the relevant API to confirm it is a live credential, not just a string that looks like one. That cuts noise hard.

For each verified finding:

1. Revoke the credential at the source (provider console).
2. Remove the secret from history with git filter-repo (do this in a separate branch first, then force-push only after team sign-off, and never on a shared branch without explicit coordination). If a teammate accidentally deletes the branch holding their cleanup work, the git reflog recovery pattern gets it back.
3. Add the leaked pattern to your gitleaks.toml baseline so it does not flag on every CI run.

If the leaked credential was a database password, rotate that too. The payload could have read environment variables on developer laptops that pulled the .env file from disk.

How do you harden CI so the next wave does not land?

Three controls would have stopped the AntV payload cold, and adopting them now means the next wave does not become your next weekend of work.

Run npm install with --ignore-scripts everywhere in CI. Lifecycle scripts (preinstall, install, postinstall) are the universal attack vector for npm worms. The Microsoft postmortem explicitly recommends this. Add ignore-scripts=true to your .npmrc for CI runners and let only known-trusted packages run their scripts via an allowlist. Tools like @lavamoat/allow-scripts make this practical.

Pin lockfile versions and verify with a fast registry alternative. The faster your detection, the less time you spend in the bad window. Configure Renovate or Dependabot to alert on any dependency update, and use Socket's GitHub App or Snyk to flag malicious-package indicators before merge. The AntV payload was flagged by Socket within 6.7 minutes of publication. That is faster than most teams check Slack.

Switch publishers off long-lived npm tokens. This is what closes the underlying class. The atool account fell because the credential was steal-able and reusable. Migrate publishing workflows to OIDC-based trusted publishing so even a fully compromised maintainer account cannot republish from an attacker's machine. The full setup is in my npm trusted publishing guide.

For an extra layer, gate every publish workflow on a GitHub Environment with required reviewers. The OIDC token is only minted after a human approves the deployment. Combined with tag-only triggers (no publish on every main push), this means the attacker needs the source repo AND a human reviewer AND a tag push, not just a credential.

What should you do this week?

If you shipped anything that touches the npm ecosystem, do these in this order today:

1. Run the lockfile audit grep. Identify whether you are in the bad window.
2. If yes, treat affected runner credentials as burned and rotate them.
3. Regenerate the lockfile with --ignore-scripts, pin safe versions via overrides.
4. Run trufflehog or gitleaks against the full git history.
5. Set ignore-scripts=true in .npmrc for CI.
6. Schedule a follow-up to migrate publishing to OIDC trusted publishing.

The Mini Shai-Hulud campaign is not over. The TanStack attack on May 11 and the AntV attack on May 19 were eight days apart, and the same actor or copycats will hit another popular maintainer account before the year ends. The defenses above stop the class of attack, not just this incident.

For more on this, see Microsoft's Mini Shai-Hulud writeup, Socket's detection postmortem, and StepSecurity's incident analysis.

Keep Reading

• How to Enable npm Trusted Publishing with GitHub Actions OIDC. Closes the long-lived-token attack class that made this incident possible.
• TanStack npm Supply Chain Attack 2026: Detailed Breakdown. The May 11 sibling attack that used valid SLSA provenance to ship malware.
• Axios npm Supply Chain Attack 2026: Recovery Playbook. Earlier incident in the same playbook. Useful prior art on token-theft recovery.

I read the Microsoft Security postmortem the morning after and the root cause stuck with me. Every one of those packages had a long-lived NPM_TOKEN somewhere. Once the attacker had the token, they could republish from a laptop in a coffee shop and the registry would accept it.

There is a fix for this and it has been generally available since 2025. It is called npm trusted publishing, and it replaces stealable tokens with short-lived OIDC tokens minted by your CI provider per workflow run. This post walks through the full setup with GitHub Actions, end to end, including the cleanup step most teams skip.

What is npm trusted publishing and why does it matter in 2026?

npm trusted publishing is a credential-free publishing flow where your package authenticates to the npm registry using an OIDC token issued by a pre-approved CI provider, not a token you store in secrets.NPM_TOKEN. GitHub Actions, GitLab CI, CircleCI, and Buildkite are the supported providers today.

The flow is roughly this. When your workflow runs, it asks GitHub's OIDC provider for a signed JWT. That JWT contains claims like repository, workflow, and ref. The npm CLI sends the JWT to the npm registry. The registry checks the claims against the trusted publisher you configured on npmjs.com for that package. If they match, the publish succeeds.

There is no shared secret. No NPM_TOKEN. No long-lived bearer token sitting in your GitHub secrets waiting to be exfiltrated by a malicious prepare script. The OIDC token is minted on demand, scoped to one workflow run, and expires in minutes.

In 2026 this stopped being optional. The Mini Shai-Hulud worm hit the TanStack ecosystem on May 11 and the @antv ecosystem on May 19, both via stolen credentials. The earlier axios npm compromise used the same playbook. Every one of these incidents had the same root primitive: a stealable, long-lived publish token.

Trusted publishing also gives you build provenance for free. Public packages published this way get a SLSA Build Level 3 attestation showing exactly which repository, commit, and workflow produced the artifact. Consumers can verify it with npm view <pkg> --json and inspect dist.attestations.

Which versions of npm and Node.js support trusted publishing?

You need npm CLI 11.5.1 or later and Node.js 22.14.0 or higher. Both are documented as hard floors on the official npm docs page.

The simplest way to check what you have is to run this on a runner image you use:

node --version
npm --version

If you run an older version, the publish step fails with an authentication error. There is no graceful fallback to OIDC. Older npm does not understand the new flow at all.

A few practical notes from setting this up across half a dozen packages:

• The Ubuntu 24.04 runner image already ships Node 20 and npm 10 by default. You must use actions/setup-node@v6 (not v4 or v5) to pin a version that supports trusted publishing.
• Self-hosted runners are not currently supported. Only cloud-hosted GitHub-hosted runners can issue OIDC tokens that the npm registry accepts.
• Each package can only have one trusted publisher configured at a time. If you maintain a monorepo that ships multiple packages, you configure trusted publishing on each package's settings page separately.

The actions/setup-node v6 action handles the npm upgrade automatically when you pin node-version: '24'. That single line pulls in npm 11.x.

How do you configure the trusted publisher on npmjs.com?

Configuration happens once per package, in the npmjs.com web UI, before you ever touch your workflow file. The order matters. If you flip the workflow first and have not configured the publisher, the publish call fails and you scratch your head for an hour.

Walk through this in a browser:

1. Log in to npmjs.com. Open the package settings page at https://www.npmjs.com/package/<your-package>/access.
2. Scroll to the Trusted Publisher section.
3. Click GitHub Actions.
4. Fill in five fields, in order:
   • Organization or user: your GitHub org or username (the part before the slash in org/repo).
   • Repository: just the repo name, without the org prefix.
   • Workflow filename: the filename only, with the .yml or .yaml extension. For example, publish.yml. Do not include the path. Do not include the .github/workflows/ prefix.
   • Environment name (optional but recommended): the name of a GitHub Environment if you want to require deployment protection rules. I use npm-publish for this and gate it on a required reviewer.
   • Allowed actions: tick npm publish, npm stage publish, or both. For most packages, just npm publish.
5. Click Save.

The trust is pinned to the exact org/repo/workflow-filename triple. If you rename the workflow file later, publishing breaks. If you fork the repo, the fork cannot publish. If someone tries to publish from a different repository, the registry rejects the OIDC token.

This is the entire point. A stolen token from any machine cannot satisfy these claims, because the claims come from GitHub's OIDC provider and are signed.

What does the GitHub Actions workflow look like?

Here is a complete .github/workflows/publish.yml that publishes on a v* tag push. Drop this into your repo, adjust the test step, and commit.

name: Publish to npm
 
on:
  push:
    tags:
      - 'v*'
 
permissions:
  id-token: write
  contents: read
 
jobs:
  publish:
    runs-on: ubuntu-latest
    environment: npm-publish
    steps:
      - uses: actions/checkout@v5
 
      - uses: actions/setup-node@v6
        with:
          node-version: '24'
          registry-url: 'https://registry.npmjs.org'
          package-manager-cache: false
 
      - run: npm ci
      - run: npm test
      - run: npm publish --access public

Three lines do the actual work and they all need to be exactly right.

permissions: id-token: write is the line that unlocks OIDC. Without it, GitHub Actions will not issue an OIDC token to the runner, and npm publish falls back to looking for NPM_TOKEN, which is not there, and fails. This permission must be at the job level or workflow level. Job level is safer because it scopes the OIDC capability to only the publish job.

registry-url: 'https://registry.npmjs.org' tells setup-node where to point the .npmrc it creates on the runner. This is what wires up npm publish to talk to the public registry. If you publish to a private registry instead, change this URL.

package-manager-cache: false disables the setup-node built-in npm cache. This is a recommendation from the npm docs specifically for release builds. The cache can mask reproducibility issues by pulling stale tarballs, and on a release run you want a clean install every time.

A few intentional choices in this file worth calling out:

• I use environment: npm-publish to gate the job on a required reviewer. The OIDC token is only minted after the reviewer approves the deployment. This is what stops a force-pushed tag from publishing.
• The trigger is a tag push, not a branch push. Tags are immutable. Branches are not. If you publish on every main push, an attacker who lands a malicious commit on main can ship to npm before anyone notices. With a tag trigger, you need to also push a tag, which requires a separate manual action.
• There is no NODE_AUTH_TOKEN, no secrets.NPM_TOKEN, no .npmrc write step. setup-node v6 handles the OIDC handshake transparently when id-token: write is set.

Push a tag and watch the workflow run. The publish step should succeed and the npm registry page for your package should show a green provenance badge.

How do you remove long-lived tokens after trusted publishing works?

Trusted publishing only stops attacks if you delete the old tokens. This is the step most teams skip. They configure trusted publishing, see the green badge, ship a release, and leave the old NPM_TOKEN sitting in GitHub secrets and the automation token enabled on npmjs.com. Now they have two ways in and one of them is still stealable.

Do this in the same browser session you used to configure trusted publishing:

1. On the npm package page, go to Settings then Publishing access.
2. Select Require two-factor authentication and disallow tokens.
3. Save.

This setting means the registry will refuse any publish attempt that does not come from a trusted publisher or a maintainer with 2FA-verified web access. Even a leaked automation token cannot publish anymore.

Then revoke the actual token:

1. Click your avatar then Access tokens.
2. Find any automation token that was used for publishing this package.
3. Click Revoke on each.

Finally, clean up GitHub secrets:

1. In the repo, go to Settings then Secrets and variables then Actions.
2. Delete NPM_TOKEN from the repository secrets and any environment secrets where it lived.

Run one more publish via the workflow to confirm nothing was relying on the deleted secret. If your workflow file still references NODE_AUTH_TOKEN or secrets.NPM_TOKEN anywhere, remove those references. The new flow does not need them.

What are the limitations of npm trusted publishing today?

Trusted publishing has real edges and you need to know them before you commit. The docs are honest about what is and is not supported.

• Self-hosted runners are not supported. Only cloud-hosted GitHub-hosted runners can issue OIDC tokens the npm registry accepts. If you run a release pipeline on your own infrastructure, trusted publishing is not yet an option for that workflow.
• One trusted publisher per package. You cannot have both GitHub Actions and GitLab CI as trusted publishers for the same package. Pick one. Multi-CI publishing remains a token-based flow.
• Provenance generation requires a public repo and a public package. If you publish a private package, the publish itself works fine over OIDC, but the SLSA Build Level 3 attestation is not generated. This is a GitHub-side restriction on which workflows can sign provenance.
• CircleCI does not get auto-provenance. It is a supported trusted publisher but provenance generation is not currently supported there.
• Other npm commands still need tokens. npm install, npm view, and npm access calls against a private registry still need a token. Trusted publishing only covers the publish path.

The biggest practical edge is the self-hosted runner constraint. I have one project on a self-hosted runner for compliance reasons and we run the publish job in a separate workflow on a cloud-hosted runner specifically to get trusted publishing. The build still happens on the self-hosted runner. Only the final publish step runs on a GitHub-hosted runner.

How does trusted publishing change the attack surface?

Trusted publishing closes one attack class cleanly and leaves another wide open. Know which is which before you ship.

What it stops: any attacker who steals an NPM_TOKEN from a developer machine, a leaked .npmrc, a logging system, a stack trace, or a compromised CI environment cannot publish anymore. The token they stole is not what the registry checks. The registry checks an OIDC claim that only a real GitHub Actions runner inside the configured repo can produce.

What it does not stop: an attacker who lands a commit on your repository. The malicious commit can ride through your trusted publishing workflow and ship to npm with a valid SLSA Build Level 3 attestation. This is exactly what the TanStack worm did on May 11, 2026. The attestation was real. The commit it attested to was malicious. Provenance proves where a package was built, not whether the source was trustworthy.

To close the second class, pair trusted publishing with these:

• Branch protection on main (and any tag pattern you publish from) requiring pull request reviews from a different account than the author.
• Required reviewers on the GitHub Environment that gates the publish job. This is why I use environment: npm-publish in the workflow above.
• Tag-trigger only. Do not publish on every main push. Require a separate tag push that a human has to perform.
• Dependency review on PRs. Catch malicious package.json prepare hooks or new transitive deps before they hit the publish job. GitHub's dependency-review-action is one option.

Trusted publishing is the floor, not the ceiling. It removes the easy attack. The hard attack still works and you defend against it with code review, environment gates, and tag-based release discipline.

What should you do this week?

I do not think trusted publishing is the last credential change npm will ship. Provenance is still optional, the self-hosted runner gap is real, and the cross-CI story is messy. But the credential-free publish flow is the most useful security upgrade npm has shipped in a decade, and the cost to adopt is one configuration screen plus a small workflow change.

If you publish anything to npm, set this up this week. Then delete the old token. Then do not publish a package without it ever again.

For more on this, see the official npm trusted publishing docs, the Microsoft Security postmortem on the AntV compromise, and the StepSecurity analysis of the AntV wave.

Keep Reading

• TanStack npm Supply Chain Attack: How the May 11 Compromise Worked. The first npm worm to ship with valid SLSA provenance. Explains why OIDC tokens alone are not enough.
• Axios npm Supply Chain Attack 2026: What Happened and How to Defend. Earlier incident in the same playbook. Useful prior art on token-theft attacks.
• AI-Driven Anomaly Detection for Supply Chain Security. Detection complement to the prevention story in this post.

I turned this on across my own staging cluster the week the release dropped. This post walks the enablement the way I ran it: the prerequisite check that decides whether your nodes can opt in at all, the smallest possible pod manifest that exercises the feature, the verification step that proves the UID mapping is doing what you expect, and the migration plan for existing workloads that break under the change.

What changed with user namespaces in Kubernetes 1.36?

User Namespaces graduated to General Availability in Kubernetes 1.36 (the Haru release, April 22, 2026), with the matching feature gate enabled by default. The pod-spec field is hostUsers: false. When set, the kubelet asks the container runtime to create the pod in a new user namespace, and the runtime maps the container's UID 0 to a non-privileged UID on the host.

The KEP behind this is KEP-127. The headline behaviors that flipped from beta to stable in v1.36 are:

The two changes that matter most in practice are the namespaced capabilities and the transparent UID translation on mounted volumes. A pod with CAP_NET_ADMIN can still tweak its own network namespace, but cannot touch the host's. A pod that mounts a host volume sees files as owned by UID 0 inside the container while the host disk still records the actual UID. No chown step, no init container shuffling permissions, no setgid sticky-bit cleanup.

Why do user namespaces matter for container escape attacks?

User namespaces matter because they sever the link between "root in the container" and "root on the host". A process that achieves a container escape on a pod with hostUsers: false lands as an unprivileged UID on the node, which means the standard escape primitives (writing to /proc/sys, mounting host paths, opening privileged sockets, loading kernel modules) all fail at the kernel boundary.

The history here is long and bloody. Every six months a new CVE in runc, containerd, or a kernel subsystem hands an attacker root on the host because they were already root in the container and the host trusted that UID. The 2019 runc symlink escape (CVE-2019-5736) was patched in a week. The 2022 cgroups v1 race (CVE-2022-0492) took longer. Then there was the 2024 runc LeakyVessels chain. The pattern is the same every time: container UID 0 reaches a kernel surface, and the kernel applies host UID 0 semantics.

User namespaces close that pattern at the source. The container still has UID 0 inside its own namespace. The kernel still applies UID 0 semantics. But the inside-the-namespace UID 0 is mapped to a non-privileged outside-the-namespace UID, so the kernel's UID-0-only operations fail when the escape lands on the host. CAP_SYS_ADMIN on the inside means namespace-local admin. CAP_SYS_ADMIN on the host? The escaped process does not have it.

The blast-radius difference shows up clearly when you run the same exploit twice. I did this on a non-production cluster with a deliberately vulnerable image based on the Stripe Open Source Defense write-up. Without user namespaces the escape gave shell access to the node. With hostUsers: false the escape produced a shell as an unmapped UID with no write access to /etc, no ability to mount, and no ability to load kernel modules.

This is defense in depth, not a silver bullet. The pod can still be malicious, the container can still be DoS'd, and a kernel bug that bypasses namespaces entirely (which is rare but does happen) gets through anyway. Pod Security Admission still applies. runAsNonRoot still applies. Seccomp still applies. User namespaces are one more layer.

How do you check kernel and runtime prerequisites?

You check the prerequisites by reading three numbers from each node: the kernel version, the container runtime version, and the storage driver. The minimum supported combination is Linux 6.3, containerd 2.0 (or CRI-O 1.30), and a filesystem that supports idmap mounts. Older kernels can work back to 5.19 with a containerd 2.0 plus overlay-with-idmap config, but 6.3 is what the official kubernetes.io docs settle on for the GA path.

Run this on each node as a quick sanity check.

# Kernel version
uname -r
# Expected: 6.3 or newer for the supported GA path
 
# Container runtime
crictl version
# Expected: containerd 2.0.x or newer, or CRI-O 1.30.x or newer
 
# Storage driver
crictl info | jq '.config.containerd.runtimes' 2>/dev/null
# Or for CRI-O:
crio config 2>/dev/null | grep storage_driver
 
# Confirm the kernel exposes the namespace feature
ls /proc/self/ns/ | grep user
# Expected: user

Two compatibility traps to watch for:

1. containerd 1.7 + Linux 5.15. The setting is honored, but you pay a measurable storage and latency penalty because overlayfs does not support idmap mounts on that kernel. Upgrade the kernel before you turn this on broadly.
2. containerd 1.6 or older. The hostUsers: false field is silently ignored. No error, no warning, no namespace. This is the worst failure mode because pods appear to be in a user namespace until you actually verify the UID mapping.

For managed Kubernetes the answer is usually simple: GKE, EKS, and AKS all ship node pools that meet the requirement on their current LTS images as of April 2026. Use the latest node image on each, and the feature is available without extra work.

For self-managed clusters on Ubuntu 22.04, the default kernel is 5.15, which means an HWE (hardware enablement) kernel upgrade is the smallest path. Ubuntu 24.04 ships 6.8 by default and is the easier choice. RHEL 9.4 ships 5.14, which means you need the 9.5 update for a 6.x kernel.

If even one node fails the prereqs, do not enable user namespaces on the whole cluster. Cordon and drain the non-compliant nodes first, or label them and use a node selector on the user-namespaced pods. Mixed clusters work, mixed-on-the-same-node configs do not.

How do you enable user namespaces on a pod?

You enable user namespaces by adding a single field to the pod spec: hostUsers: false. Everything else is automatic. The kubelet asks the container runtime to create the pod in a new user namespace, the runtime mints a UID range, and the kernel performs transparent UID translation on every namespaced operation including volume mounts.

The smallest demonstrable pod is this one.

# userns-demo.yaml
apiVersion: v1
kind: Pod
metadata:
  name: userns-demo
spec:
  hostUsers: false
  containers:
    - name: shell
      image: debian:bookworm-slim
      command: ['sleep', 'infinity']
      securityContext:
        runAsUser: 0
        runAsGroup: 0

Apply it.

kubectl apply -f userns-demo.yaml
kubectl wait --for=condition=Ready pod/userns-demo --timeout=60s

Now check what the container thinks its UID is, and what the node thinks the same process's UID is.

# What the container sees
kubectl exec userns-demo -- id
# Expected: uid=0(root) gid=0(root) groups=0(root)
 
# What the node sees. Run on the node that hosts the pod.
PID=$(kubectl get pod userns-demo -o jsonpath='{.status.containerStatuses[0].containerID}' \
  | sed 's|.*://||' \
  | xargs -I{} crictl inspect {} \
  | jq -r '.info.pid')
 
# On the node:
cat /proc/$PID/status | grep -E "^Uid:"
# Expected: a UID far from 0, e.g. Uid: 100000 100000 100000 100000

That second number is the proof that the namespace did its job. Container thinks it is UID 0. Host kernel thinks it is UID 100000 (or whatever range the runtime picked). Every kernel operation that requires real-UID-0 privileges fails on the host side.

For workloads that need more than the default 65536-UID range (large multi-tenant setups), the kubelet honors the --userns-uid-mapping config flag. I have never needed this. The default range is enough for almost every workload.

To opt out per pod, just omit the field. The default is hostUsers: true, which is the v1.35 behavior. To opt in for an entire namespace, use Kyverno or any other admission controller to inject the field. The smallest Kyverno policy is six lines and avoids the per-deployment churn.

# kyverno-userns.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: enforce-user-namespaces
spec:
  rules:
    - name: set-hostusers-false
      match:
        any:
          - resources:
              kinds: ['Pod']
              namespaces: ['team-a', 'team-b']
      mutate:
        patchStrategicMerge:
          spec:
            hostUsers: false

How do you verify the UID mapping works correctly?

You verify the UID mapping by writing a file from inside the container and reading the on-disk ownership from the node. If the in-container ownership shows UID 0 but the on-disk ownership shows a non-zero UID in the runtime's mapping range, the namespace is doing what it claims.

Set this up with an emptyDir volume.

# userns-verify.yaml
apiVersion: v1
kind: Pod
metadata:
  name: userns-verify
spec:
  hostUsers: false
  containers:
    - name: writer
      image: debian:bookworm-slim
      command: ['sleep', 'infinity']
      volumeMounts:
        - name: scratch
          mountPath: /scratch
  volumes:
    - name: scratch
      emptyDir: {}

Write a file from inside the container.

kubectl exec userns-verify -- sh -c 'echo hi > /scratch/test && ls -la /scratch/test'
# Expected: -rw-r--r-- 1 root root 3 ... /scratch/test

Now find the file on the node and check the on-disk ownership.

# Get the host path of the emptyDir
PODUID=$(kubectl get pod userns-verify -o jsonpath='{.metadata.uid}')
NODE=$(kubectl get pod userns-verify -o jsonpath='{.spec.nodeName}')
 
# On the node:
sudo find /var/lib/kubelet/pods/$PODUID -name test -exec ls -la {} \;
# Expected: -rw-r--r-- 1 100000 100000 ... /var/lib/kubelet/pods/<uid>/volumes/.../test

That UID 100000 is the host-side view of the container's UID 0. If you see UID 0 on the on-disk side instead, the namespace did not take. Common causes: containerd is 1.6.x, the kernel does not support idmap mounts, or the filesystem of the kubelet directory does not support idmap. All three appear in the kubelet logs when they fail.

For a stronger test, try a privileged operation that should fail.

kubectl exec userns-verify -- sh -c 'mount -t tmpfs none /mnt 2>&1'
# Expected: mount: /mnt: permission denied

Without user namespaces, root in a container can mount tmpfs inside the container's mount namespace. With user namespaces, the mount syscall lands as a non-privileged UID on the host kernel and fails. That permission-denied is the proof that namespaced capabilities are working.

How do you migrate existing workloads to user namespaces?

You migrate existing workloads by classifying them against three failure modes (low ports, shared-UID volumes, host paths), enabling hostUsers: false on the safe ones first, and rewriting the broken ones one at a time. Do not flip the whole namespace at once.

The three failure modes show up like this.

For the low-port case, the fix is a one-line addition.

spec:
  hostUsers: false
  containers:
    - name: web
      image: my-nginx:latest
      ports:
        - containerPort: 80
      securityContext:
        capabilities:
          add: [NET_BIND_SERVICE]

The capability is namespaced under user namespaces, so it grants the bind privilege inside the container without granting any host-side capability.

For the shared-volume case, the right answer almost always is "do not share a host volume". If you absolutely need to share with another pod, share via a NetworkVolume (PVC) or a sidecar pattern. The on-disk UID mismatch that user namespaces introduce is the kernel doing its job; do not chown your way around it.

The migration sequence that worked on my own staging cluster was:

1. Pick a low-stakes deployment with no host volumes and no low ports.
2. Patch the deployment to add hostUsers: false.
3. Wait one full traffic cycle. Watch pod restart count and the application's own error logs.
4. If clean, move the next deployment.
5. After half a dozen deployments without issue, layer a Kyverno policy that defaults the namespace to hostUsers: false and start auditing the holdouts.

That is the path the Kubernetes docs recommend and it is the path I would follow in production. The dangerous version is the all-at-once flip on a busy namespace, which surfaces the failure modes at the worst time.

What common problems should you watch for?

The common problems are containerd-1.6 silently ignoring the field, kernel versions that look new enough but lack idmap mounts on the kubelet's filesystem, and workloads with hidden CAP_SYS_ADMIN needs that fail in surprising ways.

For each, here is the signal and the fix.

# Signal 1: containerd ignores hostUsers: false silently
kubectl exec <pod> -- cat /proc/self/uid_map
# A namespaced pod prints something like:
#          0     100000      65536
# A non-namespaced pod prints:
#          0          0 4294967295
# If you set hostUsers: false and see the second line, the runtime ignored it.
# Fix: upgrade containerd to >= 2.0, or move to a node where it is.
 
# Signal 2: kubelet logs the idmap failure
journalctl -u kubelet -n 200 | grep -i idmap
# Common error: "failed to set up user namespace mount: idmap not supported"
# Fix: upgrade to a kernel >= 6.3 (5.19 is the absolute minimum), or move the
# kubelet root directory to a filesystem that supports idmap mounts.
 
# Signal 3: workload fails with CAP_SYS_ADMIN-like errors
kubectl logs <pod> | grep -E "operation not permitted|permission denied"
# This often means the workload needs a capability that is no longer host-scoped.
# Fix: add the specific capability to securityContext.capabilities.add. If the
# workload needs real host-side CAP_SYS_ADMIN, it is not a candidate for user
# namespaces. Use a privileged sidecar pattern instead.

The other quiet problem is metrics. Pods that report container PIDs to a host-side telemetry collector (for example a host-mode Prometheus exporter for cgroup stats) need to translate the PID across the namespace. The standard exporters handle this fine. Home-grown collectors usually do not. Test your observability stack on at least one user-namespaced pod before you flip a whole namespace.

For deeper context on the security primitives that compose with user namespaces, see the zero trust microservices guide for the auth-side counterpart, the AI-driven anomaly detection guide for runtime detection that pairs with kernel isolation, and the TanStack supply chain attack postmortem for the kind of supply-chain blast radius this feature reduces.

For the original sources, see the Kubernetes 1.36 user namespaces GA announcement, the user namespaces concept documentation, the task documentation for enabling user namespaces on a pod, and KEP-127.

Keep Reading

• Zero Trust Microservices with Spring Security. The auth-side counterpart to kernel-level isolation. Both layers together close the practical attack paths into a pod.
• How to Migrate from Ingress NGINX After Kubernetes Retired It. The other K8s 1.36 era change you cannot defer. Pair the migration windows.
• AI-Driven Anomaly Detection for Security. Runtime detection layered on top of the kernel isolation user namespaces provide.
• TanStack npm Supply Chain Attack 2026. The kind of supply-chain compromise user namespaces reduce the blast radius of when the malicious code lands inside a container.

Then you read the two behavior changes and realize one of them fails silently.

I went through this on my own projects over the weekend. The model ID swap took thirty seconds. Working out why one route started returning truncated answers took considerably longer, and the answer turned out to be documented in a single sentence I had skimmed past. So here is the migration in the order it actually bites you.

What is Claude Opus 5 and what does it cost?

Claude Opus 5 is Anthropic's current Opus-tier model, released 24 July 2026 with the API model ID claude-opus-5. It costs $5 per million input tokens and $25 per million output tokens, identical to Claude Opus 4.8. There is no long-context premium and no date suffix on the ID.

The specs that matter:

That "default and maximum" detail on the context window is worth reading twice. There is no smaller context variant to opt into, which removes a config knob some teams were using to control cost.

On benchmarks, Anthropic reports Opus 5 landing within 0.5% of Claude Fable 5 on CursorBench 3.2 at half the price, roughly three times the next-best score on ARC-AGI 3, and about 1.5 times the next-best pass rate on Zapier's AutomationBench. Treat vendor benchmarks as vendor benchmarks. The claim I find more useful is the shape of the gains: deep reasoning, long agentic loops, and test-time compute scaling, meaning the model converts extra effort into better answers more reliably than earlier Opus versions did.

It is available on the Claude API as claude-opus-5, on Amazon Bedrock as anthropic.claude-opus-5, and on Google Cloud and Microsoft Foundry under the bare ID. Opus 4.8 stays available everywhere, so there is no forced cutover.

One operational detail that is easy to miss: Claude Opus 5 draws on a separate rate-limit bucket from the combined Opus 4.x pool. Moving traffic over does not free headroom on the old bucket and does not inherit it. Check your tier's Opus 5 limits before you shift real volume.

Why does thinking being on by default break your max_tokens?

Because max_tokens caps thinking plus reply text together, and on Opus 4.8 a request that omitted the thinking field produced no thinking at all. The same request on Opus 5 thinks. Your reply now shares a budget it used to own outright.

This is the change that cost me an afternoon. Nothing throws. The request returns HTTP 200, stop_reason comes back as "max_tokens", and the answer is cut mid-sentence. If you are logging only errors, you will not see it.

The wire format did not change. thinking: {"type": "adaptive"} is still valid and is exactly equivalent to the new default. What changed is what happens when you say nothing:

# Opus 4.8: no thinking field means no thinking.
# 4096 tokens was all reply text.
client.messages.create(
    model="claude-opus-4-8",
    max_tokens=4096,
    messages=[{"role": "user", "content": prompt}],
)
 
# Opus 5: identical call, but thinking now runs and eats
# into the same 4096. Long answers get truncated.
client.messages.create(
    model="claude-opus-5",
    max_tokens=4096,
    messages=[{"role": "user", "content": prompt}],
)

You have two honest fixes. Raise max_tokens to leave room for both, which is what I did on every route where the answer length varies. Or keep the old behavior explicitly:

client.messages.create(
    model="claude-opus-5",
    max_tokens=4096,
    thinking={"type": "disabled"},
    output_config={"effort": "medium"},
    messages=[{"role": "user", "content": prompt}],
)

That second option comes with its own trap, which is the next section.

Audit target: every call site that never set a thinking field. Those are the ones that silently changed behavior. Routes that already passed {"type": "adaptive"} are unaffected.

Why does disabling thinking now return a 400 error?

Because on Claude Opus 5, thinking: {"type": "disabled"} is only accepted when effort is high or below. Pair it with xhigh or max and you get a 400. On Opus 4.8 the two settings were independent, so this is a genuine breaking change rather than a default shift.

The part that catches people is that validation runs per request. Effort and thinking are checked independently on every call. A conversation can run happily for twenty turns at high with thinking off, then fail the moment your code bumps effort to xhigh for a harder step. Earlier success in the same session buys you nothing.

# 400 invalid_request_error on Claude Opus 5
client.messages.create(
    model="claude-opus-5",
    max_tokens=4096,
    thinking={"type": "disabled"},
    output_config={"effort": "xhigh"},   # <- rejected
    messages=[{"role": "user", "content": prompt}],
)

Pick one of two resolutions. Keep thinking off and cap effort at high, which costs you the top two tiers. Or delete the thinking field and keep the effort tier, which costs tokens and buys back quality.

My advice is the second one, and not just for compliance. Anthropic documents two real failure modes with thinking disabled on Opus 5: the model occasionally writes a tool call into its visible text instead of emitting a tool_use block, and it can leak internal XML tags into the response. The first is nastier than it sounds. The turn completes normally, no error is raised, and the tool never runs. In an agentic loop that phantom call then sits in the history and skews later turns.

If a route genuinely must keep thinking off, the documented mitigations are counterintuitive enough to be worth stating plainly:

• Give the model permission to talk first: "You may say a brief sentence before using a tool." The tool-call-as-text failure appears to come from suppressing the preamble it wants to write.
• Delete any instruction telling it not to think or not to reason. That kind of rule makes tag leakage worse, not better.
• Write the tag instruction generically. "Do not include internal or system XML tags in your response" works better than naming thinking tags explicitly.

For most teams, running at low or medium effort with thinking on is cheaper and better behaved than disabling thinking at high.

How should you pick an effort level on Claude Opus 5?

Start at high, which is the default, then sweep in both directions against your own evals. Effort carries more weight on Opus 5 than on any earlier Opus, because the model converts extra effort into better output more reliably.

Anthropic's published starting points and the model's measured behavior pull in slightly different directions, and both are useful:

• Start xhigh for coding and agentic work, high for other intelligence-sensitive workloads. max is the top tier for the deepest reasoning, though it can overthink simple tasks.
• Then sweep downward, because low and medium are unusually strong here. They deliver good quality at a fraction of the tokens and latency on plenty of workloads.

Do run the sweep. Effort defaults inherited from Opus 4.8 or 4.7 are rarely the right setting on this model, and I would not trust a number carried over from a previous migration. If you kept effort notes from the Opus 4.7 release, re-run them rather than reusing them.

At xhigh or max, set a large max_tokens so the model has room to think and act across tool calls and subagents. Start at 64000 and tune down. And stream anything that large, or you will hit SDK HTTP timeouts before the model finishes:

with client.messages.stream(
    model="claude-opus-5",
    max_tokens=64000,
    output_config={"effort": "max"},
    messages=[{"role": "user", "content": prompt}],
) as stream:
    response = stream.get_final_message()

What new API features ship with Claude Opus 5?

Three, and two of them are behind beta headers. None are required to migrate.

Mid-conversation tool changes (beta mid-conversation-tool-changes-2026-07-01) let you add or remove tools between turns without invalidating the prompt cache. Previously the tool list was fixed for a conversation's lifetime and any edit re-billed the entire prefix, because tools render at position zero. You declare the tool up front with "defer_loading": true, then surface it later with a tool_addition block on a system message. This is the control-plane counterpart to tool search: tool search is for discovery, this is for when your application knows the tool set changed.

Default fallbacks mode (beta server-side-fallback-2026-07-01) is the one I would turn on for everyone. Opus 5 ships with stricter cybersecurity safeguards, and its classifiers can decline a request. A decline is an HTTP 200 with stop_reason: "refusal", not an error, so code that reads response.content[0] unconditionally breaks on it. The fallbacks parameter re-runs a declined request on another model server-side:

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    betas=["server-side-fallback-2026-07-01"],
    fallbacks="default",
    messages=[{"role": "user", "content": prompt}],
)
 
if response.stop_reason == "refusal":
    handle_refusal(response.stop_details)

Prefer "default" over pinning a model. It routes by refusal category, and it saves you a migration the next time a pinned fallback gets deprecated. Cyber-category refusals route to Opus 4.8. Note this is Claude API only, so on Bedrock, Google Cloud, or Foundry you need the SDK's client-side fallback middleware instead.

A lower prompt cache minimum, at 512 tokens, down from 1024 on Opus 4.8. No code change, but it is worth re-checking prompts you previously wrote off as too short to cache. Anything between 512 and 1024 tokens now creates cache entries for free.

Which prompt instructions should you delete after migrating?

The verification ones. Claude Opus 5 verifies its own work without being asked, so instructions like "include a final verification step" or "use a subagent to verify" now cause over-verification rather than preventing errors.

This inverts a standard prompting rule. "Ask the model to double-check itself" is generally sound advice and it is wrong here. If you keep a shared prompt library, that needs a carve-out for this model rather than a global rule. The same applies to framework-level scaffolding. Separate verification passes carried over from earlier models are likely redundant now.

Three other behavior shifts are worth tuning for, all documented by Anthropic:

Responses run longer. Both conversational output and files the model writes to disk. Lowering effort does not reliably shorten visible output, so this is a prompting fix, not a config one. A short conciseness instruction is the lever.

It narrates progress more in agentic sessions. If you added scaffolding to force interim status updates ("after every three tool calls, summarize"), remove it. Opus 5 does this on its own, and the combination is noisy.

It delegates to subagents more readily. This is a direction change worth flagging, because Opus 4.8 under-reached for subagents and needed prompting to delegate at all. If you added "delegate more" guidance for 4.8, take it out, and consider an explicit cap. Each subagent re-establishes context, re-explores, reports back, and then the coordinator re-reads the report. That multiplies both cost and latency.

It can also expand task scope, adding steps you did not request. A short scope-discipline instruction handles it: deliver what was asked at the scope intended, flag a concern in a sentence if the ask looks wrong, and keep going rather than quietly widening the work.

Should you migrate to Claude Opus 5 today?

If you are on Opus 4.8, yes, and the cost case makes it easy: identical pricing, better output. The two breaking changes are both mechanical and both findable with a grep.

Here is the checklist I actually used:

1. Swap the model ID to claude-opus-5.
2. Grep for call sites with no thinking field. Raise max_tokens on each, or set {"type": "disabled"} deliberately.
3. Grep for "disabled" paired with xhigh or max effort. Fix both sides of the pair.
4. Re-run your effort sweep from scratch. Do not reuse 4.8 numbers.
5. Add a stop_reason == "refusal" branch before reading content, and turn on fallbacks: "default".
6. Delete verification instructions and forced progress-update scaffolding from your prompts.
7. Confirm your rate-limit headroom on the new bucket before shifting volume.

Steps 1 to 3 are required. The rest is the difference between a migration that works and one that works well.

The thing I keep coming back to is step 6. We spent two years writing prompts that compensated for models which did not check their own work, did not narrate progress, and would not delegate. Those instructions are now actively counterproductive. Migrating to a better model is turning out to be less about adding capability and more about deleting the scaffolding you built for the last one, which is a strange and slightly humbling way to upgrade.

For the full specification, see Anthropic's What's new in Claude Opus 5, the Claude Opus 5 announcement, and the model migration guide.

Keep Reading

• Claude Opus 4.7: Pricing, Benchmarks & Breaking Changes. The previous Opus migration, where adaptive thinking and the removal of sampling parameters first landed.
• Claude Mythos and Project Glasswing: Anthropic's Locked-Down AI Model. Where the Fable and Mythos tier sits relative to Opus, and why Opus 5 at half the price matters.
• 10 Best Claude Skills for Developers in 2026. Skills that pair well with the longer agentic runs Opus 5 is built for.

Roughly half the cloud-native estate depends on this controller. The retirement does not break anything immediately, but every new CVE in the NGINX core, Lua, OpenResty, or the controller code itself stays unpatched. My own staging cluster ran Ingress NGINX behind every service. I migrated it the weekend the announcement landed, and this post is the playbook I used: which replacement to pick, the Traefik drop-in path for the panicked migration, the Gateway API with Cilium path for the proper migration, and the zero-downtime cutover sequence.

What happened to Ingress NGINX?

Ingress NGINX hit end of life on March 24, 2026, when Kubernetes SIG Network and the Security Response Committee formally retired the kubernetes/ingress-nginx project. The retirement was telegraphed in the November 2025 announcement on kubernetes.io, confirmed in the January 2026 SRC statement, and made final at the March 2026 retirement date.

What changed on that date:

• The GitHub repos became read-only.
• The controller no longer receives bug fixes, feature changes, or CVE patches.
• Helm charts and container images that already exist stay published, but new ones will not be cut.
• The planned successor InGate was also abandoned. The community did not show up to maintain it.

What did not change:

• Existing in-cluster Ingress NGINX deployments keep running.
• Existing Ingress resources keep working against any controller that implements the Ingress API.
• Cloud-managed alternatives (AWS ALB Controller, GCE Ingress, Azure Application Gateway) are unaffected because they are separate codebases.

The official recommendation from SIG Network is direct: begin migration to Gateway API or to another actively maintained Ingress controller immediately. The Kubernetes blog's own words are that staying on Ingress NGINX after retirement "leaves you and your users vulnerable to attack". If you ship anything to production behind this controller, this is the patch you cannot defer.

Who needs to migrate and how urgent is it?

Anyone running the kubernetes/ingress-nginx controller needs to migrate. The urgency depends on what kind of traffic the controller fronts. Public-facing clusters with any auth surface should treat this as a same-quarter migration. Internal-only clusters get one cycle of grace at most.

Use this matrix to scope your own work.

The thing to watch on the urgent rows is your own CVE feed. Every Nginx core CVE, every OpenResty advisory, every Lua module disclosure now stays unpatched against the retired controller. A WAF in front of it buys you time but not safety, because most of the meaningful vulnerabilities are at the HTTP layer the WAF needs to let through.

I disagree with one piece of common advice making the rounds: that you can ride out the retirement with a strict WAF policy and pinned image hashes. That works for a few weeks. It does not work for a year. The same logic that retired the controller (limited maintainer bandwidth, growing CVE backlog) applies to the WAF policy too, because the WAF authors are not building new rules against a controller nobody else uses anymore.

Which replacement should you pick?

Pick Gateway API as the long-term destination and Traefik as the safe staging point if you cannot reach Gateway API immediately. The community has aligned on Gateway API as the next-generation traffic management standard. Traefik is the only realistic drop-in replacement because its NGINX Ingress Provider reads existing nginx.ingress.kubernetes.io annotations natively.

Here is the short comparison.

For my own staging cluster I ran a two-phase migration: Traefik first, Gateway API second. The Traefik phase took two hours. The Gateway API phase took a week. Both shipped without downtime.

Cilium is the right Gateway API implementation if your cluster is already on Cilium CNI. The functionality is built in. You do not deploy a second component, and the eBPF datapath that already routes pod traffic also handles ingress.

How do you migrate to Traefik with the drop-in adapter?

You migrate to Traefik by installing the chart with the NGINX Ingress Provider enabled, leaving Ingress NGINX running side by side, and switching your DNS or LoadBalancer one host at a time. Traefik reads your existing Ingress resources directly. You do not rewrite manifests in the first pass.

The minimum Helm install looks like this.

helm repo add traefik https://traefik.github.io/charts
helm repo update
 
helm install traefik traefik/traefik \
  --namespace traefik-system \
  --create-namespace \
  --set providers.kubernetesIngress.enabled=true \
  --set providers.kubernetesIngress.publishedService.enabled=true \
  --set providers.kubernetesIngress.ingressClass=traefik \
  --set ingressClass.enabled=true \
  --set ingressClass.isDefaultClass=false \
  --set ingressRoute.dashboard.enabled=false

Two settings to call out. First, providers.kubernetesIngress.enabled=true turns on the NGINX-compatible Ingress reader. Second, ingressClass.isDefaultClass=false keeps Traefik from grabbing every existing Ingress resource at install time. You opt resources in one by one by changing their class.

Now tag the first Ingress to move.

# my-app-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    kubernetes.io/ingress.class: traefik   # was: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: traefik   # was: nginx
  rules:
    - host: my-app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app
                port:
                  number: 80

The nginx.ingress.kubernetes.io/rewrite-target annotation stays. Traefik reads it. So do most of the popular annotations in that namespace, including auth-url, auth-signin, enable-cors, force-ssl-redirect, and ssl-redirect. The Traefik documentation lists the supported set and clearly marks the ones that are not handled. Read that page once before you start, because the annotations that do not translate cleanly tend to be the regex-based rewrites and the more exotic auth flows.

Apply the manifest. Now the resource is owned by Traefik. The Ingress NGINX controller stops reconciling it the moment the class flips.

Repeat for every Ingress in the cluster. When the last one is over, scale Ingress NGINX to zero.

kubectl scale -n ingress-nginx deployment/ingress-nginx-controller --replicas=0

Leave it at zero for a week. If nothing breaks, delete the namespace. If something breaks, scale it back up and the failed resource resumes traffic the moment the class flips back.

How do you migrate to Gateway API with Cilium?

You migrate to Gateway API by installing the CRDs, enabling the Gateway API feature in your chosen implementation, declaring a Gateway resource per listener, then rewriting each Ingress as an HTTPRoute. The Cilium path is the most compact because Cilium already runs as your CNI and the Gateway API support is a flag, not a new component.

Install the standard Gateway API CRDs first.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml

Then enable Gateway API in Cilium. For an existing Cilium install this is a Helm value update.

helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --reuse-values \
  --set gatewayAPI.enabled=true \
  --set gatewayAPI.enableAlpn=true \
  --set gatewayAPI.enableAppProtocol=true

Restart the Cilium operator. The Gateway API controller comes up as part of the existing Cilium daemon set.

Declare the Gateway. This replaces what used to be the Ingress NGINX LoadBalancer Service plus the Ingress host blocks.

# gateway.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: public
  namespace: gateway-system
spec:
  gatewayClassName: cilium
  listeners:
    - name: https
      port: 443
      protocol: HTTPS
      hostname: '*.example.com'
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: wildcard-example-com
      allowedRoutes:
        namespaces:
          from: All

Now rewrite the per-app Ingress as an HTTPRoute.

# my-app-route.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-app
spec:
  parentRefs:
    - name: public
      namespace: gateway-system
  hostnames:
    - my-app.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: my-app
          port: 80

The Ingress resource and the HTTPRoute resource can coexist for the same host. Apply the route while the Ingress is still serving traffic. The Gateway API controller picks it up but does not become authoritative for the host until you redirect DNS or the LoadBalancer.

Three things the Gateway API does cleanly that Ingress did not. First, listener configuration (TLS, hostname, port) is a separate first-class resource, not a stack of annotations. Second, route attachment is cross-namespace by default, gated by allowedRoutes. Third, traffic-management features (weighted backends, header rewrites, request mirroring) are part of the spec, not vendor-specific annotations.

The Cilium-specific docs cover the eBPF datapath details and the Gateway API conformance matrix. The Cilium release notes confirm Gateway API GA in v1.15. If you are on a Cilium version older than that, upgrade Cilium first.

How do you cut over without downtime?

You cut over without downtime by running the old and new controllers side by side, swinging traffic at the DNS or LoadBalancer layer rather than inside the cluster, and watching the new controller for one full traffic cycle before scaling the old one down. The wrong order is to delete the old controller first.

Here is the sequence I used on my own cluster.

1. Install the new controller in its own namespace. Do not change any Ingress or IngressClassName yet.
2. Verify the new controller comes up healthy with a synthetic backend. A small echo deployment behind one test Ingress (or HTTPRoute) is enough.
3. For the first real workload, point a low-traffic DNS record at the new controller's LoadBalancer IP. Use a per-host record, not a wildcard, so the blast radius is bounded.
4. Watch for one full traffic cycle. At my scale that is 24 hours. For a high-traffic app it is one peak window.
5. If error rates and latency match the old controller, move the next host. Repeat.
6. When every host is on the new controller, drop the old LoadBalancer service and scale the controller to zero. Leave the deployment in place for a week as a rollback.
7. Delete the old namespace.

The DNS-level swing is the part that matters. Changing IngressClassName on a live resource also works, but it cuts traffic instantly. A DNS swing lets you bleed traffic over the TTL window, and if something is wrong with the new controller the rollback is a single record change.

For services that depend on the LoadBalancer source IP for security policy, the cutover needs a coordinated step where you tell the upstream firewall about both LoadBalancer IPs for the duration of the migration. I learned this the loud way when a third-party webhook started returning 403 from one provider after I moved their host. The provider's allowlist still pointed at the old LoadBalancer.

How do you verify the migration?

You verify the migration by running the conformance suite for your new controller, replaying a representative sample of production traffic, and watching golden signals during a full traffic cycle. If any of those three reveals a regression, do not delete the old controller yet.

For Gateway API specifically, the conformance suite is the first signal.

git clone --depth=1 https://github.com/kubernetes-sigs/gateway-api
cd gateway-api/conformance
go test ./... \
  -args -gateway-class=cilium \
  -conformance-profiles=GATEWAY-HTTP,GATEWAY-TLS

Every test that passes against your cluster certifies a piece of the spec that your config relies on. The full suite takes about ten minutes. Failures point at exactly which HTTPRoute feature your cluster does not handle.

For Traefik, the in-tree integration tests are the equivalent.

kubectl logs -n traefik-system deploy/traefik | grep -E "error|fail"

This is less rigorous than the Gateway API suite, but the Traefik controller is verbose about misconfigured Ingress resources. A clean log over 24 hours is the practical signal that your annotations translated correctly.

For traffic replay, the cleanest tool is goldpinger plus a TLS-aware HTTP probe.

# verify-route.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: verify-my-app
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: probe
          image: curlimages/curl:8.7.1
          command:
            - sh
            - -c
            - |
              for i in $(seq 1 100); do
                curl -fsS \
                  -H "Host: my-app.example.com" \
                  https://my-app.example.com/healthz \
                  || { echo "fail at $i"; exit 1; }
                sleep 1
              done

Run that against both the old and new LoadBalancer IPs while traffic is still going through the old controller. A divergence shows up fast.

The final check is the golden signals. Watch p99 latency, 4xx rate, and 5xx rate on the affected hosts for one full traffic cycle. Compare against the day before the cutover, not against the day of, because the day of will show DNS-propagation artifacts.

For deeper context on related infrastructure shifts, see the Postgres connection pool pgbouncer survival guide for the same pattern applied to database fronting, the SSH tunnel local database access guide for a non-cluster fallback when you need direct backend reach during the migration, and the Zero trust microservices with Spring Security for the auth-binding implications of the controller change.

For the original sources, see the Ingress NGINX retirement announcement, the SRC statement on the retirement, the Traefik drop-in migration guide, and the Cilium Gateway API documentation.

Keep Reading

• Postgres Connection Pool: pgbouncer Survival Guide. The same pattern of fronting a critical backend with the right adapter, applied to Postgres.
• SSH Tunnel for Local Database Access. The non-cluster fallback you reach for when you need direct backend reach during a controller migration.
• Zero Trust Microservices with Spring Security. The auth-binding model that has to follow the controller through the migration.

Quick-look format below. Each skill links to its source folder. What it does. What I use it for. Ranked by impact in real marketing work; parent-repo star counts listed for transparency.

What are Claude Skills for marketers?

A Claude Skill is a folder with a SKILL.md file that teaches Claude one specific marketing task. When you ask for the task, Claude loads the skill, follows the playbook, and runs the workflow.

The payoff is consistency. Same audit format every time. Same brief shape every time. Same brand voice on every output. Skills make Claude reliable on the boring parts so your attention goes to the parts that need a brain.

How do you install Claude Skills if you do not code?

You need Claude Code installed (free from claude.com/code) or a claude.ai web account. Three install paths:

# Path 1: Anthropic official skills
mkdir -p ~/.claude/skills
cd ~/.claude/skills
git clone --depth 1 https://github.com/anthropics/skills.git anthropic-skills
 
# Path 2: SEO skill suite (25 sub-skills, MIT license)
git clone --depth 1 https://github.com/AgriciDaniel/claude-seo.git
 
# Path 3: community skills index
# https://github.com/ComposioHQ/awesome-claude-skills

After install, restart your Claude Code session once. Skills auto-discover.

What are the 10 best Claude skills for marketers?

Ranked by impact in real marketing work. Parent-repo stars listed.

1. seo-audit

• Skill: AgriciDaniel/claude-seo · seo-audit (parent repo 7,194 stars)
• What it does: crawls up to 500 pages, detects business type, delegates to specialist sub-skills (technical, content, schema, images, sitemap, Core Web Vitals, GEO, backlinks, performance), generates a 0-100 health score.
• Use it for: any new client baseline. Replaces a multi-hour Screaming Frog + Lighthouse + schema-check sweep.

2. content-research-writer

• Skill: ComposioHQ/awesome-claude-skills · content-research-writer (parent repo 62k stars)
• What it does: turns a topic into a researched, sourced, publication-ready markdown article. Multi-pass WebSearch, primary-source collection, outline, section-by-section drafting with live feedback.
• Use it for: any 1500+ word post where research is the bottleneck. Replaces Frase briefs at zero subscription cost.

3. seo-content

• Skill: AgriciDaniel/claude-seo · seo-content (7,194 stars)
• What it does: scores a draft for E-E-A-T signals, readability, depth, thin-content patterns, first-person experience markers, and AI citation readiness.
• Use it for: pre-publish review. E-E-A-T is the biggest ranking factor in 2026 that most content tools cannot measure.

4. seo-geo

• Skill: AgriciDaniel/claude-seo · seo-geo (7,194 stars)
• What it does: scores AI-crawler accessibility (GPTBot, ClaudeBot, PerplexityBot), llms.txt compliance, passage-level citability, and platform-specific optimization for AI Overviews, ChatGPT, Perplexity, and Bing Copilot.
• Use it for: any post you want cited by AI search. The 2026 evolution of traditional SEO.

5. seo-schema

• Skill: AgriciDaniel/claude-seo · seo-schema (7,194 stars)
• What it does: detects, validates, and generates Schema.org structured data in JSON-LD. Article, Product, FAQPage, Breadcrumb, ItemList, and more.
• Use it for: any non-trivial page type (comparison, listicle, product, recipe). Schema is where rich snippets get won.

6. seo-page

• Skill: AgriciDaniel/claude-seo · seo-page (7,194 stars)
• What it does: analyzes one URL across on-page elements, content quality, technical meta, schema, and performance. Per-URL scorecard with prioritized fixes.
• Use it for: the page that should be ranking but is not. Tells you what to fix in the order that matters.

7. brand-guidelines

• Skill: anthropics/skills · brand-guidelines (parent repo 142k stars)
• What it does: turns a few brand inputs (palette, voice, examples) into a structured brand-guidelines doc and applies it consistently to every downstream output.
• Use it for: any post, email, social caption, or asset that should sound like your brand. Replaces the "paste a tone-of-voice deck in every prompt" workaround.

8. web-artifacts-builder

• Skill: anthropics/skills · web-artifacts-builder (142k stars)
• What it does: builds elaborate self-contained HTML artifacts (multi-component React + Tailwind + shadcn/ui) for landing pages, comparison tools, calculators, lead magnets.
• Use it for: the one-pager you would have hired a freelancer for. Ships in an afternoon.

9. higgsfield-generate

• Skill: higgsfield-ai/skills · higgsfield-generate (parent repo 318 stars)
• What it does: generates images via GPT Image 2 (technical diagrams), Nano Banana 2 (text-heavy), or Seedance (video). Marketing Studio mode handles ads with product imports + avatars.
• Use it for: cover images, social cards, marketing visuals. Replaces a small studio for ~$0.30 per image.

10. skill-creator

• Skill: anthropics/skills · skill-creator (142k stars)
• What it does: interviews you on trigger language, steps, and references, then writes a new SKILL.md in the right shape.
• Use it for: every repeatable marketing workflow (newsletter QA, link-build outreach, weekly report template). Build it once, reuse forever.

Which Claude skills should you install first?

Install in this order:

1. Install Claude Code from claude.com/code.
2. Clone AgriciDaniel/claude-seo into ~/.claude/skills/claude-seo/. Gives you the full SEO suite (audit, page, content, geo, schema, plus 20 more sub-skills).
3. Clone anthropics/skills into ~/.claude/skills/anthropic-skills/. Gives you brand-guidelines, web-artifacts-builder, and skill-creator.
4. Add content-research-writer from ComposioHQ/awesome-claude-skills for long-form research drafts.
5. Add higgsfield-ai/skills when your content output exceeds your photographer or stock budget.

The pattern that compounds: install one, use it for two weeks, install the next. The marketer who installs ten in one afternoon and uses none in week two saves zero hours. The one who installs one and uses it three times a week saves five hours by month two.

For more on building skills yourself, see the Anthropic engineering deep-dive on agent skills, the Claude Skills documentation, and the open standard at agentskills.io.

Keep Reading

• Claude Skills vs MCP vs Projects. The mental model for skill vs MCP server choice.
• Claude Code Routines: Async CI Automation Just Became Real. Wire skills into scheduled tasks for hands-off content ops.
• Vercel AI Gateway Deep Dive. One key for every LLM when your marketing stack needs to swap models.

I have written this mapping by hand more times than I can count, on Android apps and Ktor backends. This is the version that actually holds up: the right library, the annotations that matter, and the two or three things that bite everyone. And because typing out a 40-field class from a sample payload is soul-crushing, I will also show you the tool I built to skip that part entirely.

How do you convert JSON to a Kotlin data class?

You convert JSON to a Kotlin data class by defining a data class whose properties match the JSON keys, marking it @Serializable, and calling Json.decodeFromString. The official library for this is kotlinx.serialization, and it is the one I reach for on every new project.

Add the plugin and dependency to your Gradle build:

plugins {
    kotlin("plugin.serialization") version "2.1.0"
}
 
dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.7.3")
}

Then the class and the parse call:

import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
 
@Serializable
data class User(
    val id: Int,
    val name: String,
    val email: String,
    val active: Boolean
)
 
val json = """{"id":1,"name":"Rabi","email":"r@example.com","active":true}"""
val user = Json.decodeFromString<User>(json)

That is the whole happy path. The @Serializable annotation triggers the compiler plugin to generate the serializer at build time, so there is no runtime reflection, which is why kotlinx.serialization works on Kotlin Multiplatform and Android without extra config. Leave the annotation off and you get a compile error, not a runtime one, which is the good kind of failure.

Why kotlinx.serialization over Gson or Moshi?

Because kotlinx.serialization is the official Kotlin library, it understands Kotlin types like non-null and default values natively, while Gson and Moshi were built for Java and fake it. If you are starting fresh, this is the one to pick.

Here is the practical difference. Gson uses reflection and will happily construct a Kotlin object with null in a non-null property, because it bypasses your constructors. That gives you a value that the Kotlin type system swears cannot be null, right up until it crashes somewhere far from the parse. Moshi with its Kotlin codegen fixes that, and it is a fine choice on existing Android projects that already use it. But it is another dependency and another annotation processor.

kotlinx.serialization sidesteps all of it. It respects nullability, it respects default values, and it fails loudly at parse time when the JSON does not match, instead of handing you a broken object. On a new Kotlin codebase I do not think the comparison is close. Reach for kotlinx.serialization and move on.

How do you map snake_case JSON keys to Kotlin properties?

Use the @SerialName annotation to bind a JSON key to a differently named Kotlin property. Most APIs return snake_case, and you do not want that leaking into idiomatic Kotlin code.

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
 
@Serializable
data class UserProfile(
    val id: Int,
    @SerialName("full_name") val fullName: String,
    @SerialName("avatar_url") val avatarUrl: String,
    @SerialName("created_at") val createdAt: String
)

Now the serializer reads full_name from the JSON but your code uses fullName. This is the single most common thing people get wrong, because without it the parse throws a SerializationException complaining about a missing field, and the field is right there in the JSON. It is not missing. Its name just does not match.

If literally every key is snake_case, there is a shortcut. Configure the Json instance with a naming strategy instead of annotating every property:

import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonNamingStrategy
 
@OptIn(kotlinx.serialization.ExperimentalSerializationApi::class)
val json = Json {
    namingStrategy = JsonNamingStrategy.SnakeCase
}

I still prefer explicit @SerialName on public API models, because it documents the exact wire contract at the property. But for an all-snake_case API, the global strategy saves a lot of noise.

How do you handle nullable fields, defaults, and missing keys?

Match the JSON's optionality in the Kotlin type: nullable properties for keys that can be null, default values for keys that can be absent. These are two different problems and people conflate them constantly.

A key that is present but can hold null needs a nullable type:

@Serializable
data class Account(
    val id: Int,
    val nickname: String?,        // present, but may be null
    val plan: String = "free"     // may be absent entirely
)

The distinction that trips people up: a nullable type (String?) handles a key whose value is null. A default value handles a key that is missing from the JSON altogether. If a key might be absent and you do not give it a default, the parse fails. If it is present as null and you did not make it nullable, the parse also fails. They look similar and fail the same way, but the fix is different.

One more flag worth knowing. By default kotlinx.serialization rejects unknown keys, which is strict and safe. Real APIs add fields all the time, so on client models I usually relax that:

val json = Json {
    ignoreUnknownKeys = true
    coerceInputValues = true   // treats null on a non-null with a default as the default
}

ignoreUnknownKeys stops a new server field from breaking your app. coerceInputValues is the pragmatic escape hatch for APIs that send null where you expected a value with a default. Use them on client-side models. On server-side request bodies I keep strict parsing, because there I want the bad request rejected.

How do you map nested objects and arrays?

Model each nested JSON object as its own @Serializable data class, and map arrays to List. kotlinx.serialization handles the nesting automatically once the types line up.

@Serializable
data class Address(
    val city: String,
    @SerialName("zip_code") val zipCode: String
)
 
@Serializable
data class Customer(
    val id: Int,
    val name: String,
    val address: Address,          // nested object
    val tags: List<String>,        // array of primitives
    val orders: List<Order>        // array of objects
)
 
@Serializable
data class Order(val id: Int, val total: Double)

The rule is simple: every JSON object becomes a data class, every JSON array becomes a List of whatever the elements are. There is no special config for depth. A five-level-deep payload just means five data classes, each @Serializable. This is also where doing it by hand stops being fun, and where a generator earns its keep.

Is there a faster way than writing the class by hand?

Yes. Paste the JSON into a generator and it writes the entire data class for you, nested classes and all. This is exactly the tedium I got tired of, so I built jsontodto.com to do it.

You drop in a sample payload and it produces the matching Kotlin data class, with nested objects split into separate classes and arrays typed as List. You can toggle a plain data class or add kotlinx.serialization decorators, and it infers types from the values. For a big third-party API response, this turns a 20-minute typing session into a paste. The tool also outputs Java records, TypeScript interfaces, Python dataclasses, Go structs, and C# records from the same JSON, so it is handy across a polyglot stack, not just Kotlin.

My honest workflow: generate the class from a real sample, then hand-tune it. The generator cannot know that a field is nullable if the sample happened to have a value there, and it cannot know which keys are optional across all responses. So I paste, generate, then go through and add ? where the API can return null and defaults where keys can be absent. Generation gets you 90 percent of the way in one paste. The last 10 percent is the judgment this guide is about.

Converting JSON to a Kotlin data class is one of those tasks that is trivial until the real world shows up with snake_case keys, nullable everything, and objects nested five deep. Get the annotations right and kotlinx.serialization does the rest without complaint. Generate the boilerplate, then spend your actual attention on the nullability and optionality the generator cannot infer. That is where correctness lives.

For the full API, see the kotlinx.serialization JSON documentation and the official Kotlin serialization guide.

Keep Reading

• JSON to DTO Converter: Instantly Generate Java Classes from JSON. The Java side of the same tool, for records, Lombok, and Jackson DTOs.
• Java Libraries Beyond Lombok. More ways to cut boilerplate when you are back on the JVM side of a project.

Where they get in trouble is thinking the flag is the whole story. It quietly rewires how your web server handles requests, which executor runs your @Async methods, and how your Kafka listeners get their threads. And it exposes a bottleneck that was always there but never mattered before. This is the guide I give teammates who are about to flip the switch in production. If you want the deeper theory on what virtual threads are and how Project Loom works, I covered that in my breakdown of virtual threads in Java 25. Here I am focused on the Spring Boot side: what turning them on actually does, and what bites you.

How do you enable virtual threads in Spring Boot?

You enable virtual threads in Spring Boot by setting spring.threads.virtual.enabled=true and running on Java 21 or later. That is the entire setup.

In application.properties:

spring.threads.virtual.enabled=true

Or in application.yml:

spring:
  threads:
    virtual:
      enabled: true

This property has been available since Spring Boot 3.2, released in December 2023, and it carries forward unchanged into Spring Boot 4.x. The only hard requirement beyond the property is the runtime: virtual threads are a Java 21 feature, so on Java 17 or 20 the property does nothing useful.

I always pair it with an explicit check that the app is actually on 21+. A quick way is to log Runtime.version() at startup, or fail fast in a @PostConstruct if the major version is below 21. Silent no-ops are worse than crashes.

What does spring.threads.virtual.enabled actually turn on?

The property flips several auto-configurations at once, not just the web server. Understanding the full set is the difference between "I enabled virtual threads" and "I know where my code runs now."

Here is what Spring Boot wires up when the flag is on and you are on Java 21+:

• Web request handling. Tomcat and Jetty use virtual threads to process requests, so your controller methods run on a virtual thread per request instead of a pooled platform thread.
• The application task executor. The applicationTaskExecutor bean becomes a SimpleAsyncTaskExecutor that starts a new virtual thread per task. This is what backs @Async methods, Spring MVC async request processing, and WebFlux blocking execution support.
• The task scheduler. Scheduling is backed by a SimpleAsyncTaskScheduler on virtual threads, and it ignores pool-size properties because there is no pool to size.
• Messaging listeners. RabbitMQ and Kafka listener containers get virtual thread executors auto-configured.
• Data access helpers. Spring Data Redis' ClusterCommandExecutor runs on virtual threads.

The takeaway: this one property changes the execution model across the whole app, not just HTTP. If you have custom Executor beans wired by hand, they are not touched. Spring only swaps the executors it owns. So an app that defines its own ThreadPoolTaskExecutor for @Async keeps using platform threads there until you change it yourself.

Why is the database connection pool the real bottleneck?

Because virtual threads remove the thread limit but not the connection limit, so your JDBC pool becomes the ceiling the moment threads stop being scarce. This is the single most important thing to understand before enabling them.

The old model looked like this. Tomcat had 200 platform threads. Each request grabbed a thread, and if the thread pool was full, new requests queued. The thread pool was your concurrency limit, and it was usually smaller than your connection pool, so the pool rarely ran dry.

Flip on virtual threads and that changes. Now you can have 10,000 requests in flight, each on its own cheap virtual thread. But HikariCP still defaults to 10 database connections. So 10 requests run their query and the other 9,990 block waiting for a connection. You did not remove the wait. You moved it from "waiting for a thread" to "waiting for a connection," and the second one is easier to miss because everything looks healthy until latency spikes.

# Virtual threads let you accept far more concurrent requests.
spring.threads.virtual.enabled=true
 
# So the connection pool is now your real concurrency limit.
# Size it against your database's capacity, not your request rate.
spring.datasource.hikari.maximum-pool-size=50

There is no magic number here. The right pool size is bounded by what your database can handle, and adding virtual threads does not change that. What changes is that the pool, not the thread count, is now the thing you tune. I have seen teams enable virtual threads, see no improvement, and conclude the feature is overhyped. The feature was fine. Their 10-connection pool was the wall.

What is thread pinning and when should you worry about it?

Thread pinning is when a virtual thread cannot unmount from its carrier platform thread during a blocking call, which cancels the scalability benefit for that operation. It used to be a real concern with synchronized, and it is mostly gone now.

On Java 21 through 23, a virtual thread that blocks inside a synchronized block stays pinned to its carrier thread. If that block wraps an I/O call, like a database query guarded by a synchronized method, you lose the whole point: the carrier thread is stuck, and you are back to a bounded number of real threads.

Java 24 fixed this. JEP 491 reworked synchronization so virtual threads no longer pin on synchronized, which removes the most common pinning source for most apps. So the advice depends on your runtime:

• Java 24+: pinning from synchronized is no longer a concern. You can mostly stop worrying about it.
• Java 21 to 23: audit hot paths for synchronized around blocking I/O. Swap those specific locks to ReentrantLock, which lets the virtual thread unmount cleanly.

You do not need to rip out every synchronized in your codebase. A lock held for a quick in-memory update does not pin in any way that matters. The one to hunt for is a lock held across a network or disk call. To find them, run with -Djdk.tracePinnedThreads=full on Java 21 to 23 and watch the logs under load.

When should you not enable virtual threads?

Skip virtual threads when your workload is CPU-bound, because they solve a blocking problem, not a compute problem. This is the clearest "no."

Virtual threads shine when threads spend most of their time waiting: database calls, HTTP calls to other services, file I/O. If your service does heavy computation, image processing, large in-memory transforms, number crunching, virtual threads add nothing. You still have the same number of CPU cores, and a virtual thread doing math occupies a carrier thread the whole time just like a platform thread would. For that work, a sized ThreadPoolTaskExecutor matched to your core count is still the right tool.

A few other cases where I hold off:

• Heavy ThreadLocal usage. Each virtual thread carries its own ThreadLocal values. With millions of threads, a fat ThreadLocal cache turns into a memory problem. Audit what you stash there before scaling thread counts way up.
• Third-party libraries with internal thread pools. Enabling the Spring property does not change a library that manages its own executor. Check that your critical dependencies are Loom-friendly, not fighting it.
• You have not measured. If you do not know that thread starvation is your bottleneck, enabling virtual threads is a guess. Load test first, find the actual limit, then decide.

Enabling virtual threads in Spring Boot is not a performance cheat code, and treating it like one is how you end up disappointed. It is a scalability tool for I/O-bound concurrency, and it pays off exactly when blocked threads were what held you back. Flip the property, then go re-tune your connection pool and re-run your load test. That second step is where the actual throughput lives.

For the official details, see the Spring Boot task execution and scheduling reference and the Spring Boot 3.2 release notes where the property was introduced.

Keep Reading

• Virtual Threads in Java 25: The Complete Guide. The language-level deep dive on how virtual threads and Project Loom work under the hood.
• Modern Java with Spring Boot. How the newer Java features fit into a real Spring Boot service.

The piece that turns those raw row changes into clean, per-aggregate domain messages is the Outbox Event Router SMT. It is the single most useful transform Debezium ships, and it is also the one people misconfigure the most. This guide is the config reference I wish I had the first time I wired it up. If you need the pattern itself first (why the outbox table exists, how the transaction works), read my walkthrough of implementing the outbox pattern with CDC and come back here for the routing details.

What does the Debezium Outbox Event Router actually do?

The Debezium Outbox Event Router is a single message transform, class io.debezium.transforms.outbox.EventRouter, that reshapes raw outbox-table change events into clean messages on per-aggregate topics. It runs inside the Kafka Connect pipeline, after Debezium captures the insert and before the record hits Kafka.

Think of it as three jobs stacked together. It picks the destination topic from a column in the row. It sets the Kafka message key from another column. And it unwraps the event so the payload column becomes the message value instead of a nested change-event envelope.

By default it assumes your outbox table has these columns:

You are not locked into those names. Every one of them is remappable. But if you control the table schema, matching the defaults means less config to get wrong.

How do you add the EventRouter SMT to a connector?

You add it as a named transform in the connector config, then set the transform type to the EventRouter class. Everything else is options prefixed with that transform name.

Here is the minimum viable version on a Postgres connector, assuming default column names:

{
  "transforms": "outbox",
  "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
  "table.include.list": "public.outbox"
}

That is genuinely all you need to start. With no other options, the SMT reads aggregatetype, routes to outbox.event.<aggregatetype>, keys the message by aggregateid, and uses payload as the value. The table.include.list is not part of the SMT, but you want it: without scoping the connector to the outbox table, Debezium captures every table in the schema and the transform errors on rows that do not look like outbox events.

One thing that bites people immediately. The EventRouter expects to see the row that was inserted, so it needs the new-record state. If you are also chaining ExtractNewRecordState, order matters. Put the outbox transform first, because it already handles the unwrap itself. Stacking a second flattening transform in front of it usually strips the envelope fields the router needs.

How do you route events to per-aggregate topics?

Routing is controlled by three options that work as a small pipeline: read a field, match it with a regex, substitute it into a template. The defaults route by aggregate type into a predictable topic name.

• route.by.field (default aggregatetype): the column whose value drives the topic name.
• route.topic.regex (default (?<routedByValue>.*)): a regex with a named capture group that pulls the routing value out.
• route.topic.replacement (default outbox.event.${routedByValue}): the topic name template. The capture group name is substituted in.

So a row with aggregatetype = Order lands on outbox.event.Order. If you want a flatter naming scheme, override the replacement:

{
  "transforms.outbox.route.by.field": "aggregatetype",
  "transforms.outbox.route.topic.replacement": "domain.${routedByValue}.events"
}

Now Order goes to domain.Order.events. The regex is worth knowing about for one real case: if your aggregate type values carry a prefix you do not want in the topic name, capture only the part you want. A route.topic.regex of svc_(?<routedByValue>.*) turns svc_Order into a routing value of Order.

The message key is set separately through table.field.event.key (default aggregateid). This is the part that keeps per-entity ordering intact. Every event for the same order carries the same key, so Kafka lands them on the same partition and consumers see them in write order. If you leave the key column empty, you lose that guarantee, so treat aggregateid as required, not optional.

How do you add metadata as Kafka headers or envelope fields?

You expose extra columns with table.fields.additional.placement, which takes a comma-separated list of column:placement:alias entries. The placement is either header or envelope, and that choice changes where consumers read the value.

Say your outbox table has an eventtype column and a tracecontext column you want to propagate. You want the event type inside the message value and the trace context as a Kafka header:

{
  "transforms.outbox.table.fields.additional.placement":
    "type:envelope:eventType,tracecontext:header:traceparent"
}

Two rules I learned the hard way. header puts the value on the Kafka record header, which is perfect for cross-cutting metadata like trace ids that infrastructure reads without deserializing the body. envelope folds the value into the message value alongside the payload, which is what you want for domain fields a consumer actually maps into an object. Pick header for plumbing, envelope for data.

The alias (the third part) is the name the field or header gets on the output side. It is optional, but I always set it. Relying on the raw column name leaks your database schema into your event contract, and renaming the column later silently breaks every consumer.

How do you expand the JSON payload and handle deletes?

Two options handle the payload shape and the delete case: table.expand.json.payload and route.tombstone.on.empty.payload. Both default to false, and both are safe to leave off until you need them.

By default the payload column is passed through as-is. If you stored it as a JSON string, consumers receive a string and have to parse it themselves. Turn on expansion to get a real structured record:

{
  "transforms.outbox.table.expand.json.payload": "true"
}

With that set, the SMT parses the payload into a proper Kafka Connect struct, so downstream schemas and converters see typed fields instead of one big string. The catch is that it only works when the payload is valid JSON. Malformed content makes the transform fail the record, so this pairs best with a jsonb column that the database already validates on write.

Deletes are the other edge. The outbox table is usually append-only, so you rarely delete rows. But if you do (say a cleanup job prunes old events), Debezium emits a delete change event. Turn on route.tombstone.on.empty.payload to convert those into proper Kafka tombstone records (a null value on the aggregate key), which is what log-compacted topics expect for key removal. If your outbox topics are not compacted, leave it off.

What does a complete connector configuration look like?

Here is a full Postgres connector config with the Event Router doing real work: custom routing, header metadata, JSON expansion, and message keying. This is close to what I have shipped in production.

{
  "name": "outbox-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "postgres",
    "database.port": "5432",
    "database.user": "debezium",
    "database.dbname": "orders",
    "topic.prefix": "orders-svc",
    "table.include.list": "public.outbox",
    "tombstones.on.delete": "false",
    "transforms": "outbox",
    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
    "transforms.outbox.route.by.field": "aggregatetype",
    "transforms.outbox.route.topic.replacement": "domain.${routedByValue}.events",
    "transforms.outbox.table.field.event.key": "aggregateid",
    "transforms.outbox.table.field.event.payload": "payload",
    "transforms.outbox.table.expand.json.payload": "true",
    "transforms.outbox.table.fields.additional.placement":
      "type:envelope:eventType,tracecontext:header:traceparent"
  }
}

To verify it works, insert a row and watch the topic. A row with aggregatetype = Order and aggregateid = 42 should produce a message on domain.Order.events, keyed by 42, with the payload expanded into structured fields and a traceparent header attached. If the message lands on outbox.event.Order instead, your route.topic.replacement override is not being picked up, usually because the transform name in the property key does not match the one in the transforms list.

What are the common pitfalls with the Event Router?

Most Event Router problems trace back to a schema mismatch between the table and the config. Here are the ones I hit most, in rough order of how often they cost me an afternoon.

The connector captures more than the outbox table. Always set table.include.list to just the outbox table. Otherwise the transform sees regular business-table changes and throws because they lack aggregatetype.

The message key is null. Check that table.field.event.key points at a populated column. A null key destroys per-aggregate ordering, and it is easy to miss because messages still flow.

JSON expansion fails silently on bad data. If table.expand.json.payload is on and one row has a malformed payload, that record errors and can stall the connector. Validate at write time with a jsonb column, and consider a dead-letter queue on the connector.

Transform ordering with ExtractNewRecordState. The Event Router already unwraps the change event. Chaining a second flatten transform in front of it strips the fields it needs. If you must combine them, the outbox transform goes first.

Topic auto-creation is off. Per-aggregate routing means new aggregate types create new topics. If your broker disables auto topic creation, a brand new aggregate type produces messages that go nowhere until you create the topic. Pre-create them or enable Kafka Connect topic creation.

The Event Router is a small transform with a big payoff. Once the config matches your table, it disappears into the background and just produces clean domain events forever. The mistake I see teams make is treating it as an afterthought bolted onto the connector at the end. Design your outbox table columns around the router's defaults from day one, and the config shrinks to almost nothing.

For the full option list and version-specific behavior, see the Debezium Outbox Event Router documentation and the Debezium outbox quickstart on GitHub.

Keep Reading

• How to Implement the Outbox Pattern with CDC in Microservices. Start here for why the outbox table exists and how the transactional write works before you configure the router.
• Zero Trust Microservices with Spring Security. Once your services talk over Kafka topics, this covers securing the service-to-service boundary.

This guide walks through how to configure flake8 properly in 2026: what it actually is, where its config really lives, why pyproject.toml is the one file it refuses to read, and whether you should still reach for it now that Ruff exists. Everything here is checked against flake8 7.3.0, released June 2025.

What is flake8 and what does it actually check?

flake8 is not one linter. It is a wrapper that glues three separate tools together behind a single command, then adds a plugin system on top. When you run flake8, you are really running pyflakes, pycodestyle, and mccabe in one pass.

Each tool owns a code prefix, and knowing them makes every flake8 report readable:

• pyflakes finds logical errors: unused imports, undefined names, unused variables. Its codes start with F.
• pycodestyle checks PEP 8 style: whitespace, indentation, line length. It was once called pep8. Its codes start with E (errors) and W (warnings).
• mccabe measures cyclomatic complexity. It emits exactly one code, C901, and only when you turn it on.

flake8 adds one code of its own: E999, which means the file failed to compile at all (a syntax error). So when you see F401, that is pyflakes telling you about an unused import. E501 is pycodestyle complaining about line length. C901 is mccabe saying a function is too complex.

flake8 7.3.0 pins those dependencies tightly: pyflakes 3.4.x, pycodestyle 2.14.x, and mccabe 0.7.x. It runs on Python 3.9 and up. The project lives under PyCQA (the Python Code Quality Authority) and is maintained by Anthony Sottile and Ian Cordasco.

How do you install and run flake8?

Install it with pip and point it at your code. That is the whole starting workflow:

pip install flake8
flake8 .

That runs flake8 across the current directory. You can target a specific path or module too:

flake8 src/
python -m flake8 src/

The python -m flake8 form is worth remembering. It guarantees you run the flake8 installed in the current environment, which matters the moment you have more than one Python around. On a fresh run with no config, flake8 uses its defaults: 79-character lines and a small built-in ignore list. Most teams change both, which is where config comes in.

Where does flake8 look for its configuration?

flake8 reads three config files, all in INI format under a [flake8] section: setup.cfg, tox.ini, and a dedicated .flake8 file. It does not read anything else, and it does not merge them. It picks one.

I default to a standalone .flake8 file because it keeps linter config out of files that mean other things. Here is a realistic starting point, close to what the official docs recommend:

[flake8]
max-line-length = 88
extend-ignore = E203
exclude = .git,__pycache__,docs/source/conf.py,old,build,dist
max-complexity = 10
per-file-ignores =
    __init__.py: F401

A few of those options carry sharp edges, so here is what each one actually does:

• max-line-length defaults to 79. Setting it to 88 matches Black, which is why most projects use that number.
• extend-ignore adds codes to the ignore list without wiping the defaults. ignore (no extend-) replaces the entire default list, which usually is not what you want. Reach for the extend- forms.
• extend-select works the same way for turning checks on. It was added in flake8 4.0.0.
• exclude takes comma-separated paths and globs that flake8 skips entirely.
• per-file-ignores lets you relax rules per file. The __init__.py: F401 line above silences unused-import warnings in package init files, where re-exporting is normal.
• max-complexity turns on the mccabe check. It is off until you set it, so C901 never fires until you add a line like max-complexity = 10.

The extend-ignore = E203 in that example is there for a reason. E203 flags whitespace before a colon, which collides with how Black formats slices. Turning it off is the standard fix for running flake8 and Black together.

Why won't flake8 read your pyproject.toml?

flake8 does not read pyproject.toml, and that is a deliberate decision, not a missing feature. This is the single most common flake8 problem I see, and the answer surprises people: the maintainer has kept the request open for years and does not intend to add it as-is.

The relevant thread is PyCQA/flake8 issue #234, and it is still open. Anthony Sottile's position is direct: "I don't think toml is the way especially with pip continuing to have weird behaviour when the file even exists." His technical objection is real. The mere presence of a pyproject.toml file changes how pip builds a package, because it can force an isolated PEP 517 build. He listed two conditions that would need to hold before he would reconsider: pip no longer changing its behavior based on the file existing, and a standard-library TOML parser. Python 3.11 added tomllib, so half of that is now true, but flake8 7.3.0 still will not read the file.

So what do you do if your project standardizes on pyproject.toml and you want flake8 config to live there too? Use a plugin.

pip install Flake8-pyproject

Flake8-pyproject lets you put your config under a [tool.flake8] section and either keeps working with the normal flake8 command or gives you a flake8p entry point:

[tool.flake8]
max-line-length = 88
extend-ignore = ["E203"]
exclude = [".git", "__pycache__", "build", "dist"]

There is also pyproject-flake8 (the pflake8 command), which monkey-patches flake8 to read the file. Both work. I lean toward Flake8-pyproject because it is the lighter touch, but understand that either one is a workaround layered on top of a tool that, by design, still does not natively support the format.

How do you silence a specific warning inline?

Add a # noqa comment to the line, and be specific about which code you are silencing. flake8 gives you two forms, and the difference matters.

A bare # noqa silences every error on that line:

import os  # noqa

A coded # noqa: E501 silences only that check. Anything else on the line still gets reported, which is exactly what you want:

url = "https://example.com/a/very/long/path/that/breaks/the/line/limit"  # noqa: E501

You can list several codes with commas, like # noqa: E731,E123, and the comment is case-insensitive. I always prefer the coded form. A bare # noqa is a blunt instrument that hides real bugs the day someone adds a second problem to that line. If you ever need to audit what your # noqa comments are actually hiding, run with --disable-noqa and flake8 reports everything as if the comments were not there.

How do you run flake8 in pre-commit and CI?

Wire flake8 into pre-commit so it runs before code ever reaches a branch. The official hook is published in the flake8 repo, and the setup is a few lines in .pre-commit-config.yaml:

-   repo: https://github.com/pycqa/flake8
    rev: 7.3.0
    hooks:
    -   id: flake8

The docs show rev: '' as a placeholder, but always pin it to a real tag like 7.3.0. An unpinned linter version means your CI can start failing on a Tuesday because a new release added a check, and nobody changed a line of code. For CI itself, there is nothing special to configure. The same flake8 . command that runs locally runs in a GitHub Actions step, reads the same config file, and exits non-zero on any violation. If you are hardening a pipeline, the same discipline I described in enabling npm trusted publishing with OIDC applies here: pin your tool versions and let the config file be the single source of truth.

Which flake8 plugins are worth adding?

Plugins extend flake8 with new checks under their own code prefixes, and installing one is just a pip install into the same environment. Most start reporting immediately, no config required. You can confirm what loaded by running flake8 --version, which lists every active plugin.

The ones I reach for most:

• flake8-bugbear (prefix B) catches likely bugs and questionable design that pyflakes misses, like mutable default arguments.
• flake8-comprehensions (C4) flags list and dict comprehensions that could be simpler.
• flake8-docstrings (D) checks PEP 257 docstring conventions by wrapping pydocstyle.
• pep8-naming (N) enforces naming conventions for classes, functions, and variables.
• flake8-bandit adds security checks, and flake8-comprehensions and flake8-simplify clean up common code smells.

Once a plugin is installed, its codes behave like any other. Turn them on or off with select, ignore, extend-select, and extend-ignore in your config. That is the real strength of flake8: the checker set is yours to compose.

Should you still use flake8 in 2026?

Use flake8 if you depend on a plugin Ruff has not reimplemented. Otherwise, Ruff is probably the better default. That is the honest answer, and it comes straight from Ruff's own documentation rather than my opinion.

Ruff is a linter written in Rust that advertises being "10-100x faster than existing linters (like Flake8)." Its FAQ is refreshingly precise about compatibility. Ruff is a drop-in replacement for flake8 when you use it without many plugins, alongside Black, on Python 3 code. Under those conditions it "implements every rule in Flake8," including all the pyflakes F rules and a large subset of the E and W style rules.

But Ruff names its own limit clearly: "Ruff's primary limitation vis-a-vis Flake8 is that it does not support custom lint rules." That is the line that keeps flake8 alive. If your team relies on an internal flake8 plugin, or a niche community one Ruff has not ported, flake8 is still the tool that can run it. The same goes for a stable, working setup where migration churn buys you nothing. Speed is real, but a linter you have already tuned and trust has value too.

I have migrated projects to Ruff and kept others on flake8, and both calls were right for their context. The mistake is treating it as a moral question. Pick the tool that runs the checks you need, and let the benchmark difference decide only when the checks are equal.

For the details behind everything here, see the flake8 documentation, the still-open pyproject.toml discussion in issue #234, and Ruff's FAQ on flake8 compatibility.

Keep Reading

• How to Enable npm Trusted Publishing with GitHub Actions OIDC. Another piece of a hardened CI pipeline, where pinned tool versions matter just as much.
• How to Check Out Fork PRs Safely in actions/checkout v7. Running linters on untrusted pull requests without opening a security hole.
• How to Serve a React SPA from FastAPI with app.frontend(). More practical Python tooling, this time on the web-serving side.

The HTTP QUERY method gives you the thing GET and POST each refuse to: a safe, idempotent, cacheable request that carries a body. In this post I will walk through what the spec actually says, why GET-with-a-body and POST-for-reads are both broken, how caching works when the cache key lives in the request body, and which runtimes you can use it on today.

What is the HTTP QUERY method?

The HTTP QUERY method is a new request method, standardized in RFC 10008 in mid-June 2026, that sends a query as the request body while keeping the guarantees of GET. The spec's abstract puts it well: "A QUERY requests that the request target process the enclosed content in a safe and idempotent manner and then respond with the result of that processing."

The body is the query. Not the resource, not a command, a question. "The content of the request and its media type define the query." That media type matters: the spec says servers MUST fail the request if Content-Type is missing or inconsistent with the content. A successful query comes back as a plain 200 OK with the results in the response body.

Here is the comparison table straight from Section 1 of the RFC:

There is also a discovery mechanism. A server can advertise QUERY support with the new Accept-Query response header, a Structured Field listing the query media types it accepts for that path:

Accept-Query: "application/jsonpath", application/sql;charset="UTF-8"

The RFC was written by Julian Reschke (greenbytes), James M. Snell (Cloudflare), and Mike Bishop (Akamai), and it is a Proposed Standard on the IETF standards track. This is not a vendor experiment. It is the real thing.

Why isn't GET with a body enough?

Sending a body with GET is undefined behavior that some servers and intermediaries will actively reject. RFC 9110, the core HTTP semantics spec, is blunt about it in Section 9.3.1: "content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request, and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack."

So the standard advice has always been: put the query in the URL. That works until it does not. RFC 9110 only recommends that implementations support URIs of at least 8,000 octets, and that is a floor, not a promise. Real limits vary per proxy, per load balancer, per server, and you discover the smallest one in the chain at runtime. The spec for the 414 status code even names the workaround as the culprit: it happens "when a client has improperly converted a POST request to a GET request with long query information."

I have debugged exactly one 414 in my career and it cost me most of a day, because the request worked fine locally and only failed behind a corporate proxy with a tighter URI limit. That is the nature of this failure: it is environmental, and your test suite will not catch it.

RFC 10008's introduction lists two more problems with query-in-the-URL that I find underrated. First, URLs leak: "request URIs are more likely to be logged than request content and may also turn up in bookmarks." If your query contains anything sensitive, it is now in every access log along the path. Second, my favorite line in the whole spec: encoding queries into the URI "effectively casts every possible combination of query inputs as distinct resources." Your cache and your metrics now treat ?city=Berlin&limit=50 and ?limit=50&city=Berlin as different things, even though they are the same question.

Why not just use POST for queries?

POST works for queries right up until you need retries, caching, or honest semantics. The problem is not that POST fails. It is that POST tells every piece of HTTP infrastructure to assume the worst.

POST is not safe and not idempotent. When a connection drops mid-request, neither the client nor any proxy can know whether state changed on the server. So nothing retries automatically. Compare that with the QUERY abstract: QUERY requests "can be automatically repeated or restarted without concern for partial state changes." For genuinely unsafe operations you end up building retry safety by hand, which is exactly the machinery I covered in idempotency keys for REST APIs. Reads should never have needed that machinery in the first place.

And then there is caching. RFC 9110 Section 9.3.3 allows caching POST responses only when the response carries explicit freshness information plus a Content-Location that matches the target URI, and even then the cached response can only serve future GET and HEAD requests. It says directly: "a POST request cannot be satisfied by a cached POST response because POST is potentially unsafe." Two identical POST /search calls always hit your origin. Every dashboard that polls a search endpoint is doing full-price origin work on every tick.

QUERY flips all three defaults. Safe, so infrastructure can retry it. Idempotent, so retries need no bookkeeping. Cacheable, so identical questions can be answered without recomputing.

How does caching work for QUERY requests?

A QUERY response is cacheable, but the cache key must include the request content, not just the URI. That is the core trade the spec makes, and Section 2.7 is explicit: the cache key "MUST incorporate the request content" and related metadata.

This is genuinely harder than caching GET. The spec admits it: "caching QUERY method responses is inherently more complex than caching responses to GET, as complete reading of the request's content is needed in order to determine the cache key." A cache has to buffer the body before it can even decide whether it has seen this request before.

To avoid trivial misses, caches are allowed to normalize: they "MAY remove semantically insignificant differences from request content." The spec calls out stripping content encodings and using media type suffix knowledge, like treating anything ending in +json as JSON for normalization purposes. The flip side lands in the security section: normalize too aggressively and "normalization results in a false positive," meaning a cache can hand back the wrong query's results. If you build or configure a QUERY-aware cache, that is the failure mode to fear.

Conditional requests work too, through a clever indirection the spec calls the equivalent resource: a hypothetical GET-able resource derived by incorporating the request content into the target. Your ETag and If-None-Match validators operate against that, so a repeated query can come back 304 Not Modified.

Two response headers round out the model, and they are easy to confuse:

• Content-Location points at the query results, a URI you can plainly GET later to retrieve the same representation.
• Location points at the query itself, a URI that re-runs the query on GET without resending the body.

One redirect trap worth knowing: the historical browser behavior of rewriting a redirected POST into a GET does not apply here. The spec states the 301/302 exceptions for POST do not apply to QUERY. Only a 303 tells the client to fetch the result with GET.

Why was it named QUERY instead of SEARCH?

The spec spent its first six years named SEARCH and dropped the name to escape WebDAV baggage. The idea goes back to a 2015 individual draft (draft-snell-search-method), got revived by Asbjorn Ulsberg at the HTTP Workshop in 2019, and was adopted by the httpbis working group in March 2021, still called SEARCH.

The rename happened in November 2021. Appendix B of the RFC explains the reasoning with unusual candor. The IANA method registry already had three safe, idempotent methods: PROPFIND, REPORT, and SEARCH. All three come out of the WebDAV effort, "about which many have mixed feelings," and all use generic XML request formats. QUERY got a clean start, and as the appendix notes, the name "captures the relation with the URI's query component well."

From there it was a slow march: final draft in November 2025, IESG approval on November 20, 2025, and publication as RFC 10008 in mid-June 2026. Eleven years from first draft to standard. HTTP does not move fast, which is exactly why it is worth paying attention when it does move.

Who supports the HTTP QUERY method in 2026?

Node.js has parsed QUERY natively since early 2024, OpenAPI 3.2 can document it, and Spring's support is an open pull request. Here is the honest state of things as of July 2026:

A few of these deserve context. The Spring situation is the one I am watching closest as a Java developer: the original issue (#32975) sat for two years and was superseded by PR #34993, which the author marked ready for review the same week the RFC published. As of early July 2026 the RequestMethod enum still has no QUERY, so Spring apps cannot route it through @RequestMapping yet.

On the browser side, QUERY is not a CORS-safelisted method, and the RFC says plainly that a cross-origin QUERY "will require a preflight request." Budget for the extra OPTIONS round trip, and if preflights have bitten you before, my Spring Boot CORS guide covers how to configure them properly. MDN has no QUERY page yet and there is no caniuse entry, which tells you how early we still are.

And do not assume CDN support. Two of the RFC's three authors work at Cloudflare and Akamai, which bodes well, but I could not find any announcement from either that their edge caches handle QUERY today.

How can you try the QUERY method today?

curl plus Node 22 is enough to build a working QUERY endpoint right now, no dependencies required. curl sends any method you give it:

curl -X QUERY 'http://localhost:3000/contacts' \
  -H 'Content-Type: application/json' \
  -d '{"filter": {"city": "Berlin"}, "limit": 50}'

On the server side, Node's HTTP parser accepts QUERY natively, so req.method just works:

import { createServer } from 'node:http';
 
const server = createServer((req, res) => {
  if (req.method !== 'QUERY') {
    res.writeHead(405, { Allow: 'QUERY' });
    res.end();
    return;
  }
 
  let body = '';
  req.on('data', (chunk) => (body += chunk));
  req.on('end', () => {
    const query = JSON.parse(body);
    // Run the actual search here. The body IS the query.
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ query, results: [] }));
  });
});
 
server.listen(3000);

From a browser or any JavaScript runtime, fetch takes QUERY as a plain method string:

const res = await fetch('http://localhost:3000/contacts', {
  method: 'QUERY',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ filter: { city: 'Berlin' }, limit: 50 }),
});
 
const { results } = await res.json();

Remember the preflight: cross-origin, this fires an OPTIONS request first, every time, because QUERY is not safelisted.

If you are on Spring, you are stuck waiting or working around. Since RequestMethod does not know QUERY, requests with the method never reach your controllers through normal mapping. A servlet filter registered ahead of Spring MVC can intercept "QUERY".equals(request.getMethod()) and dispatch manually, but I would treat that as a stopgap and keep an eye on PR #34993 instead.

Should you adopt QUERY now?