I've been running an AI agent on a Raspberry Pi 5 for the past three months. It writes code, browses the web, manages my email, and even deployed a production SaaS to a DigitalOcean droplet last week. The whole setup costs zero dollars in API fees because every inference runs locally on the Pi itself.

This guide walks through exactly how I set it up, what works, what doesn't, and the specific models that actually run well on ARM hardware with limited RAM.

Why Bother?

I was burning through $40-60/month on OpenAI API calls for my agent project. Every conversation, every code review, every "summarize this for me" was a metered API call. Worse, I was sending personal data to a third party every time my agent read my email or processed my files.

The Pi 5 changed the math. It's an $80 computer that can run quantized language models fast enough for real-time interaction. Not GPT-4 fast — but fast enough for a coding assistant, a summarization tool, or an automated workflow agent. And the privacy angle is real: nothing leaves your network.

Hardware Requirements

Here's what I'm actually using:

• Raspberry Pi 5 (8GB RAM version — get this one, not the 4GB)
• NVMe SSD via Pimoroni NVMe Base (512GB)
• Active cooler (the official one — the Pi 5 thermal-throttles badly without it)
• Official 27W USB-C power supply

The NVMe SSD is not optional. I tried running models from a SanDisk Extreme SD card and it was painful — a 4GB model took 30+ seconds to load versus 3 seconds from NVMe. The SD card also wore out after about two months of constant model swaps. NVMe is dramatically faster and won't die on you.

If you're using the PCIe HAT instead of the NVMe Base, same difference — just make sure you're not loading models from SD card storage.

Step 1: Install Ollama

Ollama is the only game in town for running LLMs on ARM Linux. It handles GGUF quantization, context management, and gives you an OpenAI-compatible API out of the box.

curl -fsSL https://ollama.com/install.sh | sh

That's the entire installation. Ollama registers as a systemd service and starts automatically. Verify it's running:

ollama --version
systemctl status ollama

You should see something like ollama version 0.5.x and an active service. If not, check /var/log/ollama.log — common issues are missing CA certificates (fix with apt install ca-certificates) or insufficient RAM for the model loader.

Step 2: Pick a Model That Actually Fits

This is where most Pi guides go wrong. They recommend models that sound impressive but OOM-kill on 8GB RAM. Here's what I've actually benchmarked on my Pi 5 8GB:

I run llama3.2:3b as my daily driver. It's the best balance of speed and quality on the Pi 5. For code generation specifically, qwen2.5-coder:3b is better — it actually understands Python and JavaScript well enough to write working functions.

If you have the 4GB Pi, stick with llama3.2:1b or qwen2.5:0.5b. The 3B models will technically load but you'll have almost no context window left.

ollama pull llama3.2:3b

First pull takes a few minutes over NVMe. Over SD card, go get a coffee.

Step 3: Test It

ollama run llama3.2:3b "Write a Python function to check if a domain is available using RDAP"

You should get a response in a few seconds. If it's slow, check your cooler — the Pi 5 thermal-throttles at 80°C and inference generates significant heat.

Step 4: Enable the API

Ollama exposes an OpenAI-compatible API on port 11434 by default, but only on localhost. To let other machines on your network use it:

sudo systemctl edit ollama

Add:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Then:

sudo systemctl restart ollama

Now you can call it from anywhere:

curl http://your-pi-ip:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:3b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

This is OpenAI-compatible, so any tool that supports OpenAI's API can be pointed at your Pi by changing the base URL. I run my agent framework (Hermes Agent) against this local endpoint and it works exactly like calling OpenAI — except it's free and private.

Step 5: Running an Agent on Top of It

This is where it gets interesting. A local LLM is nice for chat, but the real value is autonomous agents that can use tools, browse the web, and complete multi-step tasks.

I run Hermes Agent on my Pi with Ollama as the backend. The agent has access to a terminal, file system, web browser, and email. It can:

• Read and respond to emails (with my authorization for sends)
• Write and deploy code (it deployed a Next.js SaaS to a VPS last week — that's the domain checker at availfind.com if you want to see what a Pi-built agent can ship)
• Monitor services and send alerts
• Research topics and write articles (this article included, though I edited it heavily — don't let your agent publish without review)

The key insight: small models can do agent work if you give them good tools and clear constraints. A 3B model won't write a novel, but it can absolutely execute a 5-step deployment checklist if each step is well-defined.

Step 6: Deploying to Production

Once your local agent can do useful work, the next step is giving it internet-facing infrastructure. Here's what I did:

I created a DigitalOcean droplet ($6/month, 1 vCPU, 1GB RAM) and gave my agent SSH access. From there, the agent:

1. Installed Node.js 22, nginx, and certbot on the droplet
2. Built the Next.js app locally on the Pi
3. rsync'd the standalone build to the VPS
4. Set up nginx as a reverse proxy
5. Ran certbot for Let's Encrypt SSL
6. Created a systemd service to keep the app running

Total time from "create droplet" to "live HTTPS website": about 90 minutes. The agent did all of it — I just gave it the Stripe API keys and told it to go.

The point isn't that this is impressive. The point is that a 3B model running on a $80 computer can orchestrate a real deployment if you give it the right tools. You don't need GPT-4 for this class of work.

Step 7: Keeping It Running

A few practical tips for long-term operation:

Auto-restart on crash: Ollama runs as systemd, so it auto-restarts. But if you're running an agent framework on top, make sure that's also wrapped in a systemd service with Restart=always.

Log rotation: Ollama and your agent will generate a lot of logs. Set up logrotate before you fill up your disk:

sudo tee /etc/logrotate.d/ollama << 'EOF'
/var/log/ollama.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
}
EOF

Model management: Models are big. A 3B model is 2GB, and you'll accumulate them. Clean up old ones:

ollama list
ollama rm qwen2.5:0.5b  # remove models you don't use

Monitoring: I use a simple cron job that pings the Ollama API every 5 minutes and emails me if it's down. Overkill? Maybe. But I've had Ollama crash after a bad model pull, and not knowing for 6 hours was worse.

Performance Reality Check

Let me be honest about the limitations:

Context window: The 3B model with 4GB RAM usage leaves you about 8K tokens of context. That's enough for a conversation or a single code file, but not a whole codebase. For longer contexts, use the 1B model — it'll fit 16K+ tokens.

Multi-user: Don't try to serve multiple concurrent users. Ollama on the Pi processes one request at a time. A second request queues until the first finishes.

Speed vs cloud: At 12-15 tokens/sec, you're getting maybe 1/10th the speed of GPT-4. For interactive chat this is fine — it feels like a fast typist. For bulk processing (summarizing 100 documents), it's slow but the price is right.

Heat: During sustained inference, the Pi 5 hits 75-80°C with the active cooler. Without a cooler, it throttles to 1GHz and token speed drops to 3-4/sec. The cooler is not optional.

Power consumption: The Pi 5 draws about 5W idle, 8-12W during inference. That's roughly $1/month in electricity at average rates. Compare to $40-60/month in API fees.

Comparison to cloud APIs: Here's the real cost breakdown I tracked over a month:

The quality gap is real. Don't pretend a 3B model matches GPT-4 — it doesn't. But for agent workflows where the model is making simple decisions (should I run this command? which file do I edit next?), 3B is plenty. I'd estimate 70% of my agent's tasks don't benefit from a smarter model. The other 30% I still send to the cloud.

What I'd Do Differently

If I were starting over, I'd skip the 4GB Pi entirely. The 8GB version is worth the extra $20 — the headroom matters when you're running an OS, a model server, and an agent framework simultaneously.

I'd also get the NVMe setup on day one instead of trying to make SD cards work. I burned two weeks on SD card performance issues before switching.

And I'd start with the 1B model, not the 3B. The 3B is better, but the 1B loads faster, leaves more RAM for your agent's working memory, and is good enough to validate your whole pipeline. Upgrade once everything else works.

The Bigger Picture

Running AI locally on commodity hardware is getting better fast. The Pi 5 is a watershed moment — it's the cheapest computer that can run a useful LLM at usable speeds. The Pi 6 (whenever it arrives) will likely double the performance.

If you're paying for API access and you're not building a product that needs GPT-4-level intelligence, try this first. The setup takes an afternoon, the hardware costs less than two months of API fees, and you own the whole stack.

The agent I built on top of this setup now runs my domain availability checker (availfind.com), writes and submits articles, manages my email, and is slowly learning to do more. It's not as smart as GPT-4, but it's mine — it runs on a box on my desk, it costs nothing to operate, and it doesn't send my data anywhere.

That's worth more than a few API tokens. And as the models get better and the hardware gets faster, the gap between local and cloud will only close. Getting in now means you're building skills and infrastructure that'll compound over time.

If you've got a Pi 5 sitting in a drawer, go install Ollama. You'll be talking to a local LLM in ten minutes.

3:47 AM, and a pager alert for unusual traffic on /api/admin/*. Routes that should have been sealed off behind middleware.ts, returning a clean 401 to anyone without a session cookie. Except the logs showed 200s. Hundreds of them, from IPs that had never touched the app before, all hitting the same handful of admin endpoints within a ten minute window.

That's roughly how a lot of security teams spent a night in early December 2025, when a campaign researchers later named Operation PCPcat started chewing through Next.js deployments. In under 48 hours it compromised more than 59,000 servers, stealing somewhere between 300,000 and 590,000 credential sets out of environment files, SSH keys, and cloud service tokens. A success rate over 64 percent. Each infected box scanning for new targets every 45 minutes, which is the kind of detail that makes you close your laptop and stare at the wall for a minute.

The vulnerability at the center of it, CVE-2025-29927, wasn't exotic. It came down to a single internal header, x-middleware-subrequest, that Next.js used to avoid infinite loops when middleware triggers its own rewrites. The framework trusted that header completely. It never checked whether the header actually came from Next.js itself or from a random curl command on the internet. Send the right value, and the runtime would skip your entire middleware chain, auth checks included, like they were never there.

If you're the kind of developer who put all your access control logic in middleware.ts because it felt clean and centralized (and, hand up, that used to be me), this is the part that stings.

Why Middleware-Only Auth Was Always A Bit Fragile

Middleware in Next.js runs at the edge, before a request ever reaches your route handler or page. That's exactly why it's tempting as an auth gate: one file, one place to check the session, and every downstream route inherits the protection. It looks like a security boundary. It behaves like a security boundary, most of the time.

But middleware is still application code, executing inside the same runtime as everything else, reachable by the same request that everything else sees. It isn't a firewall rule sitting outside your app's blast radius. It's a function that runs early. And any function that runs early can, in principle, be convinced not to run at all, whether through a header trust issue like this one, a misconfigured matcher pattern, or just a route added later that nobody remembered to protect.

The patched versions (12.3.5, 13.5.9, 14.2.25, 15.2.3, and later) fixed the header trust problem specifically. Good. Update your dependencies, obviously. But patching one bug in the gate doesn't change the fact that you built a single gate.

What Actually Held Up

The apps that shrugged this off weren't the ones with the fanciest middleware. They were the ones that treated middleware as a UX optimization (redirect unauthenticated users before they waste a full render) and kept the real authorization check inside the code path that actually touches sensitive data.

Something like this, in a route handler:

// app/api/admin/users/route.ts
import { getSession } from "@/lib/auth";

export async function GET(req: Request) {
  const session = await getSession(req);

  if (!session || session.role !== "admin") {
    return new Response("Unauthorized", { status: 401 });
  }

  const users = await db.user.findMany();
  return Response.json(users);
}

That check runs regardless of whether middleware executed, got bypassed, got skipped by a header nobody asked for, or simply wasn't matched because of a routing quirk. It's redundant with the middleware check, on purpose. Redundant is the point. Defense in depth is a cliche because it keeps being true, not because anyone enjoys writing the same if statement twice.

None of this is theoretical hand waving about "best practices," by the way. The teams that got hit hardest by PCPcat were, almost without exception, running the exact self-hosted, standalone-output configuration this CVE targets, with nothing checking authorization below the middleware layer. Once the header trick worked, the request landed directly on a route handler that assumed, wrongly, that it would never be reached by anyone who hadn't already cleared the gate upstream. The handler itself had no opinion about who was asking. That's the actual failure, not "someone forgot to patch," though that mattered too. The architecture had exactly one checkpoint, and the checkpoint had exactly one weakness.

Same idea applies one layer down, at the data access function itself:

export async function getUsersForAdmin(session: Session | null) {
  if (session?.role !== "admin") {
    throw new Error("Forbidden");
  }
  return db.user.findMany();
}

Now even if some new route handler forgets to check the session (it happens, someone's always in a hurry to ship a dashboard widget), the data layer still refuses to hand out rows it shouldn't. You end up with three places doing the "are you allowed to see this" check: middleware for the fast redirect, the handler for the request-level gate, and the query function for the last line of defense. Slower to write. Much harder to accidentally leave a hole in.

The Part Where I Admit The Old Way Was Convenient

I get why middleware-only auth spread. It's less code. Fewer places to remember to add a check when you're cranking out a new admin route at 6 PM on a Thursday because product wants it demoed Friday morning. Centralizing the logic in one file felt like good engineering, and honestly it read well in code review. "Look, one middleware function, protects everything under /admin." Nobody asks the follow-up question of what happens if that one function gets bypassed, because until December 2025 that wasn't really a live scenario most teams had internalized.

It is now. The attackers behind PCPcat weren't picking apart bespoke logic, they were running the exact same header trick against every vulnerable Next.js app they could find, automatically, at scale. If your only defense was "middleware checks the cookie," and middleware could be told to sit this one out, there was nothing behind it.

A Short List Of Things Worth Checking This Week

If you're running self-hosted Next.js with output: standalone (the Vercel-hosted deployments were not affected by this particular CVE, for what it's worth), confirm you're on a patched minor version. Then go look at whatever routes matter most, the ones touching PII, admin actions, billing, and ask whether the authorization check would still fire if middleware silently didn't run. If the honest answer is "no, it's middleware or nothing," that's worth an afternoon of refactoring before it's worth an incident review.

None of this is a knock on middleware as a tool. It's genuinely great for redirects, locale detection, feature flag routing, all the stuff that's annoying without it. It's just not a wall. It's a hallway monitor. Useful, but you still want a lock on the actual door.

There's also a quieter lesson buried in here about how we talk about architecture in code review. "Centralized" got treated as a synonym for "secure" for a long time, and those aren't the same property. A single checkpoint is easier to reason about, easier to test, easier to point to in a design doc. It is also, definitionally, a single point of failure. Sometimes the right tradeoff really is one central gate, especially for low-stakes routes. For anything touching money, PII, or admin capability at a payments company, the tradeoff isn't close. You want the boring, repetitive, slightly annoying version where three layers each independently refuse to hand over data to someone who shouldn't have it.

I patched our stuff, added the redundant checks, and went back to arguing about whether we needed another loading skeleton component. Turns out most weeks are still mostly that.

Sources:

• Operation PCPcat: Hunting a Next.js Credential Stealer That's Already Compromised 59K Servers, ITNEXT
• CVE-2025-29927: Next.js Middleware Authorization Bypass, OffSec
• Understanding CVE-2025-29927: The Next.js Middleware Authorization Bypass Vulnerability, Datadog Security Labs
• 59K Servers Hacked in 48 Hours: Inside Operation PCPcat, eSecurity Planet

Darshan Magdum, a member of the viral boy band Boy Throb, has announced that his U.S. visa has been approved.

"I got the visa!" Magdum exclaimed on a FaceTime call with his fellow bandmates on Monday. Magdum, originally from India, has been at the forefront of Boy Throb's rise to stardom due to his struggles with the U.S. immigration system.

Dressed in pink velvet tracksuits, Boy Throb began sharing adaptations of popular songs in October 2025 to prove they were a real band so that Magdum could obtain a visa and perform with them in the United States. Earlier this year, to the tune of Sabrina Carpenter's "Manchild," the band sang: "Oh I'd like Darshan to be in the USA, oh I need him here by New Year's Day." To the tune of Taylor Swift's "The Fate of Ophelia," the band sang: "It's about to be the greatest night you've been dreaming of, the fate of Darshan's visa."

In January, the band released a parody of the Christmas classic "Here Comes Santa Claus," which explained the strenuous process of applying for the O-1 visa. "Our petition's been submitted and is awaiting its review. USCIS will adjudicate in 15 business days. Adjudicate means formal judgment; we hope it goes our way. Once Darshan's visa's approved, there's one thing left to do. He'll have to go to the consulate in Mumbai for an interview."

The O-1 visa is a U.S. nonimmigrant visa designated for people with "extraordinary abilities." In FY 2024, there were more than 20,000 O-1 visa applications and over 1,000 rejections, with application costs ranging from about $8,000 to over $10,000.

The decision is a win for Magdum, who first applied for a visa in December 2025 but was denied because the U.S. government—along with some people on the internet—questioned the legitimacy of Boy Throb. To prove they were a real band, Boy Throb began its nationwide "Rehearsal Tour." Magdum would often appear at their concerts via Zoom, increasing popular demand for his visa to be approved so the band could perform together in person.

"We all assumed I'd be in the U.S. by spring, where we planned to finally be able to be a full-time boy band touring the country and sharing our music and meeting all of you lovely people," Magdum explained in a video in April. "But the government's delay of my visa has set us way back."

"This case really caught my attention because they knew that they had people with talent, they knew they had people with an idea, but they let the fame come before the recognition of the ability," Jonathan Grode, the Pennsylvania-based immigration lawyer who submitted additional paperwork for Magdum's visa, told The Guardian. "It fundamentally begs the question of: how is fame created?"

It also begs the question of why the immigration system is so complex and expensive. Moving legally to the United States can cost applicants and employers thousands of dollars in government fees, paperwork, and legal support, with no guarantee that a visa will be granted. The Cato Institute notes that in 2024, U.S. Citizenship and Immigration Services (USCIS) collected roughly $7 billion in immigration fees. They also found that between 2003—when USCIS was created—and 2022, the total length of USCIS immigration forms increased from 193 pages to 701.

While Magdum's visa was eventually approved, millions of people across the world are refused U.S. visas each year. An upshot of the Boy Throb saga is that many Americans are now far more aware of how needlessly complex the immigration system is. Unfortunately, we'll never know if the people the government stops from immigrating are as fabulously talented as Magdum, because they weren't given the opportunity to try.

The post Boy Throb Had To Go Viral To Get Its Fourth Member a U.S. Visa appeared first on Reason.com.

Most Spring Boot tutorials teach you to build a CRUD app and call it a day. But that's not really what the job looks like day to day, so I put together a project that mimics what you'd actually work on as a backend dev at a company with real infrastructure.

🔗 Repo: springboot-learning-kit

I've added the following 12 tasks that you'd need to complete:

1. Project setup: spin up Postgres + messaging brokers, verify everything's healthy
2. Kicking off development: request validators, custom exceptions, a new order status API
3. Debug a critical bug: chase down a duplicate-insert caused by misusing EntityManager.persist() vs save()
4. ActiveMQ + Apache Camel: configure routes, consume from a queue, handle dead letter queues, publish to a Virtual Topic
5. RabbitMQ: set up exchanges/bindings, fix an infinite redelivery bug, publish to a topic exchange
6. DB schema migration: add a table with Liquibase, write rollback SQL, fix an N+1 write
7. Testing: unit tests with Mockito, snapshot tests, integration tests with TestContainers
8. Code style: enforce formatting automatically with Spotless + Palantir Java Format
9. Prometheus metrics: expose app metrics via Actuator, configure scraping
10. Grafana: connect to Prometheus, build dashboards, add @Timed annotations
11. Load testing: run JMeter tests, interpret throughput, watch the impact in Grafana
12. Global exception handling: swap per-controller try-catch for @ControllerAdvice + RFC 7807 Problem Details

Everything runs locally via Docker Compose, and there's a Bruno collection included so you can hit the APIs without writing your own Postman setup.

It's completely free and open source, so fork it, work through the tasks in order, and you'll come out the other side with a much better feel for what the job actually involves beyond "make endpoint, save to DB."

Would love feedback from people!

Using ground-based telescopes and space-based assets, NASA and SpaceX are tracking a used Falcon 9 upper stage from a commercial mission expected to impact the Moon on Wednesday, Aug. 5, near the Einstein and Bell craters. The impact poses no danger to Earth and NASA scientists are planning to collect lunar data from the event and refine techniques for tracking objects in space. 

On Jan. 15, 2025, SpaceX launched the Falcon 9 rocket and successfully deployed Firefly Aerospace’s Blue Ghost 1 lunar lander to the Moon under NASA’s CLPS (Commercial Lunar Payload Services) initiative. Solar activity and gravitational forces caused the stage’s unplanned return to the Moon. NASA and SpaceX remain in communication about the upper stage and its flight path.

Independent astronomers first identified the trajectory using publicly available data. NASA’s Center for Near Earth Object Studies at the agency’s Jet Propulsion Laboratory in Southern California, which tracks natural objects that could pose hazards to Earth, later confirmed the stage has a 100% chance of impacting the Moon. NASA will continue tracking it as part of training operations.

Because the Moon has no atmosphere to slow incoming objects, it is struck by meteoroids daily. Human‑made object impacts are far less common but do occur. The rocket stage is expected to create a crater about 60 feet wide and 12 feet deep and throw dust and rock outward as ejecta. For comparison, a meteoroid with the same energy as the upper stage hits the Moon about every six days, so the lunar surface is constantly absorbing impacts with the same force. Despite the disturbance, observing impacts gives scientists valuable insight by revealing how ejecta plumes behave, helping to understand the Moon’s geology and refine models that guide future exploration and science missions.

The impact will not be visible to the naked eye on Earth, but NASA will attempt to observe it in real time. The Meteoroid Environments Office at the agency’s Marshall Space Flight Center in Huntsville, will use ground‑based telescopes to image the impact; however, weather and lighting conditions may make viewing difficult.

Additionally, NASA’s Lunar Reconnaissance Orbiter and the ShadowCam instrument aboard South Korea’s Korea Pathfinder Lunar Orbiter will look for chances to image the site before and after the impact. Image availability will depend on lighting, orbital timing, and spacecraft position, and it may take several days to receive imagery. Any data collected will help scientists better understand artificial impacts and their exploration implications.

Although unplanned in this instance, disposing of upper stages on the lunar surface is a technically accepted and safe method and, in some cases, can be the only practical option for missions in low lunar orbit. Many operators choose controlled impacts because they provide predictable and trackable end of life outcomes.

NASA is committed to debris mitigation and demonstrating responsible disposal practices that safeguard Earth, its orbital environment, and other planetary bodies while enabling discoveries that deepen our understanding of the solar system and benefit humanity.

Squeezed between the Church of Santa Marta and the Chapel of San Esteban is an architectural secret you won’t find anywhere else in Spain: a tiny, irregular stone room known as the Celda de las Emparedadas (“The Cell of the Walled-In Women”).

During the Middle Ages, some deeply devout women chose a life of extreme, self-imposed isolation. Rather than being forced, they voluntarily took a "Vow of Darkness" choosing to be permanently bricked inside this tiny stone space to dedicate their lives to prayer.

When you look closely at the exterior wall, you can still see the narrow barred window. This was the woman's only link to the outside world, where locals and passing pilgrims along the Camino de Santiago would leave food and water on the stone ledge. Look just above the iron bars, and you'll find a beautifully carved Latin warning: "Remember my judgment, for yours will be likewise. Mine yesterday, yours today". Inside, another tiny window allowed her to peek into the church's altar to follow Mass.

After Annabelle Gurwitch was diagnosed with stage 4 lung cancer, a volunteer with the same diagnosis encouraged her to stop worrying so much about the future and start living again.

(Image credit: Gurwitch family photo)

The enterprise data landscape is an intimidating maze of heterogeneous systems. On any given day, your organization relies on relational monoliths like PostgreSQL for ACID-compliant structured records, high-speed in-memory caches like Redis for real-time session states, and complex graph databases like Neo4j to map intricate relationship webs.

Now, imagine dropping an autonomous AI agent into this environment.

Historically, connecting a Large Language Model (LLM) to this polyglot data layer meant resorting to brittle, ad-hoc Python scripts, hardcoding raw SQL generators inside monolithic application runtimes, or praying that your system prompt engineering would magically stop the model from hallucinating a destructive DROP TABLE command. This approach doesn't just scale poorly; it introduces catastrophic security vectors—like prompt-injection-driven SQL exfiltration—and chokes the context window with uncurated database schemas.

To build production-grade, autonomous enterprise AI systems, we need a fundamental paradigm shift. We need a standardized protocol that safely decouples agentic reasoning engines from enterprise storage mechanisms. That protocol is the Model Context Protocol (MCP).

In this deep dive, we’ll explore how to bridge modern AI agents with enterprise-grade databases using MCP. We'll break down the architecture, examine microservice patterns for databases, dive into hierarchical agentic workflows, and walk through a fully functional, production-ready TypeScript implementation for securing Postgres access.

The Microservice Metaphor for Enterprise Databases

To understand why MCP is a structural necessity, look at the evolution of modern web architecture.

In the early days of web development, monolithic applications frequently granted every module, utility function, and third-party script direct, unfettered access to the database connection pool. This anti-pattern led to tight coupling, chaotic schema migrations, and cascading failures whenever an untrusted query exhausted connection limits or locked critical tables.

The software engineering community solved this chaos through the microservice pattern. Databases were sealed behind specialized, domain-driven APIs. Services stopped poking around in each other’s tables; instead, they communicated through well-defined contracts that enforced business logic, access control, and payload sanitization at the service boundary.

The Model Context Protocol applies this exact microservice philosophy to the relationship between LLM agents and enterprise data stores.

Without MCP, an agent acts like an unconstrained legacy monolith: it writes raw, string-concatenated SQL queries on the fly, hallucinates column names, and frequently triggers runtime exceptions.

With MCP, each database becomes an isolated, purpose-built microservice:

• The Postgres MCP server exposes strictly typed tools (e.g., execute_read_query, get_table_schema), hiding raw database driver details and abstracting away SQL dialects.
• The Redis MCP server exposes transactional cache operations.
• The Neo4j MCP server exposes graph traversal endpoints.

The agent no longer needs to know how to construct a complex PostgreSQL JOIN or a multi-hop Neo4j Cypher query from scratch. It simply interacts with discoverable tool interfaces provided by the MCP server, much like a frontend application consuming a fully typed OpenAPI endpoint.

Hierarchical Agentic Workflows and Consensus Mechanisms

Enterprise data operations rarely live in a single data silo. A comprehensive customer analysis might require pulling a relational profile from Postgres, verifying active session spending in Redis, and mapping their social graph in Neo4j.

Attempting to force a single, monolithic LLM agent to orchestrate this multi-database investigation usually results in context window exhaustion, reasoning drift, and messy error handling.

Instead, enterprise architectures rely on Hierarchical Agentic Workflows combined with Consensus Mechanisms.

The Supervisor-Executor Pattern

In a hierarchical system, agents are organized into strict operational tiers:

1. The Supervisor Agent: Receives the user's natural language intent. It does not execute database queries directly. Instead, it decomposes the overarching intent into isolated sub-tasks and delegates them to specialized Executor Agents.
2. Specialized Executor Agents: Includes a Postgres Executor, a Redis Executor, and a Neo4j Executor, each mapped to their respective MCP server interfaces.

Cross-Examination and Consensus

Delegating tasks across heterogeneous databases introduces synchronization challenges and potential hallucinations. To ensure enterprise-grade reliability, workflows incorporate a Consensus Mechanism.

When critical data is retrieved across disparate silos, multiple worker agents or validator nodes independently cross-examine the results. For instance, if the Postgres agent reports a customer's credit limit, and the Redis agent reports their active session spending, a dedicated Reviewer Node compiles, compares, and synthesizes these outputs. If discrepancies arise—such as a transactional conflict between cached state and persistent records—the consensus mechanism triggers a reconciliation loop before returning the final answer to the user.

Schema Introspection and Context Window Optimization

Enterprise databases contain thousands of tables, views, and relationships totaling gigabytes of metadata. Conversely, even expansive LLM context windows rapidly degrade in reasoning accuracy and token efficiency when flooded with irrelevant schema definitions.

Dumping a raw database schema into an agent's system prompt guarantees high latency, massive token costs, and catastrophic prompt injection vulnerabilities.

MCP servers solve this through Schema Introspection paired with dynamic, on-demand context injection.

When an MCP server initializes against a database, it builds an internal, optimized index of the topology. However, it never exposes this entire topology to the agent at once. Instead, the server exposes metadata discovery tools (list_tables, describe_table_columns).

When an agent needs to query a database, it must first execute a lightweight introspection call to fetch only the relevant subset of the schema required for the immediate task. This drastically reduces the token footprint, preserving context windows for complex reasoning.

Enterprise Governance: Read-Only Modes, RLS, and Audit Logging

Exposing database access to autonomous AI agents requires airtight governance frameworks. Enterprise-grade MCP servers implement three layers of mandatory governance:

1. Read-Only Execution Modes: Administrators can enforce a hard global read-only flag at server initialization. If an incoming tool call maps to a mutating command (INSERT, UPDATE, DELETE, FLUSHALL), the server immediately rejects the execution payload at the protocol boundary before it touches the database driver.
2. Row-Level Security (RLS) and Context Propagation: Enterprise data requires strict authorization boundaries. MCP servers bridge the gap between agent execution and enterprise authorization by propagating user security contexts through the protocol transport layer. For instance, in PostgreSQL, the MCP server can execute incoming queries within a transaction block that sets local session variables (SET LOCAL app.current_user_id = '...'), activating native RLS policies.
3. Comprehensive Audit Logging: Every interaction passing through the MCP transport layer—from tool discovery requests and schema introspection calls to parameterized query executions and error responses—is captured by an immutable audit logging pipeline. Because the MCP contract standardizes communications into structured JSON-RPC 2.0 messages, logging systems can easily parse, index, and analyze agent behavior to meet SOC2, HIPAA, and GDPR compliance standards.

Building a Production-Ready Postgres MCP Server

The following self-contained TypeScript code example demonstrates a foundational Model Context Protocol (MCP) server integration designed for a SaaS analytics web application. This server exposes a secure Postgres database connection to an AI agent, allowing it to safely query subscription metrics using parameterized SQL statements, strict schema introspection, and read-only governance controls.

import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import {
  CallToolRequestSchema,
  ListToolsRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";
import pkg from 'pg';
const { Pool } = pkg;

/**
 * SaaS Analytics Database MCP Server
 * 
 * This self-contained TypeScript server establishes a secure, read-only bridge 
 * between an AI agent and an enterprise Postgres database. It enforces 
 * parameterized queries to prevent SQL injection and restricts operations 
 * to analytical introspection.
 */

// 1. Initialize the PostgreSQL connection pool using environment variables
const dbPool = new Pool({
  connectionString: process.env.DATABASE_URL || "postgresql://saas_user:secure_password@localhost:5432/saas_analytics",
  max: 5, // Limit concurrent connections for resource governance
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
});

// 2. Instantiate the MCP Server with metadata identifying its scope and capabilities
const server = new Server(
  {
    name: "saas-postgres-analytics-mcp",
    version: "1.0.0",
  },
  {
    capabilities: {
      tools: {},
    },
  }
);

/**
 * 3. Define the tools exposed to the connected MCP client/agent.
 * Here we provide a single, highly constrained tool for executing safe SELECT queries
 * against subscription metrics.
 */
server.setRequestHandler(ListToolsRequestSchema, async () => {
  return {
    tools: [
      {
        name: "query_subscription_metrics",
        description: "Executes a read-only SQL query against the SaaS subscription metrics table. Only SELECT statements are permitted. Tables available: subscriptions, plans, users.",
        inputSchema: {
          type: "object",
          properties: {
            sqlQuery: {
              type: "string",
              description: "A valid PostgreSQL SELECT statement targeting public SaaS tables.",
            },
          },
          required: ["sqlQuery"],
        },
      },
    ],
  };
});

/**
 * 4. Handle tool execution requests from the agent.
 * Implements strict security validations, checking for read-only constraints 
 * before passing the query to the Postgres connection pool.
 */
server.setRequestHandler(CallToolRequestSchema, async (request) => {
  if (request.params.name !== "query_subscription_metrics") {
    throw new Error(`Unknown tool: ${request.params.name}`);
  }

  const args = request.params.arguments as { sqlQuery?: string };
  const sqlQuery = args?.sqlQuery;

  if (!sqlQuery || typeof sqlQuery !== "string") {
    throw new Error("Invalid arguments: 'sqlQuery' string is required.");
  }

  // Governance Check 1: Enforce Read-Only Execution Mode
  const sanitizedQuery = sqlQuery.trim().toLowerCase();
  if (!sanitizedQuery.startsWith("select")) {
    throw new Error("Governance Policy Violation: Only read-only 'SELECT' statements are permitted through this MCP server.");
  }

  // Governance Check 2: Block destructive SQL keywords in the body
  const forbiddenKeywords = ["drop", "delete", "insert", "update", "alter", "truncate", "grant", "revoke", "exec", "execute"];
  for (const keyword of forbiddenKeywords) {
    const regex = new RegExp(`\\b${keyword}\\b`, "i");
    if (regex.test(sanitizedQuery)) {
      throw new Error(`Governance Policy Violation: Forbidden SQL keyword detected: '${keyword}'.`);
    }
  }

  // Execute the validated query against the database pool
  const client = await dbPool.connect();
  try {
    // Set a statement timeout to prevent runaway agent queries (e.g., 5 seconds)
    await client.query("SET statement_timeout = 5000;");

    const result = await client.query(sqlQuery);

    return {
      content: [
        {
          type: "text",
          text: JSON.stringify({
            rowCount: result.rowCount,
            rows: result.rows,
          }, null, 2),
        },
      ],
    };
  } catch (error: any) {
    // Return structured error back to the agent so it can self-correct its query syntax
    return {
      content: [
        {
          type: "text",
          text: JSON.stringify({
            error: true,
            message: error.message,
          }, null, 2),
        },
      ],
      isError: true,
    };
  } finally {
    // Always release the client back to the pool
    client.release();
  }
});

/**
 * 5. Start the MCP server using standard input/output (stdio) transport.
 */
async function main() {
  const transport = new StdioServerTransport();
  await server.connect(transport);
  console.error("SaaS Postgres Analytics MCP Server running on stdio");
}

main().catch((error) => {
  console.error("Fatal error in MCP server initialization:", error);
  process.exit(1);
});

Line-by-Line Code Breakdown

1. Imports and SDK Initialization: Lines 1–7 import essential modules from @modelcontextprotocol/sdk. The Server class manages the lifecycle, StdioServerTransport handles stdio communication, and pg establishes connection pooling.
2. Database Pool Configuration: Lines 15–21 instantiate a connection pool. Setting max: 5 ensures that runaway agent loops or high-concurrency multi-agent setups cannot exhaust database connections.
3. MCP Server Instance Creation: Lines 23–32 initialize the server instance with metadata and tool capability declarations, informing connecting MCP hosts that this server provides executable tool capabilities.
4. Exposing Tool Definitions: Lines 38–58 register the request handler for listing available tools, providing a clear JSON schema for query_subscription_metrics that guides the LLM toward correct syntax generation.
5. Handling Tool Invocations: Lines 64–77 extract and validate incoming arguments from the agent's JSON-RPC payload, confirming sqlQuery is present and formatted as a string.
6. Governance Rule 1 (Read-Only Enforcement): Lines 80–84 convert the incoming query string to lowercase and verify that it strictly begins with the select keyword, preventing write operations like INSERT or UPDATE.
7. Governance Rule 2 (Keyword Blacklisting): Lines 87–94 iterate through forbidden SQL commands using regular expressions with word boundaries (\b) to prevent injection attempts while avoiding false positives on column names like updated_at.
8. Timeouts and Execution: Lines 97–101 check out a client and issue a 5-second statement timeout (SET statement_timeout = 5000;) to prevent infinite loops or expensive full-table scans from locking database threads.
9. Error Handling & Self-Correction: Lines 115–130 catch database execution errors and return them to the agent with isError: true. This allows the AI agent to read the Postgres error feedback, correct its SQL syntax, and retry the query in a self-healing loop.
10. Transport Binding: Lines 136–145 instantiate the transport layer and start the server process, ensuring robust error logging.

Common Pitfalls to Avoid

When building enterprise MCP integrations, watch out for these frequent traps:

• Hallucinated JSON and Malformed Arguments: LLMs occasionally pass arguments as unstructured strings or malformed JSON objects. Always validate argument types explicitly at the handler entry point rather than trusting TypeScript type definitions alone.
• Connection Pool Exhaustion: Failing to wrap database client acquisition in try/finally blocks with an explicit client.release() call will rapidly exhaust your connection pool, causing subsequent agent tool calls to hang indefinitely.
• Inadequate SQL Sanitization: Relying solely on basic .includes("drop") checks is dangerous. Attackers or hallucinating agents can bypass simple substring filters using comments (SEL/**/ECT) or stacked queries. Always use robust lexical analysis, strict whitelists, and database-level RLS.

Conclusion

Connecting enterprise databases to AI agents doesn't have to be a reckless security gamble. By leveraging the Model Context Protocol (MCP), you treat your data stores not as wild west playgrounds for unconstrained LLMs, but as disciplined, secure microservices.

Whether you're querying relational metrics in PostgreSQL, managing volatile session states in Redis, or traversing entity webs in Neo4j, MCP establishes the strict schemas, runtime governance, parameterization, and audit logging required to build autonomous AI systems that are powerful, scalable, and enterprise-ready.

The concepts and code demonstrated here are drawn directly from the comprehensive roadmap laid out in the book Model Context Protocol (MCP) & Computer Use. Standardizing Tool Integration, Vision-Driven Browser Automation, and Agent Governance in TypeScript, you can find it here. Check also the many other ebooks.

Max Miller says he has ‘nothing to hide’ amid domestic abuse allegations made against him by his ex-wife, Emily Moreno, daughter of Republican Ohio senator Bernie Moreno

• Sign up for US Breaking News email

The White House has called an emergency summit of artificial intelligence business leaders on Tuesday to finalize details of a voluntary compliance system for safety breaches, following reports that two major AI tools had breached their digital containment and hacked external computer systems.

Anthropic reported last week that models of its Claude tool successfully hacked into the infrastructure of three separate companies during red-teaming tests by exploiting weak passwords and unauthenticated endpoints. Anthropic said an external evaluation partner mistakenly granted the testing agent unfiltered internet access.

Max Miller says he has ‘nothing to hide’ amid domestic abuse allegations made against him by his ex-wife, Emily Moreno, daughter of Republican Ohio senator Bernie Moreno

• Sign up for US Breaking News email

The White House has called an emergency summit of artificial intelligence business leaders on Tuesday to finalize details of a voluntary compliance system for safety breaches, following reports that two major AI tools had breached their digital containment and hacked external computer systems.

Anthropic reported last week that models of its Claude tool successfully hacked into the infrastructure of three separate companies during red-teaming tests by exploiting weak passwords and unauthenticated endpoints. Anthropic said an external evaluation partner mistakenly granted the testing agent unfiltered internet access.

Mount Vernon official is accused of driving getaway car after son allegedly opened fire outside Bronx court

A New York state police department fired its deputy commissioner on Tuesday after she allegedly acted as a getaway driver for her son in what is being investigated as an attempted murder.

The city of Mount Vernon said in an Instagram post that it had terminated Jennifer Lackard, deputy police commissioner for safety, for conduct “unrelated to her duties”.

Mount Vernon official is accused of driving getaway car after son allegedly opened fire outside Bronx court

A New York state police department fired its deputy commissioner on Tuesday after she allegedly acted as a getaway driver for her son in what is being investigated as an attempted murder.

The city of Mount Vernon said in an Instagram post that it had terminated Jennifer Lackard, deputy police commissioner for safety, for conduct “unrelated to her duties”.

Google has renamed NotebookLM as Gemini Notebook, tying its source-grounded research workspace more closely to the wider Gemini ecosystem. The change is more than a new label: Google says notebooks will sync across the Gemini app and Google Search as the rollout expands, creating a broader home for organizing sources, chats, and AI-assisted analysis.

The rebrand provides the clearest context for recent attention around file support, chat-based creation, and deeper analysis. Gemini Notebook already supports a range of source formats, while Gemini can generate files from chat and offer Deep Research for structured reporting. But these capabilities have different scopes and limits, particularly around generating multiple files in a single prompt.

According to Google's official Gemini Notebook announcement, the July 16, 2026 change begins a staged expansion across interfaces. Google says Gemini Notebook will be available as the integration proceeds for Workspace customers and personal accounts.

A notebook workspace becomes part of Gemini

NotebookLM was designed around working with a defined collection of source material. Under the Gemini Notebook name, that organizing model is being extended into Gemini Apps, where notebooks can group chats and sources rather than remaining a separate destination. Google's support material describes supported sources including Google Docs, Slides, Sheets, PDFs, URLs, and audio.

That matters because research workflows often break down when a user must repeatedly move context between a document repository, a search tool, and an AI chat interface. Synchronization across the Gemini app and Google Search points to a more connected workflow: a notebook can serve as the source and organization layer while Gemini provides the conversational interface.

The supplied material does not specify any new pricing or plan changes tied to the rename. It instead establishes an ecosystem and rollout shift, with availability extending across Workspace and personal accounts as Google deploys the integration.

File support is broad, with an important Word-format distinction

Google's Gemini Notebook documentation confirms support for common documents and images. Users can add PDFs, PNG images, DOCX files, PPTX presentations, and XLSX spreadsheets, among other supported text-based sources through Google Drive imports. This gives teams a practical route to combine reference documents, visual material, and office files in one research context.

The distinction between DOCX and DOC is important. Google's documentation specifically identifies Microsoft Word .docx support. It does not establish support for the older .doc format, so organizations with legacy files should avoid assuming that the two formats are interchangeable.

For enterprise use, broader source support can reduce the preparation required before analysis. It also raises familiar governance questions about what materials are imported, who can access notebooks, and how source content is handled within an organization's Google environment. Those decisions remain operational responsibilities for each Workspace customer.

Chat can generate files, but not several at once

Gemini's chat experience can generate files directly from a conversation, including Docs, Sheets, Slides, and PDFs. This is a meaningful productivity capability because it can turn an AI interaction into a usable work artifact without requiring the user to manually recreate the output in another application.

However, Google's April 2026 Workspace Update states a clear current limitation: Gemini supports one generated file per prompt. The available official information therefore supports in-chat file generation, but not concurrent generation of multiple artifacts from a single request.

That distinction is relevant for teams designing repeatable workflows. A task that requires a report, presentation, and spreadsheet may still benefit from chat-based creation, but it must be handled through separate prompts or another orchestration approach until Google changes the stated limit.

Deep Research adds structured analysis

Google positions Deep Research as a research assistant that gathers information from web sources and Drive content into multi-page, structured reports. Advanced tiers can add charts and other visuals. In practice, this is the strongest basis for describing Gemini's analysis as deeper: the capability is built for synthesizing multiple sources into a report rather than returning only a short chat response.

The value of that approach depends on the quality and relevance of the selected material. A well-curated Gemini Notebook can give Deep Research a more useful working context, while poor, outdated, or incomplete sources can still limit the output. The rebrand does not remove the need for users to review findings, validate important claims, and apply their own judgment.

Organizations assessing how Gemini Notebook, Drive content, and chat-based generation fit into internal workflows can work with Scalevise on AI architecture, workflow automation, and implementation planning.

Frequently Asked Questions

What is Gemini Notebook?

Gemini Notebook is the new name for Google's NotebookLM. Google is integrating notebooks more closely with the Gemini ecosystem, including planned synchronization across the Gemini app and Google Search.

Which file types does Gemini Notebook support?

Google lists support for formats including PDF, PNG, DOCX, PPTX, and XLSX, along with other text-based files imported through Google Drive. Its documentation specifically identifies DOCX, not the older DOC format.

Can Gemini generate multiple files from one chat prompt?

Google says Gemini can generate files such as Docs, Sheets, Slides, and PDFs from chat, but it currently supports generating one file per prompt.

How does Deep Research relate to Gemini Notebook?

Deep Research can synthesize web sources and Drive content into structured, multi-page reports. Gemini Notebook provides a way to organize chats and source material that can support source-based research workflows.

Conclusion

Gemini Notebook reframes NotebookLM as part of Google's larger AI product strategy, with notebook synchronization across Gemini and Search as the most consequential announced change. Its existing support for common source formats, chat-based file creation, and Deep Research can strengthen research workflows, but users should plan around the documented one-file-per-prompt generation limit and verify AI-produced analysis against their source material.

Hoy buena parte del código que entra a mis proyectos lo escribe un modelo. No es un experimento, es cómo trabajamos. Y eso me trajo una pregunta que antes casi no me hacía, porque el código lo escribía yo y más o menos sabía dónde estaban los huecos: si no escribí esto, ¿cómo sé que sirve?

La respuesta obvia es "mido cobertura". Yo la mido. Y una vez tuve todo en verde mientras la página se caía en producción.

Este post es sobre esa contradicción, y sobre las decisiones que hacen que el número signifique algo o no signifique nada.

Primero, qué mido

Nada exótico: flutter test --coverage genera el lcov.info y después filtro con lcov --remove antes de sacar el porcentaje.

lcov --remove coverage/lcov.info \
    '**/*.freezed.dart' \
    '**/*.g.dart' \
    '**/*.config.dart' \
    '**/constants/*.dart' \
    '**/theme/*.dart' \
    '**/di/*.dart' \
    '**/router/*.dart' \
    -o coverage/lcov_filtered.info

Lo que saco de la cuenta y por qué:

Los archivos generados (freezed, json_serializable, la config de inyección) no los escribí yo. Testearlos es testear al generador. Si freezed está roto, ese no es mi test.

Constantes y theme no tienen ramas. No hay una entrada que los haga comportarse distinto. Un test ahí solo confirma que una constante vale lo que vale.

DI y routing son cableado. Verificar que el contenedor resuelve una dependencia es testear el framework.

La regla, si tuviera que resumirla, es que excluyo lo que no puede tomar una decisión equivocada. Todo lo demás entra.

Y acá conviene decir algo incómodo antes de que lo diga otro: la lista de exclusiones es donde se puede hacer trampa. Puedo llegar al porcentaje que quiera moviendo esos patrones. Por eso no la toco cuando el número no me gusta. Si tengo que justificar por qué saqué algo, la respuesta no puede ser "porque no llegaba".

El test que ejecuta todo y no verifica nada

Este es el malentendido que hace que gente con experiencia desconfíe de la cobertura, y tienen razón en desconfiar.

lcov mide qué líneas se ejecutaron. No mide si comprobaste algo. Son cosas distintas y un modelo generando tests las confunde todo el tiempo.

test('aplica descuento', () {
  final result = calculator.applyDiscount(100, 0.2);
  expect(result, isA<double>());
});

Ese test ejecuta cada línea de applyDiscount. Cobertura: 100%. Verificaciones reales: cero. Si la fórmula estuviera invertida, precio * descuento en vez de precio * (1 - descuento), pasaría igual de verde.

test('aplica 20% de descuento', () {
  expect(calculator.applyDiscount(100, 0.2), 80.0);
});

test('rechaza descuentos mayores al 100%', () {
  expect(() => calculator.applyDiscount(100, 1.5), throwsArgumentError);
});

Misma cobertura de líneas. La diferencia no la mide ninguna herramienta. La mide una pregunta: ¿el test afirma algo, o solo ejecuta?

Los modelos escriben el primero por defecto. Es lo que mejor hacen: recorren el código, llegan al verde, suben el número. Cuando reviso un test generado no miro si el porcentaje subió, miro qué pasa si rompo la función a propósito. Si el test sigue pasando, el test no existe.

La capa de presentación no entra, y no es por pereza

Esta fue la decisión que más me costó y la que más defiendo.

Empecé escribiendo widget tests como todo el mundo. Dos cosas me hicieron parar.

La primera es tonta pero real: los expect no encontraban el widget. A veces sí, a veces no, según cómo hubiera quedado el árbol. Terminaba peleándome con los finders en vez de verificar comportamiento. Cualquiera que haya escrito widget tests en Flutter sabe de qué hablo.

La segunda es la que importa. En la app hay opciones que aparecen o no según el rol del usuario, y hay bastantes casos. Pero esa decisión no vive en el widget: vive en el BLoC. El widget solo pinta lo que el estado le dice. Entonces si testeo desde el widget que "el usuario X ve la opción Y", estoy verificando la misma regla que ya verifiqué en el BLoC, pero por una puerta más lenta, más frágil y que se rompe cuando alguien cambia un Padding.

No es que no testee la UI. Es que no testeo dos veces lo mismo, y elijo hacerlo donde el test es barato y estable.

(Lo que sí me falta, y lo digo porque un post honesto también dice dónde no llegó: golden tests para lo que sí es estable. Está en la lista. No lo hice todavía.)

Legacy: no persigo el pasado

En proyectos que arranco de cero apunto a cubrir toda la lógica. Pero también mantengo código heredado, sin tests, escrito por gente que ya no está.

Ahí no persigo el 100% hacia atrás. Sería meses escribiendo tests para código que no voy a tocar, y peor, tests que congelan la implementación actual en vez de proteger una conducta.

Lo que hago es más simple: el código viejo se queda como está, pero todo lo nuevo que le agrego entra con sus tests. La barra sube cada vez que toco ese módulo. No freno la entrega para cubrir el pasado, freno la sangría.

Es la diferencia entre "este proyecto tiene 100%" y "este proyecto no vuelve a perder cobertura". El segundo es alcanzable sin parar el mundo.

La caída

Y acá está la parte donde el número me mintió en la cara.

Mi suite corre sobre mocks. Toda la capa de datos está mockeada, así que los tests no hablan con un servidor: hablan con una versión congelada de lo que el servidor decía la última vez que escribí ese mock.

Tenemos un sistema de puntos por pedidos entregados. Un día, por un error del backend, un pedido entregado no cargó sus puntos y el campo vino null. Nunca antes había venido null. La app no lo aguantó, la web tampoco, y la página tiró error.

Todos mis tests en verde. Todos.

Y el motivo es casi humillante de tan simple: mi mock seguía devolviendo el mundo viejo. Yo tenía cobertura completa de una realidad que ya no existía. El test nunca vio un null porque yo nunca escribí un mock que lo devolviera, y no lo escribí porque en mi cabeza ese campo siempre venía.

Esa es la respuesta que le doy a cualquiera que dice que la cobertura no significa nada. Tenés razón, y acá está la prueba desde mi propia suite: el 100% significa que ejecutaste tus líneas contra los supuestos que vos mismo escribiste. Si tus supuestos están viejos, medís qué tan bien se testea tu imaginación.

La lección no fue "los mocks son malos". Los mocks los necesito. Fue que cobertura de líneas y cobertura de casos no son lo mismo, y que el caso nulo es el que más veces se me escapó. Desde entonces los nulos y los cambios de contrato son ciudadanos de primera clase en mis tests, no un pensamiento tardío.

test('maneja puntos nulos del backend', () {
  expect(() => calculator.applyDiscount(null, 0.2), throwsArgumentError);
});

Sigo sin tener una respuesta completa a esto, para ser honesto. Un test de contrato contra la respuesta real sería lo correcto, pero hoy no lo tengo armado. Por ahora lo que hago es asumir que cualquier campo puede venir nulo, aunque el backend jure que no.

Lo que en realidad estoy diciendo

La cobertura te dice dónde no miraste. No te dice que lo que miraste esté bien.

Un modelo te regala líneas ejecutadas gratis, todo el día. Lo que no te regala es decidir qué merece un test, cuál afirma algo de verdad, qué no vale la pena cubrir, y cuándo el número te está mintiendo.

Eso sigue siendo trabajo tuyo. Por ahora.

¿Vos qué excluís de tu cobertura, y por qué? Me interesa sobre todo si alguien resolvió bien el tema de los contratos de API contra mocks, porque yo todavía no.

US secretary of state comments come after treasury secretary, Scott Bessent, said Washington could reach a deal ‘by tomorrow’

• US and Qatar report progress on Iran ceasefire and reopening Hormuz strait

Iran and Oman have made progress toward a deal to reopen the strait of Hormuz, a potential breakthrough that could help end the war in the Middle East, regional officials have said.

Under the emerging agreement, ships would enter the Persian Gulf through an Iranian-controlled route and exit through a route controlled by Oman, with service fees charged for providing security and preserving the maritime environment, two regional officials told the Associated Press.

A Spark job commits a table update. The catalog writes the change to Postgres. Then the network drops between the catalog and the client, and the client never sees the response. The client does the sensible thing and retries. This time the catalog sees that the table has already moved past the base snapshot in the request, so it returns 409 Conflict. The client reads that 409 as a failed commit and deletes the metadata files it just wrote. The commit is now recorded in the catalog, and the files it points at are gone.

That is data loss. It comes from a network blip, not from a bug in anyone's query engine.

Apache Polaris 1.7.0 shipped on August 2, 2026, tagged by JB Onofré at commit 4ac2f05. It fixes that specific failure and a long list of others in the same family. If you skim the changelog you see hundreds of entries, most of them dependency bumps, and it looks like a maintenance release. It is not. Underneath the noise there are four real stories: idempotent writes, a new beta API for semantic models, a much stricter approach to credential vending and location validation, and a serious pass over orphan file cleanup.

I am going to walk through all four, and I am also going to tell you which parts of this release create work for you rather than saving you work. Both kinds show up here.

This piece assumes you know what a table is and roughly what a data lake is. Everything past that gets defined as it comes up.

What a catalog actually does, and why its bugs are expensive

Apache Iceberg is a table format. It describes how to lay out data files and metadata files in object storage so that many engines read the same table the same way. Iceberg tracks a table's current state through a chain of files: a metadata file points at snapshots, snapshots point at manifest lists, manifest lists point at manifests, and manifests point at the actual Parquet data files.

One question that chain does not answer is: which metadata file is current right now? Every change to a table writes a brand new metadata file. Something has to record the swap from the old one to the new one, and it has to do that atomically so two writers cannot both think they won.

That something is the catalog. In its smallest form, a catalog is a pointer store. It maps a table name to the path of the current metadata file, and it swaps that pointer atomically on commit.

Apache Polaris is an open source implementation of that pointer store, speaking the Iceberg REST protocol. The REST protocol matters because it moves catalog logic out of the client. With older designs like Hive Metastore, every engine embedded its own catalog client code, and every engine had its own opinions about credentials and connection handling. With a REST catalog, the engine speaks HTTP to a service, and the service handles storage credentials, access control, and the commit protocol. Polaris was co-created with Snowflake, donated to the Apache Software Foundation, and graduated to Top-Level Project on February 18, 2026.

Because the catalog owns the pointer swap, catalog bugs have a nasty property. They do not corrupt one query. They corrupt the table. A dropped pointer, a prematurely deleted metadata file, or a credential scoped one character too wide affects every engine that reads that table afterward. This is why a release like 1.7.0, which is heavy on correctness fixes and light on flashy features, deserves more attention than a release full of new endpoints.

Here is the shape of what changed, before the details.

Idempotent writes, the headline feature

Go back to the failure I opened with. The root cause is that HTTP gives the client no way to distinguish "your request never arrived" from "your request succeeded and the response got lost." Those two cases demand opposite responses. In the first case the client should retry. In the second case the client should stop and treat the commit as done.

The Iceberg community's answer is an Idempotency-Key header on mutation endpoints, following the same design as the IETF draft for idempotency keys that payment APIs have used for years. The client generates a unique key per logical operation and sends it with the request. The server remembers the key and the outcome. On a retry with the same key and the same payload, the server returns the original result without executing anything again.

Polaris 1.7.0 implements the server half of that for two operations. Huaxing Gao's work landed entity-property idempotency for createTable in #4912 and opt-in idempotency for updateTable in #5037.

Read the word "opt-in" carefully, because it is the whole story for operators. Idempotency on updateTable is not on by default. You turn it on, and clients have to send the key. Nothing about upgrading to 1.7.0 makes your existing writers retry-safe by itself.

How a client finds out

The interesting design choice is capability discovery. A client has no business guessing whether a catalog honors idempotency keys, because guessing wrong in the unsafe direction produces exactly the corruption we are trying to prevent. So the catalog advertises it.

Every Iceberg REST catalog exposes a GET /v1/config endpoint that clients call at connection time. It returns two property bags: defaults, which the client applies unless it overrides them, and overrides, which the server forces. #5118 added an idempotency key lifetime to what Polaris returns there.

A response now carries something in this shape:

{
  "defaults": {
    "clients": "4"
  },
  "overrides": {
    "idempotency-key-lifetime": "PT30M"
  },
  "endpoints": [
    "GET /v1/{prefix}/namespaces/{namespace}/tables/{table}",
    "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}"
  ]
}

The lifetime is the retention window for remembered keys. PT30M is ISO-8601 duration notation for thirty minutes. Inside that window, a repeat of the same key returns the original outcome. Outside it, the server has forgotten, and the retry runs as a fresh request with all the ordinary conflict semantics.

That window is a real operational parameter, and picking it is a tradeoff you own. Set it too short and a client that backs off aggressively falls outside the window before it retries, which puts you right back in the original failure mode. Set it too long and the server tracks more keys than it needs to, which costs persistence space and lookup time on every mutation. Thirty minutes covers the retry behavior of most engines with room to spare. Start there, then look at your longest observed client backoff before you change it.

The storage design got simpler mid-flight

One detail in the changelog rewards a second look. #5086 removed an unused IdempotencyStore and an idempotency_records table.

That is the sound of a design being reconsidered before release. An earlier approach kept idempotency records in a dedicated table, which means a separate write on every mutation and a separate cleanup job to expire old rows. The shipped approach attaches idempotency state to the entity itself, which is what "entity-property idempotency" in the createTable PR title describes. Fewer moving parts, no second table to vacuum, no second failure domain.

If you were tracking this feature from the development branch and built tooling against idempotency_records, that table is gone. Check before you upgrade.

What to do about it

Turning this on is a two-sided change, and the client side is not entirely in your hands yet.

1. Upgrade Polaris to 1.7.0 and confirm the config endpoint reports the lifetime you expect.
2. Check which of your engines send an Idempotency-Key. Support arrives engine by engine as the Iceberg client work lands, so verify rather than assume.
3. For engines that do not send one yet, nothing regresses. You get the same behavior you have now.
4. Watch for 422 responses after you enable it. Under the design, a repeated key with a different payload is a client bug, and the server rejects it rather than guessing which version you meant.

The last point is the one that surprises teams. Idempotency keys make a class of client bugs visible that used to hide inside retry loops. That is a feature. It also generates support tickets in week one.

The catalog starts learning what a metric is

The second story in 1.7.0 is smaller in code and larger in implication. #4816 added scaffolding for an OSI semantic-model API, and #4983 marked it beta.

OSI stands for Open Semantic Interchange. It is an industry specification effort, convened by Snowflake with a broad group of analytics and BI vendors, that defines a YAML format for semantic models. A semantic model in this sense holds the things a table does not: datasets, the relationships between them, dimensions, and metrics. The example everyone reaches for is revenue. Every dashboard defines it, no two definitions agree, and the finance number never matches the sales number.

The OSI spec gives that definition a portable form. A semantic model contains datasets, relationships, and metrics, with SQL expressions attached and optional context annotations written for language models to read.

So why is this landing in a table catalog?

Because the catalog is the one component every engine already talks to. If your metric definitions live in your BI tool, they are available to your BI tool. If they live next to the tables, in the service that Spark and Trino and Flink and your agent framework all authenticate against, they are available to everything. The same argument that moved credential vending and access control into the catalog applies to semantics.

The AI angle is the forcing function. An agent writing SQL against a lakehouse has the schema and nothing else. It sees a column named amt_net and guesses. Give it a metric definition that says net revenue excludes returns and intercompany transfers, and the guessing stops. That is the thin part of the problem that semantic models solve, and it is the part where wrong answers are most expensive because they arrive fluent and confident.

Two supporting changes matter more than they look. #4926 added a catalog config endpoint registry, later moved into runtime/service by #5052. A registry for config endpoints is how a server grows optional API surfaces without every extension hard-coding itself into the core request path. Semantic models are the first tenant of that mechanism. They will not be the last.

The honest assessment

Beta means beta. The PR titles say scaffolding, the API is explicitly marked as unstable, and the OSI core spec itself is young. Do not build a production metric layer on this in August 2026.

What to do instead: read the OSI spec, write a semantic model for one domain you already argue about internally, and see whether the format holds your actual business logic. The feedback loop for a young specification is people trying to express real definitions in it and reporting where it breaks. That is worth more to you and to the project than waiting for version 1.0 of the API.

One more reason to care, independent of which platform you run. Nearly every analytics vendor ships some form of semantic layer, and each one holds your metric definitions in its own format. A portable definition format means those definitions survive a change of vendor. That is worth something regardless of who you buy from today.

Credential vending got stricter, and one fix was a real hole

Credential vending is the feature where the catalog, rather than the engine, holds the cloud storage credentials. An engine asks for a table, the catalog checks whether that principal is allowed, then calls AWS STS or Azure or GCS to mint a short-lived credential scoped to just the paths that table needs. The engine gets a token that opens a narrow door instead of a bucket-wide key.

The security of the whole arrangement rests on one thing: the scoping has to be correct. A credential scoped one prefix too wide hands a reader access to a neighbor's data. 1.7.0 fixes three separate ways that went wrong.

#4860 fixed native catalog credential vending skipping re-validation of allowedLocations. Read that plainly. There was a path where the list of locations a catalog is permitted to touch was not checked again at vending time. That is the kind of fix you upgrade for on its own.

#4884 fixed a GCS downscoped credential prefix boundary problem for locations without a trailing slash. This is the classic prefix bug. A credential scoped to gs://bucket/data/sales with naive prefix matching also opens gs://bucket/data/sales-archive and gs://bucket/data/sales_pii, because both start with the same characters. The trailing slash is what makes the boundary a boundary.

#4707 added GCS principal attribution to vended credentials through Workload Identity Federation. Attribution means the cloud audit log records which Polaris principal triggered the access, rather than showing every request coming from one service account. If you have ever tried to answer "who read this table last Tuesday" from a GCS audit log and found a single identity behind every entry, this is the change that fixes your investigation.

Three more storage changes round out the area. #4954 added a session policy parameter to SigV4 connections, which lets you attach an additional IAM policy that further narrows an assumed role. #4991 added bare ADLS vended credential keys for PyIceberg compatibility, a small fix with a large blast radius given how much Python tooling reads Iceberg tables directly. #5004 propagated storage HTTP client settings to S3FileIO for table operations, so proxy and timeout configuration finally applies to the catalog's own file reads rather than only to the vending path.

Location validation tightened everywhere

Alongside vending, 1.7.0 tightened where a table is allowed to say its data lives. This is the same class of protection viewed from the other end.

• #4422 validates default-base-location against the storage configuration when a catalog is updated, not just when it is created.
• #5114 validates locations when registering tables and views.
• #5115 validates Iceberg metadata locations during table updates.
• #4966 fixed ALLOW_EXTERNAL_METADATA_FILE_LOCATION not being overridable at catalog level.
• #5012 deprecated the external table location flag outright.
• #4606 made default table and view locations unique, and #4975 encoded them with UTF-8.

Register-table is the operation worth understanding here. It points the catalog at an existing metadata file rather than creating a table from scratch, which makes it the natural way to adopt tables that another system wrote. It is also the natural way to point a catalog entry at a location it has no business owning. Validating on register closes that.

Plan for these to reject something. If you have tables whose metadata sits outside the configured allowed locations, and that arrangement has worked because nobody checked, 1.7.0 checks. Audit before you upgrade rather than after.

Authorization, and a multi-tenant isolation fix

Polaris has a two-layer role model. Principal roles attach to service principals, which are the identities engines and users authenticate as. Catalog roles carry the actual privileges on catalogs, namespaces, and tables. You grant catalog roles to principal roles, and a principal gets the union of what its roles allow.

Polaris also supports delegating authorization decisions to Open Policy Agent, usually shortened to OPA. OPA is a general policy engine. Instead of Polaris deciding, Polaris sends a structured input document describing the request and asks OPA for a verdict, which lets you write policy in one language across many systems.

#4992 fixed a gap in that input: the realm identifier was missing. A realm in Polaris is a tenant boundary, the mechanism that keeps separate organizations on one deployment from seeing each other. If your OPA policy receives a request that names a catalog and a table but not the realm, and two realms happen to use the same catalog name, your policy has no way to tell them apart. Any operator running multi-tenant Polaris with OPA should treat this as the reason to upgrade.

The rest of the authorization work is structural. Y Sung's #4356 refactored catalog handlers and the admin service onto a shared resolveAuthorizationInputs path, which consolidates how a request turns into an authorization question. Alexandre Dutra's #5085 refactored PolarisPrincipal to hold generic attributes, followed by an AttributeMap interface in #5139. Generic principal attributes are the groundwork for policies that key on claims your identity provider issues rather than on a fixed set of fields Polaris knows about.

Two changes help the humans. #4406 put the missing privilege and the target entity into 403 messages. A denial that says "access denied" starts a thirty-minute investigation. A denial that names the privilege and the object it applied to ends in thirty seconds. #5011 correlates OPA server logs with the Polaris request ID and adds observability for non-200 responses from OPA, which turns "policy evaluation is behaving strangely" into a traceable event.

Rounding out the area: #5112 clarified principal role selection semantics, #5113 aligned token exchange scope handling, #5096 fixed view grants on federated catalogs, and #4869 optimized the byte-to-long conversion in privilege set bit manipulation.

Orphan files, failed commits, and the cleanup story

This is the section that earns its keep for anyone running Polaris at volume.

Every Iceberg commit writes new metadata before the catalog swaps the pointer. When a commit fails, those files are already in object storage and nothing references them. They are orphans. Orphans cost money forever, and at high commit rates a small orphan rate compounds into a large bill and a slow bucket listing.

Polaris runs asynchronous tasks to clean this up. 1.7.0 rewrote a meaningful part of how those tasks behave.

The most serious fix is #4920, titled as fixing data corruption via premature metadata deletion in commitTransaction. Deleting a metadata file that is still referenced is the exact failure I opened this article with, arriving from the server side rather than the client side. Paired with it, #4934 cleans up metadata files on transaction failure and #5057 cleans up orphan metadata files on failed table and view commits. Together they draw a clear line: on failure, delete the files the failed attempt created, and never touch anything else.

Several fixes target the cleanup tasks themselves.

• #4828 fixed a ManifestReader resource leak in the cleanup handler. Leaked readers hold file handles and memory, and the symptom is a slow drift toward instability under sustained load rather than a clean crash.
• #4941 taught the manifest cleanup handler to handle delete manifests. Delete manifests track row-level deletes in merge-on-read tables. Skipping them means a specific category of file was never cleaned.
• #4970 fixed an infinite loop triggered by a non-positive TABLE_METADATA_CLEANUP_BATCH_SIZE. Set that value to zero and the old code spun.
• #4871 fixed a duplicate setId() in the table cleanup handler that burned entity IDs on every run.
• #4914 fixed async task retry when handlers return false, and #4962 changed TaskHandler.handleTask to return void so success and failure travel through exceptions instead of a boolean nobody checked consistently.

Then there is a performance thread with a direct line to your cloud bill. #4850 added bulk deletion to batch file cleanup. #4928 removed a redundant existence check before deleteFile, and #5005 eliminated double existence checks in batch cleanup.

Those last two are worth dwelling on. Calling exists before delete looks defensive and reads well. Against object storage it doubles your API call count for zero benefit, because delete on a missing key is already a no-op on every major provider. If a cleanup pass touches a million files, you just paid for two million requests instead of one million. Bulk delete then collapses the remaining calls into batched operations. For a deployment with heavy compaction and expiration activity, this is a line-item change.

Events grow up: OpenTelemetry and Kafka

Polaris has an event listener framework that emits events as catalog operations happen. Table created, view committed, entity dropped. 1.7.0 gave that framework two destinations that matter.

#4836, from first-time contributor hkwi, added an OpenTelemetry event listener. OpenTelemetry is the vendor-neutral standard for traces, metrics, and logs, and nearly every observability backend ingests it. Emitting catalog events as OpenTelemetry data means a table commit shows up in the same trace view as the query that caused it, without a custom bridge.

#4923, from Mark McKeown, added an extension for publishing events to Kafka. This one is about governance rather than observability. A durable, ordered log of every catalog mutation is the substrate for lineage systems, data mesh contract enforcement, downstream cache invalidation, and change-driven pipelines. Reading catalog events off a Kafka topic is a far better integration point than polling the catalog on a timer.

Three supporting changes make the eventing usable. #4956 fixed PolarisEventMetadata.eventId() returning a different UUID on every call, which is exactly the bug that breaks deduplication in any consumer built on at-least-once delivery. #4981 replaced the EventEntity.REALM_SCOPED sentinel with a nullable catalog_id, trading a magic value for an honest null. #4877 avoids setting up metrics persistence when events are only buffered in memory, which removes a startup cost for deployments that never enabled persistence.

If you are building anything that reacts to catalog changes, the Kafka extension is the piece to look at first. Start with a consumer that does nothing but log, run it for a week, and read what your catalog actually emits before you design around it.

Persistence: several queries stopped reading more than they needed

The JDBC persistence layer, which for most people means Postgres, got a focused optimization pass. The pattern repeats across the fixes, and the pattern is the lesson.

• #5038 stopped lookupEntityVersions from fetching full entity rows.
• #5078 did the same for lookupEntityGrantRecordsVersion.
• #5134 stopped writeEntities from issuing a wasteful full-row lookup per entity.
• #4973 bounded the JDBC hasChildren existence check with LIMIT, and #5066 fixed the same method fetching all rows and all columns.
• #5020 eliminated redundant metastore lookups when resolving principal roles.

Every one of these is the same mistake in a different place. The code needed one small fact, a version number or a yes/no answer, and asked the database for entire rows to get it. On a catalog with a few thousand entities nobody notices. On a catalog with hundreds of thousands, resolving principal roles on every single request while reading full rows is how a p99 latency graph develops a shelf.

hasChildren is the clearest example. The question is "does this namespace contain anything," and the answer is yes the moment one row exists. Without a LIMIT, the database happily returns all of them, and the cost of asking scales with the size of the namespace instead of staying constant.

One more in the same family: #5027 made TreeMapMetaStore range reads return copies. Returning live references from an in-memory store lets a caller mutate state it does not own, and the resulting bugs are the kind that reproduce once a month in production and never in a test.

Error semantics, or why a 503 is kinder than a 500

A small cluster of changes fixed what Polaris says when it cannot do something. These matter more than their size suggests, because clients make retry decisions from status codes.

#4646 changed concurrent rename to return HTTP 503 instead of 500. The distinction is not pedantic. 500 means the server broke and a retry is pointless. 503 means the server is temporarily unable and a retry is sensible. A concurrent rename is a transient contention event, so 503 is the honest answer, and every well-behaved HTTP client already knows what to do with it.

#5144 went further and set a Retry-After header when a rename fails with a concurrent modification error. Now the client knows both that retrying is worthwhile and roughly when. That turns a hot retry loop into a scheduled one.

#4793 centralized drop-failure error mapping and fixed misleading messages. Scattered error mapping produces the situation where the same underlying condition surfaces as three different messages depending on which code path found it. #4990 fixed diagnostic extra info not rendering in PolarisDiagnostics.fail messages, which had been silently dropping the context attached to failures.

Taken together with the 403 improvement mentioned earlier, this release meaningfully reduces the number of Polaris errors that require reading source code to interpret.

The platform underneath: Jackson 3, Quarkus 3.37, and test infrastructure

Robert Stupp drove a large migration to Jackson 3, the JSON library Polaris uses for serialization, across JDBC metrics, NoSQL pagination tokens, core serialization helpers, and the NoSQL layer. Quarkus moved to 3.37, Gradle to 9.6.1.

This kind of work never shows up in a feature list and always shows up in your incident history if it is skipped. A project that lets its serialization library go three major versions stale eventually finds itself unable to take a security patch without a month of migration work.

#4913 added a readiness check for reflection-free serializers. Reflection-free serialization is what lets a Quarkus application start fast and compile to a native image, and a startup check that verifies it is actually in effect prevents a silent fallback to the slow path.

The test infrastructure moved from localstack to Floci testcontainers for AWS, GCP, and Azure emulation, with integration tests migrated to a shared server runner and pushed down into the extensions they belong to. Faster and better-isolated tests sound like an internal concern. They are the reason the next release ships with fewer regressions.

Operationally useful odds and ends:

• #4755 added maintenance support to the Helm chart.
• #4921 added HTTP histogram buckets, which gives you real latency distributions instead of averages.
• #4996 made admin tool bootstrap idempotent for already-bootstrapped realms, so rerunning bootstrap in automation stops being dangerous.
• #5044 removed the schema version option from the admin bootstrap command.
• #4772 and #4770 fixed credential exposure in Python CLI debug logs and hardened profile secret handling and config storage.
• #4936 added --catalog-url for custom Iceberg REST base URIs, and #5043 added non-HTTP scheme support to the CLI.
• #4849 added a Trino guide, contributed by a first-time contributor.

That Python CLI credential fix deserves emphasis. Secrets in debug logs are how credentials end up in log aggregation systems with different retention and access rules than your secret store. If anyone on your team has ever run the Polaris CLI with debug logging on, rotate those credentials.

Upgrading: a walkthrough

Here is the order I recommend, with the reasoning attached rather than left implicit.

Step 1: audit locations before you touch anything

1.7.0 validates locations in places that previously went unchecked. Find out now whether any of your tables fail those checks.

# List every catalog and its configured base location
polaris catalogs list --output json \
  | jq -r '.[] | [.name, .properties["default-base-location"]] | @tsv'

# For one catalog, list tables and their metadata locations
polaris tables list --catalog analytics --output json \
  | jq -r '.[] | [.name, .metadataLocation] | @tsv'

What you are looking for is any metadata location that sits outside the catalog's configured base location and outside the storage config's allowed locations. Those are the entries that start failing on update or registration. The --catalog-url flag added in 1.7.0 helps here if your Polaris sits behind a path-rewriting proxy.

If you find violations, decide deliberately. Either widen the allowed locations to legitimately include those paths, or move the tables. Do not upgrade first and discover it through a failed production write.

Step 2: pull the new image and check readiness

docker pull apache/polaris:1.7.0
docker pull apache/polaris-admin:1.7.0

The 1.7.0 artifacts are signed and checksummed like every Apache release. Verify them rather than trusting the registry alone:

curl https://downloads.apache.org/polaris/KEYS -o KEYS
gpg --import KEYS
gpg --verify apache-polaris-1.7.0.tar.gz.asc
shasum -a 512 --check apache-polaris-1.7.0.tar.gz.sha512

Step 3: bootstrap safely

Admin bootstrap is now idempotent for already-bootstrapped realms, which makes it safe to leave in a deployment pipeline. Note that the schema version option was removed from the bootstrap command, so pipelines that pass it need editing.

docker run --rm apache/polaris-admin:1.7.0 bootstrap \
  --realm my-realm \
  --credential my-realm,root,secret

Step 4: configure the pieces you want

A Helm values file covering the areas this release touched looks roughly like this. Treat it as a map of the knobs, not as a drop-in config.

image:
  tag: "1.7.0"

# Persistence. Postgres is the common choice for production.
persistence:
  type: relational-jdbc

# Event listeners. Multiple listeners run side by side.
# The OpenTelemetry and Kafka destinations are new in 1.7.0.
polarisServerConfig:
  polaris:
    event-listener:
      types: "opentelemetry,kafka"

# HTTP latency histograms rather than averages.
    metrics:
      http:
        histogram-buckets: "50ms,100ms,250ms,500ms,1s,2s,5s"

# Delegate authorization decisions to Open Policy Agent.
# The realm identifier is now part of the input document.
    authorization:
      type: opa
      opa:
        base-uri: "http://opa.data-platform.svc:8181"

# Async cleanup task sizing. A non-positive batch size
# no longer loops forever, but set a sane value anyway.
    tasks:
      metadata-cleanup-batch-size: 100

The event listener line is the one people get wrong. types takes a comma-separated list, and support for multiple simultaneous listeners arrived in an earlier release, so adding OpenTelemetry alongside an existing listener does not displace it.

Step 5: verify what the server advertises

After the rollout, ask the catalog what it thinks it supports. This is the same call your clients make.

curl -s -H "Authorization: Bearer $TOKEN" \
  "https://polaris.example.com/api/catalog/v1/config?warehouse=analytics" \
  | jq

Look for the idempotency key lifetime in the returned properties. If it is absent, either the feature is not enabled in your configuration or you are not running what you think you are running. Checking the advertised capability beats checking the deployed tag, because the advertisement is what clients act on.

Step 6: watch four things for a week

• HTTP 422 responses. New under idempotency. A repeated key with a changed payload is a client bug and now surfaces as a rejection.
• HTTP 503 with Retry-After on rename. Expected and healthy. A rise in volume points at contention worth investigating, not at a Polaris problem.
• 403 message content. They now name the privilege and entity. Any log parsing built on the old shape needs updating.
• Cleanup task throughput and object storage request counts. Bulk deletes and removed existence checks should move both. If they do not, your cleanup tasks are not running as often as you assume.

Failure modes worth knowing about

Every release closes some doors and opens others. These are the sharp edges I expect to generate questions.

Idempotency lifetime set too short. Client retries that fall outside the retention window get treated as fresh requests. The corruption scenario from the opening returns, and the metrics look fine because the feature is technically enabled. Set the window longer than your slowest client's maximum backoff, and revisit it when you change engine retry configuration.

Assuming clients send keys. Server support does not create client behavior. A dashboard that shows idempotency enabled tells you nothing about whether your Spark jobs are using it. Verify at the request level.

Location validation surprises. Covered above, and repeated here because it is the most likely upgrade-day incident. A table registered years ago against a path outside the current allowed locations has been quietly working. It stops.

Semantic model API churn. It is beta and marked beta. Anything you build against it now, you rewrite.

Removed idempotency_records table. If internal tooling read it from a development build, that tooling breaks.

Log parsing on 403 and drop-failure messages. Message text changed for the better. Alerting rules that match on old strings will go quiet, which is the worst way for an alert to fail.

Cleanup tasks that were never running. Several fixes in this release make cleanup faster and more correct. None of them help if your async task workers are starved, misconfigured, or crashing quietly. Before you credit 1.7.0 with a drop in orphan files, confirm the tasks execute at the rate you expect. The infinite-loop fix for a non-positive batch size is a hint that at least one deployment somewhere had cleanup wedged without noticing.

Attributing a latency improvement to the wrong change. The persistence fixes, the removed existence checks, and the Quarkus upgrade all landed together. If your p99 improves after upgrading, resist the urge to explain it. Measure one workload before and after, keep the configuration otherwise identical, and let the numbers stay unexplained rather than mis-explained.

Where the catalog layer is going

Three trends run through this release and they all point the same direction.

The first is that catalogs are becoming full transactional systems rather than pointer stores. Idempotency keys, Retry-After semantics, orphan cleanup on failed commits, and correct conflict status codes are the vocabulary of a database, not of a lookup service. The Iceberg REST spec started as an interface for finding a metadata file. It is turning into a protocol for coordinating concurrent writers across engines that know nothing about each other.

The second is that the catalog is accumulating the metadata that engines cannot agree on among themselves. Access control moved there first, then credential vending, then policy. Semantic models are next in line. The pattern holds because the catalog is the one service every engine authenticates against, which makes it the only natural place to put something all engines need to share.

The third is that AI workloads are the pressure driving both. An agent issuing queries is a client with no institutional knowledge, aggressive retry behavior, and no human reviewing each result. It needs metric definitions it can read, credentials scoped tightly enough that a mistake stays contained, and write paths that survive retries. Every one of those needs shows up in this release.

There is a fourth thing worth naming, which is the health of the project itself. Eight first-time contributors landed changes in 1.7.0, from a Trino guide to an OpenTelemetry listener to a Kafka extension. The contributor list spans many employers. For anyone evaluating whether to build on Polaris, that distribution matters as much as any feature. A catalog is infrastructure you keep for a decade, and single-vendor projects have a way of changing direction on someone else's schedule.

Conclusion

Apache Polaris 1.7.0 is not a release you adopt for a headline feature. It is a release you adopt because the failure modes it closes are the ones that cost you data.

The order of importance for most deployments: the credential vending re-validation fix and the OPA realm isolation fix are security work, and they come first. The premature metadata deletion fix in commitTransaction is data-integrity work, and it comes next. Idempotency is the feature everyone will write about, and it deserves the attention, but it requires deliberate enablement and client cooperation before it does anything for you. The semantic model API is a signal about where this project is heading rather than something to build on this quarter.

Audit your locations first. Upgrade second. Turn on idempotency third, once you know which engines can use it.

The unglamorous truth about catalogs is that the best possible outcome is that nobody thinks about them. A release like this one, mostly made of correctness fixes that prevent incidents nobody will ever see, is what that outcome is built from.

Keep Going

If this piece was useful, I have written a lot more on catalogs and lakehouse architecture.
Apache Polaris: The Definitive Guide, which I co-authored for O'Reilly, covers the access control model, credential vending, and federation in the depth a release note cannot.
You can find every book I have written, across lakehouse architecture,
Apache Iceberg, Apache Polaris, and AI, at
books.alexmerced.com.

Bari Weiss’s latest hire for “60 Minutes,” whose major journalistic output includes smearing genocide victims and putting a target on the backs of reporters in Gaza, shows that the full-blown right-wing tabloidification of the television news program is charging ahead. 

If one held out hope that U.S. media would seek to reverse course after its support for genocide in Gaza over the past 34 months, those hopes were severely misplaced. The mainstream media’s lockstep support for genocide, which I document in my recent book on the subject, has not abated, and in key ways it’s only ramped up, become more explicit, and removed any remaining pretense of editorial independence.

The most obvious example of this is the Ellison family’s $8 billion purchase of Paramount and overtly right-wing revamp of CBS News, which kicked off in earnest last year. A former gold standard of centrist journalism, Weiss’s takeover of CBS News has seen numerous high-profile firings, scandals, and credible allegations of undermining journalistic independence, allegations of Trump toadyism, and a broader editorial push to the right.

Related

Scott Pelley Shows How Legacy Media Got It Wrong — Before Bari Weiss Made It Worse

This campaign has been, in large part, aimed at sucking up to Trump to push through the much bigger prize of a $111 billion Warner Bros. Discovery takeover. But there’s another element at play: The Ellison family’s documented support for Israel and its broader ideological aims. As I noted last December, Weiss was put in place by David Ellison precisely because of her years of strident, pro-Israel coverage and willingness to push the boundaries of racism and cynicism to pursue this cause. So far, Weiss’s attempt to turn the prestigious “60 Minutes” brand into another New York Post has been met with major internal resistance from high-profile reporters, producers, and news management. 

And despite Ellison’s buzzy op-ed for The New York Times today, where he pushes back on “speculation about my politics, my loyalties, my intentions” and pledges “independence” for the newsroom, Weiss’s latest hire makes clear that her ideological coup of the once-vaulted “60 Minutes” brand is right on schedule. 

The hiring of Tanya Lukyanova, a video journalist for The Free Press, as an associate producer for “60 Minutes,” was announced Friday. Lukyanova has previously done video and research work for the Wall Street Journal, the New Yorker, and Semafor. She rose the ranks of journalism doing critical — if boilerplate — reporting on Russia, but over the past year, under Weiss’s direction, she has veered into outright right-wing tabloidism. This climaxed with her contribution to two of the most shameful supposedly journalistic contributions since Israel’s genocide in Gaza began in late 2023: an article she produced herself in September 2025 helping justifying Israel’s unprecedented attacks on journalists in Gaza, and an article she co-bylined in August 2025 downplaying mass starvation in Gaza.  

The September 2025 article, “Gazan Journalists Say Al Jazeera Works Hand in Glove with Hamas,” relies almost entirely on alleged Gazan sources procured by the Center for Peace Communications. This is a pro-Israel front group that poses as a “peace organization” and has peddled similar dubious narratives elsewhere, almost aways designed to create the narrative that Gaza is little more than 2 million Palestinians being held captive by Hamas, who desperately want Israel to bomb their way to liberation. The group’s board of directors includes major pro-Israel funders, as well as Dennis Ross, a long-time pro-Israel pundit and fellow at the Washington Institute for Near East Policy, itself a spin-off from the American Israel Public Affairs Committee. The CPC is run by Joseph Braude, who in 2004 pleaded guilty in federal court for illegally smuggling artifacts out of Iraq, namely 4,000-year-old marble and alabaster stone seals he knew to be stolen. 

At no point in the story was the shady past of the “Center for Peace Communications,” or its overt Zionist advocacy and funding, mentioned by Lukyanova, who presents them merely as a good-faith humanitarian organization. The shameful guilt-by-vague-association piece proceeds to imply, over and over again, that any connection to Hamas, no matter how remote, makes journalists a fair target for Israeli killing — without once reckoning with the numerous active and former members of the Israeli military who make up the ranks of Israeli and U.S. media.

It relies heavily on New York Post reporting because the New York Post is exactly what Weiss wants CBS News to look like in the near future: lurid incitement mixed with shoddy sourcing and anti-Palestinian racism. Lukyanova dutifully lends this shoddy racist incitement the veneer of respectability. 

Israel has killed more than 240 media workers in Gaza since October 2023, including several prominent Al Jazeera journalists such as Mohammed Samir Wishah, Hossam Shabat, Mahmoud Wadi, and Anas al-Sharif. In airstrikes in late 2023 and early 2024, Israel killed the wife, 7-year-old daughter, 15- and 27-year-old sons, and two nephews of longtime Al Jazeera reporter and Gaza bureau chief Wael al-Dahdouh, effectively wiping out his whole family while failing to kill al-Dahdouh himself after he was injured in a separate attack.

Related

Palestinian Rights Groups That Document Israeli Abuses Labeled “Terrorists” by Israel

The Costs of War project at Brown University deemed Israel’s war on Gaza the deadliest conflict for media workers ever recorded. Lukyanova’s anonymously sourced “exposé” accusing Al Jazeera reporters of being “affiliated with Hamas” came just two weeks after +972 Magazine revealed a secret Israeli military propaganda campaign to tie critical reporters to the designated “terrorist organization,” using what it calls a “Legitimization Cell.”

Lukyanova’s other major contribution, published along with Olivia Reingold, was “They Became Symbols for Gazan Starvation. But All 12 Suffer from Other Health Problems.” The article led to widespread backlash and criticism for its callousness, bizarre logical leaps, and the implication that starving children was No Big Deal if those being killed by Israel’s deliberate campaign of hunger as a weapon of collective punishment happen to have preexisting conditions.

Related

Bari Weiss’s Free Press Wants You to Know Some Kids Being Starved by Israel Were Already Sick

As Natasha Lennard noted for The Intercept at the time, Anne Frank wasn’t directly killed by the Nazis but died of typhus as a result of the Nazis creating the condition for such a disease to spread in their concentration camps to claim its intended victims. Drop Site News followed up on the 12 “Gazans” whose brutal suffering Reingold and Lukyanova decided to well, actually and found “their underlying health conditions did not drive the deterioration of their health. Instead, it was the lack of access to food and medicine that drove their acute medical crises. Such is the hallmark of a famine. The first to fall victim are generally those who had underlying conditions to begin with.” 

This is consistent with much of Lukyanova’s non-Gaza output. While ​​plenty of her journalistic output is conventional in nature, she has published several articles promoting schlocky regime-change narratives aimed at various U.S. enemies, namely Iran and Venezuela. Lukyanova has shown a willingness — and apparent eagerness — to use her influential platform to promote war, starvation as a weapon of war, attacks on reporters, and related ideological orthodoxies of U.S. and Israel aggression.

And this shameless propagandist, who has repeatedly shown a willingness to be callous in the face of widespread famine and launder pro-Israel talking points about Israel-critical reporters being secret terrorists, will now be an associate producer at the most respected news program in all of the American media.

“60 Minutes” has always been a flawed institution, but hiring people willing to wade into the lowest gutter of incitement reporting — putting targets on the backs of disabled, starving kids and journalists documenting a genocide — is a major escalation in Weiss’s battle to turn “60 Minutes” into a billionaires’ ideological plaything. 

This is who gets ahead in U.S.-based journalism: not who is most humane, or accurate, or fair, or speaks truth to power, but those who serve the needs of power — whether it be the U.S. and Israeli governments or billionaire media owners.

Related

The Architects of the Iraq War: Where Are They Now?

Just as The Atlantic’s now-editor-in-chief Jeffrey Goldberg was rewarded for his lies attempting to tie Saddam Hussien to 9/11 in the run-up to the Iraq War with becoming the head of the most influential liberal magazine in the country; just as Joe Scarborough was rewarded for cheering on the Iraq War with a three-hour morning block on the most influential liberal cable news show; just as Iraq War cheerleaders from Ezra Klein to Anne Applebaum to Bret Stephens to David Remnick have all enjoyed prestigious positions in today’s elite liberal media — those who lied about, covered up, or otherwise downplayed the genocide in Gaza will continue to rise to the top. This dynamic is only becoming more shameless and acute now that pro-Israel multibillionaires are buying up old legacy media and turning them into their personal tabloid outlets.  

The post Bari Weiss’s “60 Minutes” Hires Producer Who Downplayed Gaza Starvation, Justified Killing Journalists appeared first on The Intercept.

The company will release its second quarter earnings on Tuesday, giving investors a look at its potential profit

For the first time since SpaceX went public, the world will get a first-hand look into the trillion-dollar corporation’s financials. The Elon Musk-run business will report its second quarter earnings on Tuesday, while nervous investors look to gain more insight into the rocket ship company’s potential for profit.

SpaceX had a blockbuster initial public offering in June with the largest-ever stock market debut in history. The IPO transformed SpaceX into a $2tn company and briefly crowned Musk the world’s first trillionaire. But since then, the company’s stock has plummeted by 24%, erasing nearly $500bn in market cap.