OCT. 22, 2026 (英语).
随着企业云平台的规模为数百万并行用户服务,紧密相接的微服务引入了任何静态跑本或阈值警报都无法预料的连锁故障模式。 在像黑色星期五闪光发售这样的高速度事件期间,站点可靠性工程师(SREs)面临一个不可饶恕的现实:诊断出几秒钟后就分布在各地的噪音遥测,或者遭受连锁断电,每分钟花费数百万美元。
虽然基因化的AI copilots加速了代码的编写和日志总和化,但AI agents为可靠地补救活生生的停产而挣扎。 待命工程师的战斗连接池饿死不需要AI agent提供故障排除提示或未经核实的拉力请求,这些请求可能无法解决根本原因;他们需要一个受管的自发的AI系统,能够:
- 实时诊断在缓存和数据库各级分布的根源
- 通过将可执行的缓解工具合成、核实和安装到苍蝇上实况服务器中间软件,进行自拍
- 通过计算每项行动的边际功劳,将有效的补救与有害的反射区分开来,达到自我标准
- 记住,通过将加密签名的操作活性变体分解成语义记忆,以减少重复发生相同故障的可能性
——通过动态重接自己的多agent分级来进行再造,用人类的签注来吸收未来的激增。
这个范式是"自演AI".
在这个代号为"Chrysalis工程"的Google Cloud研究参考架构中,我们演示了三个基础研究突破是如何结合到一个统一的,生产级的建筑,可以在Gemini平台和Agent开发基(ADK)上运行。 纵观模拟的"黑色星期五"缓存-挂牌出局,这一受管架构将从15分减到45分减到1.8秒,将HTTP 5xx错误率从72.7%降低到80.0%再降低到0.0%,将P99的延迟从519.8ms降低到18.2ms,并在~0.5ms的BigQuery评价了400个反事实交通政策。
关键的是,这种自演的建筑并不是作为不受约束的"黑盒"来运作的。 为确保agent保持安全、可靠、一致并遏制无赖行为,每个自主行动都以Google Cloud的防守深度安全堆绑在密码上:用于对提示词注入进行进取遥测和数据泄漏的模型装甲;Google Cloud密钥管理服务(Cloud KMS)和二进制授权中间软件,用于加密签名合成补丁并阻断未验证代码的安装;Open Policy Agent(OPA)政策控制器,用于执行严格的SLA护栏来防止自毁出行;以及双库托迪Human-the-Loop(HITL)治理,在对agent层级作任何永久性修改之前,需要明确的SRE签注。
为了了解为什么静态游戏本和copilots咨询在压力下失败,考虑一个经典的分布式故障级联:
——"内侵突袭":一闪一闪的促销带动了寒冷库存密钥上同时提出的退票请求突起。
- Cache Eviction: Google Cloud Memorystore (Rdis) 在严格的"最不近用"(LRU)内存限制下将冷键驱逐出,将缓存命中率折叠为零。
- 连接池饥饿:无缝交通级联直接进入Google Cloud SQL. 一旦 PostgreSQL 连接集合饱和度,接通的查询队列会超过超时阈值,服务开始以广泛的HTTP 500出错而失败。
- Fatal"3:00 AM Panic Reflex":在操作压力下,人机响应器和天真LLMs本能都试图重启数据库连接池。 在一次直播中,投放连接会锁定数据库内核,将尾翼的延后率提升近十倍,并驱动出错率到全部停用。
常规AI agents在部署时被用固定的系统提示和预先登记的工具被冻结。 当一个新的基础设施瓶颈袭击时,一个自我演变的AI架构通过在运行时合成针对具体情况的缓解工具来扩展这些基线能力,对每次干预的遥测影响进行评分,并为未来事件编纂经过核实的护栏。
对于真正的自主复原力,agent本身必须基于关于什么可行什么失败的经验证据来演化:
- 自主自选(工具合成):当agent诊断出数据库饿死是由一无所动的内进突起所驱动并意识到它缺乏内置的交通成形工具时,它在苍蝇上会写出自己的中间软件补丁(一个内出征令牌-活塞速率限制器),验证其抽象语法树(AST),用Cloud KMS签名,并直接将它放入API网关。
- 因果自律(评价什么是有效的,什么是失败):在稳定出局后,agent评价了事件期间所采取每一项行动的遥测三角洲。 它在数学上惩罚数据库重启反射,因为它恶化了延迟性,同时奖励其合成的速率限制器消除服务器错误。
- 认知内存与组织自重:为降低重复出行的可能性,agent将一个签名的操作原理写入向量内存中,在缓存被戳出时阻断数据库重启。 然后它修改了自己在RAM中的多agent地貌,委托了一位在主管agent下的长期交通造型专家Agent,因此未来的涌出在到达数据库之前被塑造。
自演云 agents是一个活跃的应用研究领域,重点是通过统一Google Cloud上的三个研究支柱来解决SRE Trilemma(平衡探索安全,结构适应性,多步信用分配):
1. 无服务反事实模拟(梦-RSI):
挑战:在现场生产中测试未核实的费率限制参数,有使合法收入紧缩或重新触发停产的风险。
机制:生产请求的痕迹被输出到Google BigQuery,引擎在分毫秒矢量化内存扫描中重放上百个候选符号-bucket配置,以识别离线的Pareto-优化政策。
2. 三层联合演进(APEX):
挑战:单层即时调制提供了脆弱的带状助听器,而不受约束的代码自改则有系统不稳定的风险。
机理:Agent演化被分出三个受支配的时间尺度:
- 第1层(Ephemeral Hotpatching):合成并挂载入模中间软件工具,以在秒内停止活出。
- 第2层(原则分解):将Cloud KMS签名的教程编码为Agent平台矢量搜索,因此禁用的行动对未来的警报被屏蔽。
- 第3层(Topology Rewiring):委托永久专家子agents进入活性的多agent定向回旋图(DAG).
3. 分级分层分配(AgentEvolver):
挑战:在混乱的事件中,有多种干预措施依次发生;粗糙的结果奖励不能孤立哪些步骤可以治愈服务,哪些步骤会造成附带损害。
机制:AgentEvolver 使用干涉前和后遥测三角塔(P99 latency,HTTP 5xx rate,以及数据库池饱和度)计算每个执行周期的边际差分信用,将受益补丁与错误补丁相隔离。
我们在Gemini平台上将这个架构作为互动应用程序实施。 上面的图表说明了整个生产目标服务、Agent开发工具包(ADK)多Agent Plane、自演引擎和防御深度治理管道的闭路工作流程。
agent自我进取的阶段:
- 发生什么:经过5个请求的热量基准后,50个请求同时的闪售猛增达到 15个冷库存键。
- 系统影响:内存存储器撞入率从100%降至0.0 %,云SQL连接池达到100%饱和度(2/2 有效连接锁定),P99 耐久性突起为312.8至519.8 ms,HTTP 500出错跳出为72.7%至80.0%(40个未通过检出).
- What Happens:遥测警报在Model Armor检查了进入的警报有效载荷以阻断提示词注入后,唤醒了由Gemini 4 Argon提供动力的Agent(ADK)开发工具(ADK)主管路由器Agent.
- 自主自选:诊断池耗竭,APEX 第1热patch合成器生成一个内相活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性活性在安装到API Gateway中端软件之前,补丁会通过三个自动治理门:
- AST Sandbox 访问器核实没有禁止的系统模块和动态执行汇。
- 业务厅政策主计长确认补充率符合企业最低服务水平(++ 10.0雷克/秒)。
- Cloud KMS & 二进制授权在文物摘要上签名(EC SIGN P256 SHA256)并验证中件边界的密钥出处。
- 系统影响:通过第二次50个请求激增,HTTP 500个出错降至0.0%,Cloud SQL池饱和度从100%降至0%,P99 耐久度恢复到18.2 ms,过快流量通过HTTP 429被清理出(40个请求节奏,10个来自温暖缓存)。
- What Happens: AgentEvolver Credit Attribution Engine对事件执行跨度进行审计,惩罚数据库重启反射(在跳到3,800 ms后)并奖励象征-bucket中件补丁(+0.97 credit).
原理分解:APEX 第2层原理分解器标志和索引一个不可改变的操作规则在Gemini平台矢量搜索中:"Cache explained: catabase reclain recorded forficious-bucket shedding".
- Tier-2 双拥核准:系统不依赖临时热拍,而是提议Tier-2 结构更新:委托永久交通整流专家Agent. 在UI中执行暂停,并呈现出一个双Custody SRE批准 Modal,要求待召SRE明确签出。
- 记忆ADK图重回:
在生产云运行时间(Google Kubernetes Engine, Cloud Run)中,容器图像是不可改变的;活出时在磁盘上重写出源文件会侵犯容器的完整性并需要干扰重启。 相反,APEX地层3 Cloud Toopology Adapter在发布密码人类认可令牌后,会执行内置定向回程图(DAG)更新:
- 利用专用的限速、排队深入检查和象征性补充工具,为RAM中的交通分层专家Agent提供证据。
- 将专家agent直接添加到主管路由器Agent的活性子agent图上,作为与诊断Agent(脱离诊断节点而未修改)平行的对等点。
- 更新主管的路由指示,并以零故障时间和零容器重启的方式,将直播的Google ADK Runner重新装入内存。
- 发生什么:事件要求将痕迹流入Google BigQuery,其中大查询反事实重放引擎执行400个反事实政策模拟,跨越象征能力和补充率。
- 子-Millisecond Discovery & Chaos Proof:矢量化的记忆中微量回放以~0.5毫秒完成所有400个扫射,识别出Pareto-优化政策(5-token容量,5.0 req/s速率,0-5xx出错). 最后,"好奇心混乱"(Curiosity Chaos Red-Team Agent)通过清理100%的缓存并同时点出200个请求,在被隔离的影子沙盒中测试了进化的系统。 新出道的wired交通造型专家Agent以0HTTP500出错和13%最大DB负载来吸收了突起,认证了该架构具有弹性。
指导原则:AI agent可以自主地发展其保护系统健康的业务策略,但其执行权在密码学上以不可改变的企业政策、硬件关键出处和明确的人监护为界限。
为了在生产中安全地部署自发的agents,我们的建筑实施了一个不间断的四门防御-深入的监管链。 每个合成补丁,d
本文由 AI宝库自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
OCT. 22, 2026
As enterprise cloud platforms scale to serve millions of concurrent users, tightly coupled microservices introduce cascading failure modes that no static runbook or threshold alert can anticipate. During high-velocity events like a Black Friday flash sale, Site Reliability Engineers (SREs) face an unforgiving reality: diagnose noisy distributed telemetry in seconds, or suffer cascading outages costing millions of dollars per minute.
While generative AI copilots have accelerated code authoring and log summarization, AI agents struggle to reliably remediate live production outages. An on-call engineer battling connection pool starvation does not need an AI agent offering troubleshooting tips or unverified pull requests that might not fix the root cause; they need a governed, self-evolving AI system that can:
- Diagnose distributed root causes across cache and database tiers in real time,
- Self-patch by synthesizing, verifying, and mounting executable mitigation tools into live server middleware on the fly,
- Self-critique by computing the marginal credit of every action to separate effective fixes from harmful reflexes,
- Remember by distilling cryptographically signed operational invariants into semantic memory to reduce the chance of identical failures recurring, and
- Re-architect by dynamically rewiring its own multi-agent hierarchy in memory with human sign-off to absorb future surges.
This paradigm is Self-Evolving AI.
In this Google Cloud research reference architecture, codenamed Project Chrysalis, we demonstrate how three foundational research breakthroughs combine into a unified, production-grade architecture that can run on Gemini platform and the Agent Development Kit (ADK). Across simulated Black Friday cache-stampede outages, this governed architecture cut time-to-first-mitigation from 15 to 45 minutes down to 1.8 seconds, reduced HTTP 5xx error rates from 72.7% to 80.0% down to 0.0%, dropped P99 latency from 519.8 ms to 18.2 ms, and evaluated 400 counterfactual traffic policies in BigQuery in ~0.5 ms.
Crucially, this self-evolving architecture does not operate as an unconstrained "black box". To ensure the agent remains secure, reliable, aligned, and contained against rogue behavior, every autonomous action is cryptographically bounded by Google Cloud's defense-in-depth security stack: Model Armor to sanitize incoming telemetry against prompt injections and data leaks; Google Cloud Key Management Service (Cloud KMS) and Binary Authorization middleware to cryptographically sign synthesized patches and block unverified code from mounting; Open Policy Agent (OPA) Policy Controller to enforce strict SLA guardrails that prevent self-inflicted outages; and Dual-Custody Human-in-the-Loop (HITL) governance to require explicit SRE sign-off before any permanent changes are made to the agent hierarchy.
To see why static playbooks and advisory copilots fail under pressure, consider a classic distributed failure cascade:
- The Ingress Surge: A flash promotion drives a sudden burst of concurrent checkout requests across cold inventory keys.
- The Cache Eviction: Google Cloud Memorystore (Redis) evicts cold keys under strict Least Recently Used (LRU) memory limits, collapsing the cache hit ratio to zero.
- Connection Pool Starvation: Uncached traffic cascades directly into Google Cloud SQL. Once the PostgreSQL connection pool saturates, incoming queries queue past timeout thresholds and the service begins failing with widespread HTTP 500 errors.
- The Fatal "3:00 AM Panic Reflex": Under operational stress, both human responders and naive LLMs instinctively try to restart the database connection pool. In a live stampede, dropping connections locks the database kernel, spikes tail latency nearly tenfold, and drives error rates toward total outage.
Conventional AI agents are frozen at deployment with fixed system prompts and pre-registered tools. When a novel infrastructure bottleneck strikes, a self-evolving AI architecture extends those baseline capabilities by synthesizing situation-specific mitigation tools at runtime, scoring the telemetry impact of every intervention, and codifying verified guardrails for future incidents.
For true autonomous resilience, the agent itself must evolve based on empirical evidence of what worked and what failed:
- Autonomous Self-Patching (Tool Synthesis): When the agent diagnoses that database starvation is driven by an unthrottled ingress surge and realizes it lacks a built-in traffic-shaping tool, it writes its own middleware patch on the fly (an in-memory token-bucket rate limiter), validates its Abstract Syntax Tree (AST), signs it with Cloud KMS, and hot-mounts it directly into the API Gateway.
- Causal Self-Critique (Evaluating What Worked vs. What Failed): After stabilizing the outage, the agent evaluates the telemetry delta of every action taken during the incident. It mathematically penalizes the database restart reflex because it worsened latency, while rewarding its synthesized rate limiter for eliminating server errors.
- Cognitive Memory & Organizational Self-Rewiring: To reduce the likelihood of a repeat outage, the agent writes a signed operational principle into vector memory that blocks database restarts during cache stampedes. It then modifies its own multi-agent topology in RAM, commissioning a permanent Traffic Shaping Specialist Agent under the supervisor agent so future surges are shaped before they reach the database.
Self-evolving cloud agents are an active area of applied research focused on resolving the SRE Trilemma (balancing exploration safety, structural adaptability, and multi-step credit assignment) by unifying three research pillars on Google Cloud:
1.Serverless Counterfactual Simulation (Dream-RSI):
Challenge: Testing unvetted rate-limiting parameters in live production risks throttling legitimate revenue or re-triggering outages.
Mechanism: Production request traces are exported to Google BigQuery, where the engine replays hundreds of candidate token-bucket configurations in sub-millisecond vectorized memory sweeps to identify Pareto-optimal policies offline.
2. Tri-Level Co-Evolution (APEX):
Challenge: Single-layer prompt tuning provides fragile band-aids, while unconstrained code self-modification risks system instability.
Mechanism: Agent evolution is separated into three governed timescales:
- Layer 1 (Ephemeral Hotpatching): Synthesizes and mounts in-memory middleware tools to halt active outages in seconds.
- Layer 2 (Principle Distillation): Encodes Cloud KMS-signed lessons into Agent Platform Vector Search so contraindicated actions are blocked on future alerts.
- Layer 3 (Topology Rewiring): Commissions permanent specialist sub-agents into the active multi-agent Directed Acyclic Graph (DAG).
3. Step-Level Shapley Attribution (AgentEvolver):
Challenge: During chaotic incidents, multiple interventions occur in sequence; coarse outcome rewards cannot isolate which step healed the service and which caused collateral damage.
Mechanism: AgentEvolver computes marginal Shapley credit across each execution span using pre- and post-intervention telemetry deltas (P99 latency, HTTP 5xx rate, and database pool saturation), isolating beneficial patches from erroneous ones.
We implemented this architecture as an interactive application on Gemini platform. The diagram above illustrates the closed-loop workflow across the Production Target Service, Agent Development Kit (ADK) Multi-Agent Plane, Self-Evolution Engine, and Defense-in-Depth Governance Pipeline.
Each phase of the interactive demonstration showcases a distinct stage of the agent's self-evolution:
- What Happens: Following a 5-request warm baseline, a 50-request concurrent flash-sale surge hits 15 cold inventory keys.
- System Impact: Memorystore hit ratio drops from 100% to 0.0%, the Cloud SQL connection pool hits 100% saturation (2/2 active connections locked), P99 latency spikes to 312.8 to 519.8 ms, and HTTP 500 errors jump to 72.7% to 80.0% (40 failed checkouts).
- What Happens: Telemetry alerts wake the Agent Development Kit (ADK) Supervisor Router Agent powered by Gemini 4 Argon, after Model Armor inspects the incoming alert payload to block prompt injections.
- Autonomous Self-Patching: Diagnosing pool exhaustion, the APEX Layer 1 Hotpatch Synthesizer generates an in-memory token-bucket rate limiter (10-token capacity, 40.0 req/s replenishment). Before mounting into the API Gateway middleware, the patch passes three automated governance gates:
- AST Sandbox Visitor verifies that forbidden system modules and dynamic execution sinks are absent.
- OPA Policy Controller confirms the replenishment rate meets the enterprise minimum SLA (>= 10.0 req/s).
- Cloud KMS & Binary Authorization signs the artifact digest (EC_SIGN_P256_SHA256) and verifies key provenance at the middleware boundary.
- System Impact: Across a second 50-request surge, HTTP 500 errors drop to 0.0%, Cloud SQL pool saturation drops from 100% to 0%, P99 latency recovers to 18.2 ms, and excess burst traffic is cleanly shed via HTTP 429 (40 requests throttled, 10 served from warm cache).
- What Happens: The AgentEvolver Credit Attribution Engine audits the incident execution spans, penalizing the database restart reflex ( after spiking latency to 3,800 ms) and rewarding the token-bucket middleware patch (+0.97 credit).
- Principle Distillation: The APEX Layer 2 Principle Distiller signs and indexes an immutable operational rule into Gemini Platform Vector Search: "Cache eviction stampede: database restart contraindicated; apply token-bucket load shedding."
- Tier-2 Dual-Custody Approval: Rather than relying on temporary hotpatches, the system proposes a Tier-2 structural update: commissioning a permanent Traffic Shaping Specialist Agent. Execution pauses in the UI and presents a Dual-Custody SRE Approval Modal requiring explicit sign-off from the on-call SRE.
- In-Memory ADK Graph Rewiring:
In production cloud runtimes (Google Kubernetes Engine, Cloud Run), container images are immutable; rewriting source files on disk during a live outage violates container integrity and requires disruptive restarts. Instead, once issued a cryptographic human-approval token, the APEX Layer 3 Cloud Topology Adapter executes an in-memory Directed Acyclic Graph (DAG) update:
- Instantiates the Traffic Shaping Specialist Agent in RAM with dedicated rate-limiting, queue-depth inspection, and token-replenishment tools.
- Appends the specialist agent directly to the Supervisor Router Agent's active sub-agent graph as a parallel peer alongside the Diagnostic Agent (leaving the diagnostic node unmodified).
- Updates the supervisor's routing instructions and re-binds the live Google ADK Runner in memory with zero downtime and zero container restarts.
- What Happens: Incident request traces flow into Google BigQuery, where the BigQuery Counterfactual Replay Engine executes 400 counterfactual policy simulations across token capacities and replenishment rates.
- Sub-Millisecond Discovery & Chaos Proof: Vectorized in-memory trace replay completes all 400 sweeps in ~0.5 milliseconds, identifying the Pareto-optimal policy (5-token capacity, 5.0 req/s rate, zero 5xx errors). Finally, the Curiosity Chaos Red-Team Agent stress-tests the evolved system in an isolated shadow sandbox by purging 100% of the cache and firing 200 concurrent requests. The newly wired Traffic Shaping Specialist Agent absorbs the surge with 0 HTTP 500 errors and 13% max DB load, certifying the architecture as resilient.
The Governing Principle: An AI agent may autonomously evolve its operational tactics to defend system health, but its authority to execute is cryptographically bounded by immutable enterprise policies, hardware key provenance, and explicit human custody.
To deploy self-evolving agents safely in production, our architecture enforces an unbroken four-gate defense-in-depth chain of custody. Every synthesized patch, d
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.