人工智能
以 Amazon Nova 法案建立意图驱动 UI 测试
为了将意图驱动的UI测试规模推广到其产品团队Epsilon，一个数据，技术和营销服务公司以及Publicis Groupe的一部分，转向了Amazon Nova Act. 随着数以亿计的消费者简介和上千个品牌关系，Epsilon需要UI测试，可以跟上快速的应用变化。 解决方案无法为每个产品组创造维护费。
在Epsilon内部，Agentic平台团队(50名工程师)建设了共享的基础设施，使其他产品团队能够安全地建设，部署，治理并运行AI agents. 通过这个共享平台，球队获得基于技能的agent建设者。 他们还获得联邦身份和飞行前授权、示范背景协议(MCP)整合和交叉账户提供。 当一个产品团队想采用AI工具时，他们获得的能力没有额外的配置，也没有自己重新解析身份，安全，浏览器基础设施，或者模型集成。
管理能力之一是UI测试。 以Amazon Nova Act和Amazon Bedrock Agent Core浏览器(Amazon Bedrock Agent Core的能力)，埃普西隆的平台团队可以提供弹性，意向驱动的测试作为平台上的团队服务。
我们透过Epsilon的架构提供UI测试作为管理平台服务
挑战：扩大覆盖面，而不是取而代之
Epsilon 现有的UI测试实践没有被打破。 团队维护了一个基于Playwright的回归套件，验证了整个网络应用程序的功能，运行良好。 剧作家今天仍然是他们的回归支柱。
Epsilon被识别的机会是添加剂，而不是纠正：
- 脚本Playwright测试对确定回归检查有效。 然而，它们与选择者和页面结构紧密结合。 化妆的UI修改即使在基础功能是声音时也能打破。
- 多个产品组需要针对自己的应用进行全面测试。 建立浏览器基础设施，管理依赖性，并与每个团队的身份系统相融合，造成了重复的努力。
- 平台团队希望UI测试成为符合他们现有的agent模型的管理能力，而不是每个机上团队必须操作的自定义CLI.
目标：在已存在的Playwright回归套件上进行层层意图测试。 作为多租户平台服务交付，使每个Epsilon产品团队能够通过他们用于其他agent能力的相同的API表面来消耗。
为什么Epsilon选择了Amazon Nova Act和Amazon Bedrock Agent Core浏览器工具
Epsilon根据两项要求评价了选项：自然语言测试作者(以补充基于选择器的Playwright测试)和管理的浏览器基础设施(以适应其多租户平台模型). 随着Amazon Nova Act与Amazon Bedrock Agent Core浏览器工具的结合，Epsilon可以同时解决两者。
- 自然语阶。 测试步骤是由Nova Act执行的平正英语指令。 它们对化妆的UI变化具有弹性，非工程师可以读取。 类似“确认奖励的平衡是显而易见的”这样的一步，其年龄远优于简单的CSS选择器。
- AWS管理，ephemeral浏览器会话。 使用Amazon Bedrock Agent Core浏览器工具，团队会获得孤立的浏览器实例，没有二进制符可以发送，也没有显示服务器可以维护。 Amazon Bedrock Agent Core浏览器工具也为人类观察或接收提供活视图(可用于CAPTCHAs或多要素认证)，会话录音到Amazon简单存储服务(Amazon S3)和AWS CloudTrail审计，均没有附加配置。
- AWS身份和访问管理(AWS IAM)基于认证。 通过Nova Act，您可以通过标准AWS认证链(任务角色)认证。 这符合Epsilon现有的安全模式，而不分发单独的API密钥。
- 交叉账户友好。 使用Nova Act和Amazon Bedrock Agent Core浏览器工具，您可以进入您现有的AWS安全Token Service(AWS STS) AssumeRole 提供设备。 每个入职的团队可以在自己的AWS账户内在一个范围化的角色下运行测试。 这保持了孤立和可审计性。
Epsilon特别选择了Amazon Bedrock Agent Core浏览器工具，因为它取消了管理浏览器二进制，显示服务器和会话生命周期的操作负担。 代理平台小组对拥有这一基础设施没有兴趣。
解决办法：结构和执行
Epsilon的解决方案有三个主要组成部分：一个用于创建工作流程的平台API，一个用于执行它们的背景任务执行者，以及驱动浏览器的Nova Act和Amazon Bedrock Agent Core浏览器工具集成。
平台架构
Epsilon的UI测试能力遵循一条清晰的管道：
- 用户通过平台API创建工作流程，指定一个目标 url和自然语言步骤列表。
- 在创建时，平台在AWS上登记了Nova Act工作流程定义。
- 执行时，一个背景任务运行器(Celery，用于同步任务处理的Python框架)打开了Amazon Bedrock Agent Core浏览器工具浏览器会话。 它通过Nova Act在Chrome DevTools Protocol(CDP) WebSocket(程序上控制Chromium浏览器的标准界面)上运行每个步骤，包括有条件的逻辑(例如，“如果出现旅行盒，跳过它”).
- 系统每步抓取一个截图，每步通过/失败音轨，计算总体通过率，并存储一切(结果为JSON，截图为Amazon S3，会话ID).
- 完成钩起火，结果被记录到平台上
图1 显示 EpsilonUI测试服务的分层架构。 用户请求从Agent Cloud前端通过一个API Gateway进行认证和权限检查。 API层(Django REST)暴露出工作流程，执行，自动化套件和统计的端点。 业务逻辑层包括一名工作流程管理员、执行操作员和成套协调员。 测试任务由Amazon简单排队服务(Amazon SQS)排队，由运行于Amazon弹性容器服务(Amazon ECS)上的跑跑者服务接取，其中包含Nova Act引擎和无头铬浏览器。 结果，截图，轨迹，和HTML报告被存储在Amazon S3中。
该能力被曝光为标准agent(ui-test-agent)，具有两种技能：
- ui-test-recorder：以交互方式探索一种情景并记录自然语言步骤。
- ui-test-executor：运行已记录步骤并返回结构化结果。
机上团队消耗UI测试的方式与消耗任何其他平台agent的方式相同。 无需增加基础设施。 入职过程很简单。 平台团队为新团队提供跨账户的IAM角色，注册其目标应用URL，并通过现有的平台API授权访问ui-test-agent. 从此，团队创建了工作流程，触发了处决，并通过他们已经用于其他平台能力的相同接口来获取结果。
核心一体化
Nova Act + Amazon Bedrock Agent Core 浏览器工具集成简洁：
这种模式(打开一个有管理的浏览器会话，通过 CDP WebSocket 连接，并用自然语言驱动动作)是基础。 Epsilon用他们的平台的多租房管弦、结果捕捉和可观察层来包装。
示例：启动 agent 流
为了说明基于意图的测试，这里有一个来自Epsilon平台的代表性工作流程。 它进行登录，表单输入，有条件处理，滚动，并进行校验。 以下步骤突出了新法的不同模式：
步骤4和步骤7显示了有条件的逻辑：“ 如果元素出现，处理它 ” 和 “ 如果目标不可见，滚动直到它出现 ”。 这些模式要求在传统框架内明确分支。 Nova Act通过语言理解来自然地处理它们。
运行时间隔离
Epsilon在专用跑道服务(FastAPI sidecar)中运行UI测试，该测试将nova-act 3.x与Workflow和IAM认证支持相接，外加Amazon Bedrock Agent Core浏览器工具客户端。 这使得依赖性与主后端分离，如果跑者残疾，工人将回到执行过程中，在部署期间提供复原力。
最佳做法
根据他们建立UI测试作为平台能力的经验，Epsilon团队建议：
1. 将步骤写成意图，而不是机械
基于意向的步骤在UI重新设计后幸存。 产品经理和质量保证(QA)领导谁可能不写代码可以直接审查。
2. 隔离新法案运行时间
Pin Nova Act和Amazon Bedrock Agent Core浏览器工具在专用服务或容器中的依赖性。 Epsilon得知Nova Act的签名端点针对我们东一区，用自己的依赖锁文件在单独的服务中运行它可以防止难以调试的冲突。
3. 重新使用记录的轨迹来降低成本
Epsilon在草稿状态下使用轨迹重试，保存成功的轨迹，并在活性状态下重放。 重复运行跳过冗余的大型语言模式(LLM)“思考，”既能减少延迟，又能降低回归式再处决的成本。
4. 从第一天起多租期的设计
使用跨账户的AssumeRole，并带有IAM政策范围。 每个房客的测试都是在他们自己的职责下进行的。 这提供了孤立、可审计性和故障范围控制，没有每个小组的定制基础设施。
5. 对管理下的可观察性有优势
使用Amazon Bedrock Agent Core浏览器工具，团队获得现场取景，会话录音，并进行AWS CloudTrail集成。 Epsilon使用这些内置能力，而不是建立自定义可观察性，减少维护，并让合规小组熟悉审计表面。
6. Amazon Bedrock Agent Core浏览器账户工具查看器行为
在提示Nova Act步骤时，要说明Amazon Bedrock Agent Core浏览器工具的浏览器取景口默认值。 引用滚动步骤(例如“滚动到目标声明文本框可见”)证实Nova Act正确处理页面下部的内容。
成果和影响
Epsilon的UI测试能力在多个团队中运行，多租户架构在生产中得到了证明。
以下是衡量的结果：
- 4-6x更快的测试作者(基于Epsilon的内部基准测试). 一个18步的代表工作流程用5-10分钟的时间来写作Nova Act，而对应的Playwright脚本(80-150行代码)则需要30-90分钟。 Playwright脚本中的每个交互步骤都需要检查DOM，选择一个稳定的选择器，并编码明确的等待，断言，以及有条件的个案的分支。 Nova Act用18个自然语言指令和零选择器或DOM检查来取代每步的间接费用，并用简单的英语表达有条件的逻辑(如游览箱和滚动至可视步骤).
- 轨迹回放100%通过率(基于Epsilon的内部基准测试). 在工作流程通过初始探索阶段(典型的为4–5即时迭接)后，Epsilon保存了成功的轨迹并重放其确定性。 节省的轨迹实现100%的通过率通过反复运行。
- 低屏障测试作者。 台阶读作普通英语。 核心测试工程师以外的团队成员可以阅读，审查和提议测试，扩大质量所有权。
- 内置审计线索 AWS 计算机
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Artificial Intelligence
Building intent-driven UI testing with Amazon Nova Act
To scale intent-driven UI testing across its product teams, Epsilon, a data, technology, and marketing-services company and part of Publicis Groupe, turned to Amazon Nova Act. With hundreds of millions of consumer profiles and thousands of brand relationships, Epsilon needed UI testing that could keep pace with rapid application changes. The solution could not create maintenance overhead for every product team.
Within Epsilon, the Agentic Platform team (50 engineers) builds shared infrastructure so other product teams can build, deploy, govern, and run AI agents safely. With this shared platform, teams get a skills-based agent builder. They also get federated identity and pre-flight authorization, Model Context Protocol (MCP) integration, and cross-account provisioning. When a product team wants to adopt AI tooling, they get capabilities without additional configuration and without re-solving identity, security, browser infrastructure, or model integration on their own.
One of those managed capabilities is UI testing. With Amazon Nova Act and Amazon Bedrock AgentCore Browser (a capability of Amazon Bedrock AgentCore), Epsilon’s platform team could deliver resilient, intent-driven testing as a service to teams on the platform.
In this post, we walk through Epsilon’s architecture for delivering UI testing as a managed platform service, show the core Nova Act and Amazon Bedrock AgentCore Browser Tool integration code, highlight a concrete 18-step test workflow, and share best practices and measured results.
The challenge: Enhancing coverage, not replacing what works
Epsilon’s existing UI testing practice was not broken. The team maintained a Playwright-based regression suite that validated functionality across their web applications, and it worked well. Playwright remains their regression backbone today.
The opportunity Epsilon identified was additive, not corrective:
- Scripted Playwright tests are effective for deterministic regression checks. However, they are tightly coupled to selectors and page structure. Cosmetic UI changes can break them even when the underlying functionality is sound.
- Multiple product teams needed full testing coverage against their own applications. Standing up browser infrastructure, managing dependencies, and integrating with identity systems per team created duplicative effort.
- The platform team wanted UI testing to be a managed capability that fit their existing agent model, not a custom CLI that every onboarded team had to operate.
The goal: layer intent-based testing on top of the existing Playwright regression suite. Deliver it as a multi-tenant platform service so that each Epsilon product team could consume through the same API surface they used for every other agent capability.
Why Epsilon chose Amazon Nova Act and the Amazon Bedrock AgentCore Browser Tool
Epsilon evaluated options against two requirements: natural-language test authoring (to complement selector-based Playwright tests) and managed browser infrastructure (to fit their multi-tenant platform model). With Amazon Nova Act combined with Amazon Bedrock AgentCore Browser Tool, Epsilon could address both.
- Natural-language steps. Test steps are plain-English instructions executed by Nova Act. They are resilient to cosmetic UI changes and readable by non-engineers. A step like “Confirm the rewards balance is visible” ages far better than a brittle CSS selector.
- AWS managed, ephemeral browser sessions. With Amazon Bedrock AgentCore Browser Tool, teams get isolated browser instances with no binaries to ship and no display server to maintain. Amazon Bedrock AgentCore Browser Tool also offers live view for human observation or takeover (useful for CAPTCHAs or multi-factor authentication), session recording to Amazon Simple Storage Service (Amazon S3), and AWS CloudTrail audit, all without additional configuration.
- AWS Identity and Access Management (AWS IAM)-based authentication. With Nova Act, you can authenticate through the standard AWS credential chain (the task role). This fits Epsilon’s existing security model without distributing separate API keys.
- Cross-account friendly. With Nova Act and Amazon Bedrock AgentCore Browser Tool, you can slot into your existing AWS Security Token Service (AWS STS) AssumeRole provisioning. Each onboarded team can run tests inside their own AWS account under a scoped role. This maintains isolation and auditability.
Epsilon chose Amazon Bedrock AgentCore Browser Tool specifically because it eliminates the operational burden of managing browser binaries, display servers, and session lifecycle. The Agentic Platform team had no interest in owning that infrastructure.
The solution: Architecture and implementation
Epsilon’s solution has three main components: a platform API for creating workflows, a background task runner for executing them, and the Nova Act and Amazon Bedrock AgentCore Browser Tool integration that drives the browser.
Platform architecture
Epsilon’s UI testing capability follows a clear pipeline:
- A user creates a workflow through the platform API, specifying a target_url and a list of natural-language steps.
- On create, the platform registers a Nova Act workflow definition on AWS.
- On execute, a background task runner (Celery, a Python framework for asynchronous job processing) opens an Amazon Bedrock AgentCore Browser Tool browser session. It runs each step through Nova Act over a Chrome DevTools Protocol (CDP) WebSocket (the standard interface for programmatically controlling Chromium browsers), including conditional logic (for example, “if a tour box appears, skip it”).
- The system captures a screenshot per step, tracks pass/fail per step, computes an overall pass rate, and stores everything (results JSON, screenshots in Amazon S3, session ID).
- A completion hook fires so the result is recorded back into the platform.
Figure 1 shows the layered architecture of Epsilon’s UI Testing Service. User requests flow from the Agent Cloud Frontend through an API Gateway with authentication and permission checks. The API layer (Django REST) exposes endpoints for workflows, executions, automation suites, and stats. The business logic layer includes a Workflow Manager, Execution Orchestrator, and Suite Coordinator. Test jobs are queued in Amazon Simple Queue Service (Amazon SQS) and picked up by the Runner Service running on Amazon Elastic Container Service (Amazon ECS), which contains the Nova Act engine and a headless Chromium browser. Results, screenshots, trajectories, and HTML reports are stored in Amazon S3.
The capability is exposed as a standard agent (ui-test-agent) with two skills:
- ui-test-recorder: Explore a scenario interactively and record natural-language steps.
- ui-test-executor: Run recorded steps and return structured results.
Onboarded teams consume UI testing the same way they consume any other platform agent. No additional infrastructure to manage. The onboarding process is straightforward. The platform team provisions a cross-account IAM role for the new team, registers their target application URLs, and grants access to the ui-test-agent through the existing platform API. From that point, teams create workflows, trigger executions, and retrieve results through the same interface they already use for other platform capabilities.
The core integration
The Nova Act + Amazon Bedrock AgentCore Browser Tool integration is concise:
This pattern (open a managed browser session, connect through the CDP WebSocket, and drive actions with natural language) is the foundation. Epsilon wraps it with their platform’s multi-tenant orchestration, result capture, and observability layers.
Example: Automating the ‘Create an agent’ flow
To illustrate intent-based testing, here is a representative workflow from Epsilon’s platform. It exercises login, form entry, conditional handling, scrolling, and verification. The following steps highlight the differentiating patterns of Nova Act:
Steps 4 and 7 demonstrate conditional logic: “if an element appears, handle it” and “if a target isn’t visible, scroll until it is.” These patterns would require explicit branching in a traditional framework. Nova Act handles them naturally through language understanding.
Runtime isolation
Epsilon runs UI tests in a dedicated runner service (a FastAPI sidecar) that pins nova-act 3.x with Workflow and IAM authentication support, plus the Amazon Bedrock AgentCore Browser Tool client. This keeps dependencies separate from the main backend so they don’t collide with the Strands agent stack. The worker falls back to in-process execution if the runner is disabled, providing resilience during deployments.
Best practices
Based on their experience building UI testing as a platform capability, the Epsilon team recommends:
1. Write steps as intent, not mechanics
Intent-based steps survive UI redesigns. Product managers and quality assurance (QA) leads who might not write code can review them directly.
2. Isolate the Nova Act runtime
Pin Nova Act and Amazon Bedrock AgentCore Browser Tool dependencies in a dedicated service or container. Epsilon learned that the signing endpoint for Nova Act targets us-east-1 regardless of the browser session’s region. Running it in a separate service with its own dependency lockfile prevents hard-to-debug conflicts.
3. Reuse recorded trajectories to cut cost
Epsilon uses trajectory retries in draft state, saves the successful trajectory, and replays it in active state. Repeat runs skip redundant large language model (LLM) “thinking,” reducing both latency and cost for regression-style repeated executions.
4. Design for multi-tenancy from day one
Use cross-account AssumeRole with scoped IAM policies. Each tenant’s tests run under their own role in their own account. This provides isolation, auditability, and failure scope containment without custom infrastructure per team.
5. Lean on managed observability
With Amazon Bedrock AgentCore Browser Tool, teams get live view, session recording, and AWS CloudTrail integration. Epsilon uses these built-in capabilities instead of building custom observability, reducing maintenance and giving compliance teams familiar audit surfaces.
6. Account for Amazon Bedrock AgentCore Browser Tool viewport behavior
When prompting Nova Act steps, account for the browser viewport defaults of Amazon Bedrock AgentCore Browser Tool. Steps that reference scrolling (for example, “scroll down until the Goal Statement textbox is visible”) verify that Nova Act handles content in the lower portion of the page correctly.
Results and impact
Epsilon’s UI testing capability is operational across multiple teams, with the multi-tenant architecture proven in production.
Here are the measured outcomes:
- 4-6x faster test authoring (based on Epsilon’s internal benchmarks). A representative 18-step workflow took 5–10 minutes to author with Nova Act, compared to 30–90 minutes for the equivalent Playwright script (80–150 lines of code). Each interactive step in the Playwright script requires inspecting the DOM, choosing a stable selector, and coding explicit waits, assertions, and branching for conditional cases. Nova Act replaces that per-step overhead with 18 natural-language instructions and zero selector or DOM inspection, with conditional logic (such as the tour-box and scroll-until-visible steps) expressed in plain English.
- 100 percent pass rate on trajectory replay (based on Epsilon’s internal benchmarks). After a workflow passes the initial explore phase (typically 4–5 prompt iterations), Epsilon saves the successful trajectory and replays it deterministically. Saved trajectories achieve a 100 percent pass rate across repeated runs.
- Lower barrier to test authoring. Steps read like plain English. Team members beyond core test engineers can read, review, and propose tests, broadening ownership of quality.
- Built-in audit trail. AWS Cl
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.