TCS 如何基于 Amazon Bedrock AgentCore 构建智能体测试编排器

This post was co-written with Varun Valsan from Tata Consultancy Services (TCS).

Tata Consultancy Services (TCS) runs large-scale testing for Banking, Financial Services, and Insurance (BFSI) clients, where the slowest part of a release is rarely writing application code. The larger constraint is the chain of work that follows: interpreting requirements, designing scenarios, writing and maintaining automation, preparing execution environments, investigating failures, and assembling evidence for review.

BFSI clients need to increase testing throughput while preserving the controls expected in financial services. The objective wasn’t to replace quality engineers. It was to remove repetitive translation and execution work so that engineers could spend more time on risk analysis, exploratory testing, and release decisions. These industry needs generated four specific requirements for TCS. First, every test must be grounded in project-specific context rather than generic prompts. Second, it had to cover both API and browser-based UI testing to match how BFSI applications are built. Third, security is paramount, so every tenant’s data and knowledge store must be isolated. Finally, it must provide a repeatable path from requirement ingestion to test evidence and reporting, so the gains hold across teams and releases. In this post, we show how TCS built an agentic test orchestrator that turns requirements into executable, governed tests, and what it changed for its BFSI testing teams.

Why conventional automation and systems weren’t enough

TCS already uses traditional test automation for its clients, and it does improve execution speed. But it leaves the slow work untouched: teams must still create the scenarios, convert them into scripts, maintain locators and dependencies, and diagnose failures. When a linked user story changes, scripts generated for an earlier story can quickly become stale. A single generative AI prompt also doesn’t provide the control needed for a multi-stage engineering workflow.

TCS therefore designed the Agentic Test Orchestrator to decompose testing into specialized tasks. Each task has its own agent, tools, validation rules, and context. A deterministic supervisor routes work, while an execution workflow controls retries, error classification, persistence, and reporting.

To run these agents, TCS chose Amazon Bedrock AgentCore, a service to build, connect, and optimize agents at scale, with any framework or model. Three capabilities drove the decision. First, Amazon Bedrock offers a choice of models, so each agent uses the model that fits its task, such as scenario design or code generation, and TCS can adopt newer models as they become available. Second, AgentCore runtime, a capability of Amazon Bedrock AgentCore, scales with demand, so testers across many tenants can run generation and execution sessions at the same time without TCS provisioning or tuning agent servers. Third, the runtime runs each session in its own isolated microVM, which gives TCS session-level separation for its multi-tenant platform without having to build that layer itself. Together, these capabilities let TCS focus its engineering effort on testing logic rather than agent infrastructure, which shortened the time to bring the platform to its BFSI clients.

Solution overview

The following diagram shows the end-to-end flow of the platform across five stages, with a per-tenant security layer underneath. Inputs from Jira, business requirement documents (BRDs), and GitLab enter a tenant web application that runs on Amazon Elastic Kubernetes Service (Amazon EKS) behind Amazon CloudFront, with sign-in through Amazon Cognito. TCS uses Amazon Bedrock AgentCore runtime to host the platform’s specialized agents, built with the Strands Agents SDK. These agents generate the tests, refine them with the tester, and manage their execution. Amazon Elastic Container Service (Amazon ECS) runs the generated tests. Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB store the results, and Amazon CloudWatch captures the logs. Throughout, AWS WAF, Amazon Cognito, AWS Key Management Service (AWS KMS), and AWS Secrets Manager enforce per-tenant security and isolation.

Figure 1: High-level solution architecture

The TCS Agentic Test Orchestrator is a multi-tenant software as a service (SaaS) solution built on AWS. A tester provides requirements through Jira stories, business requirement documents (BRDs), Swagger specifications, uploaded documents, GitLab source folders, or captured UI flows. From those inputs the platform then generates test scenarios, Gherkin feature files, and executable Java or Python tests. It runs those tests on Amazon ECS, stores artifacts and results, and shows progress and reports in a per-tenant web application. The sections that follow describe how each part of the platform meets one of the four requirements TCS set.

1. Separate the control plane from each tenant application plane

TCS required strict isolation of each tenant’s identity, data, and keys, so the platform uses pooled compute with siloed data and identity. A shared control plane provides the Amazon EKS cluster, container registry, administrative portal, tenant registry, onboarding automation, Amazon CloudFront distributions, and AWS WAF protection. Each tenant application plane receives its own Kubernetes namespace, Amazon Cognito user pool, AWS KMS key, Amazon S3 bucket, Amazon DynamoDB tables, and Amazon Bedrock Knowledge Bases. This separation gives each BFSI tenant the data and key boundaries that financial-services controls demand.

Tenant onboarding is event driven. The administrative API writes a pending tenant record, emits an event through Amazon EventBridge, and starts an AWS CodeBuild project that deploys the tenant infrastructure with AWS Cloud Development Kit (AWS CDK). This pattern makes tenant provisioning repeatable while retaining tenant-specific identity and data boundaries that TCS needed.

2. Ground every agent in project context

TCS’s first requirement was that generation reflect each project’s domain rather than a generic prompt, so grounding is built into the platform. The platform stores uploaded requirements and generated artifacts in Amazon S3 and session metadata in Amazon DynamoDB. Project documents and synchronized source files are indexed in Amazon Bedrock Knowledge Bases using vector embeddings. Before material reaches an agent, retrieval is filtered by tenant, project, and session context, so one client’s context doesn’t inform another’s tests.

This context layer supports the many ways a BFSI team expresses requirements: Jira stories and linked story chains, BRDs, Swagger definitions, reference code, locator documents, setup modules, test data, and GitLab source folders. For UI automation, a separate capture service uses Puppeteer to convert a URL and a natural-language goal into a UI flow document and an XPath locator document.

3. Use specialized agents instead of one large prompt

Because a single prompt can’t give a regulated workflow the control it needs, the platform divides the work among specialized agents that run as containerized services in Amazon Bedrock AgentCore runtime using the Strands Agents SDK. A generation runtime routes requests to dedicated scenario, Gherkin, code-generation, and validation tools. Additional runtimes support conversational refinement, BRD-to-story conversion, incremental generation across linked Jira stories, and test execution. Each agent owns a stage of the lifecycle. Here’s a brief overview of each specialized agent:

Generation agent: Creates structured scenarios, converts them to Gherkin, generates Java or Python automation, and validates generated code.

Chat agent: Classifies a request as an information query or an update request, then answers or revises the active artifact.

Dynamic task agent: Retrieves BRD context and produces a structured user story with requirements and acceptance criteria.

Incremental agent: Extends scenarios, Gherkin, or code across a linked Jira story chain instead of starting from scratch.

Execution agent: Fetches artifacts, chooses an execution environment, runs tests, classifies failures, applies controlled fixes, and saves final results.

4. Route API and UI tests to the right execution environment

TCS needed to cover both API and browser-based UI testing, which place different demands on infrastructure, so the execution agent detects the test type from the generated code and routes it accordingly. API tests are routed to Amazon ECS on AWS Fargate. Selenium UI tests are routed to an Amazon ECS cluster backed by Amazon Elastic Compute Cloud (Amazon EC2), where the executor can run Chrome with a virtual display.

For UI runs, the executor can capture the virtual display with FFmpeg and publish an HTTP Live Streaming (HLS) stream to Amazon S3 for delivery through Amazon CloudFront. Test logs are sent to Amazon CloudWatch, and generated Cucumber reports are stored in Amazon S3 with time-limited access.

5. Control the execution and correction loop

To keep automated correction safe enough for a BFSI pipeline, the execution agent defines its fetch, execute, fix, and save steps as a LangGraph workflow. LangGraph’s explicit state graph gives TCS a fixed step order and a bounded retry loop, while AgentCore runtime hosts, scales, and isolates the agent that runs it. The fetch node retrieves the test project and generated assets, sanitizes files, and detects the framework. The execute node launches the selected ECS task, monitors status, retrieves logs, and parses results.

When a failure is classified as a code error and the retry policy allows it, a fixing agent receives a constrained error context and proposes a targeted correction. Safety checks reject empty files, unexpected new files, or extreme changes in file size. Environment errors, such as connectivity or authentication failures, are not sent to the code fixer. The save node persists the final code, status, attempts, logs, parsed results, and report location.

6. Security and operational controls

Because the solution serves regulated BFSI tenants, it applies controls at the identity, network, data, and application layers. For identity, separate Amazon Cognito user pools serve platform administrators and each tenant, with backend JSON Web Token (JWT) validation and role-based authorization. For data, each tenant has its own AWS KMS keys and encrypted Amazon S3 storage, DynamoDB access is tenant-scoped, and AWS Secrets Manager holds Jira and GitLab credentials. At the workload level on Amazon EKS, Kubernetes namespaces, NetworkPolicy, and ResourceQuota separate tenants, and containers run as non-root with dropped Linux capabilities. At the network edge, AWS WAF managed rules and request-rate controls protect the platform. Traffic is delivered over HTTPS through Amazon CloudFront and the Application Load Balancer, and responses carry restrictive security headers. Across all of these, OpenTelemetry traces capture agent activity while Amazon CloudWatch logs and Container Insights provide application and execution observability.

Outcomes

TCS runs the Agentic Test Orchestrator in production for its BFSI clients. To quantify the effort savings, TCS measured the solution against its existing process on a representative BFSI workload of about 100 test scenarios. In the manual process, each scenario takes roughly two hours of engineer effort to design, script, and execute, or about 200 hours for the full set. With the Agentic Test Orchestrator, the average effort was approximately 14 minutes per scenario, bringing the same workload to about 24 hours. This represents an 88 percent reduction in effort in this comparison. (These figures come from a controlled internal benchmark that TCS ran across approximately 100 scenarios. Actual results vary by workload, application complexity, and team.)

For TCS, the effort number matters less than the operating model behind it. Requirement ingestion, generation, execution, correction, and reporting now form one traceable workflow instead of a chain of disconnected handoffs. Quality engineers review and govern AI-produced artifacts rather than hand-building them, which returns their time to the risk analysis, exploratory testing, and release decisions that the manual process crowded out. That is the outcome TCS set out to reach.

Conclusion

The problem TCS set out to solve was not simply slow test execution. It was the fragmented, manual path from business requirements to trustworthy test evidence. This is the part of a BFSI release that consumes engineering time without adding assurance. In this post, we showed how TCS built the Agentic Test Orchestrator on AWS to close that gap by combining contextual retrieval, specialized agents, workload-aware execution, isolated per-tenant services, and observable infrastructure.

The result is a reusable pattern for applying generative AI to software quality engineering without reducing testing to a single prompt. Amazon Bedrock AgentCore hosts and scales the agents, Amazon Bedrock and Amazon Bedrock Knowledge Bases supply the AI capabilities and project context, Amazon EKS hosts the per-tenant application layer, Amazon ECS provides fit-for-purpose execution, and AWS security, storage, integration, and observability services support the full lifecycle. For TCS’s BFSI teams, the payoff is a faster, more consistent testing process that returns engineers’ time to risk, quality, and release decisions.

To learn more and deploy the solution in your own environment, visit the TCS Agentic Test Orchestrator in AWS Marketplace. If you would like help applying this pattern to your own testing workflows, contact the AWS team. Have questions or a similar use case? Share them in the comments.