1. Introduction

When developing an application or service, one of the common questions we face is how many concurrent requests it can handle and how the services it depends on will respond. Even if we verify that all functionalities work correctly, this does not guarantee that the system will maintain its performance as demand increases.

This is why software testing goes beyond verifying functional requirements. It also allows us to evaluate aspects such as performance, stability, and responsiveness. In particular, load testing is not simply about pushing a system until it fails, but rather about understanding how it behaves under different conditions and identifying when it stops meeting the expected requirements.

In the previous article, we built a service to identify duplicate products using Amazon Bedrock, S3 Vectors, and DynamoDB, including an AWS Lambda function to perform real-time searches. During our initial tests, the service responded within a few seconds, but one question remained unanswered: how would it behave if many users accessed it simultaneously?

To answer this question, in this second part, we will expose the service through Amazon API Gateway and evaluate its behavior under load. Before running the tests, however, we need to define what we want to measure and which criteria we will use to evaluate the results. To do this, we will introduce some fundamental concepts of Site Reliability Engineering (SRE). We will then automate load testing with Grafana k6 on AWS and, finally, generate a report in which generative AI will help us interpret the collected metrics.

2. Understanding Site Reliability Engineering (SRE)

2.1. What Is SRE?

Before evaluating how our API behaves under load, we need to establish what acceptable performance and reliability actually mean. A service may continue responding to requests, but that does not necessarily mean it is meeting the expected performance requirements.

This is where Site Reliability Engineering (SRE) comes into play. Originally developed at Google, SRE is a discipline that applies software engineering principles to operating and maintaining reliable systems. It focuses on defining measurable objectives, selecting indicators that reflect service behavior, and using automation to reduce repetitive operational tasks.
For our use case, these concepts provide a foundation for moving beyond simply checking whether the API responds. Instead, we can evaluate specific indicators such as response times, error rates, and performance under different load conditions.

2.2. Seven Principles of SRE

SRE is built on a set of principles that guide how reliable services are designed, operated, and maintained. These range from defining reliability objectives and managing risk to automation, monitoring, and keeping systems as simple as possible.
The following infographic summarizes seven of these principles, based on Google's Site Reliability Engineering book.

Illustration by the author. Based on Google's Site Reliability Engineering book.

2.3. Capacity Planning and Performance

Service reliability depends not only on availability but also on the ability to handle demand while maintaining acceptable performance. In this context, SRE includes several practices aimed at anticipating problems, optimizing resources, and reducing the impact of changes in production.

Among these practices, we can highlight:

• Capacity Planning: ensuring that a system has sufficient resources and redundancy to handle expected demand without compromising availability objectives. To support this, Google recommends regular load testing to assess the actual capacity of a service.
• Efficiency and Performance: understanding how performance changes as load increases, identifying potential degradation, and finding opportunities to optimize resource utilization.
• Change Management: reducing the risks associated with changes in production. According to Google's documented experience, approximately 70% of service outages are caused by changes to live systems, highlighting the importance of detecting problems early. These practices are directly related to our use case, as we aim to evaluate our API's performance under different load conditions.

2.4. SLI, SLO, and SLA

Evaluating a service's capacity and performance requires more than simply running tests and collecting metrics. We also need to define what we want to measure and which service levels we consider acceptable.

In this context, SRE introduces three related concepts: SLI (Service Level Indicator), SLO (Service Level Objective), and SLA (Service Level Agreement). Together, they connect technical measurements with service objectives and commitments.

The following infographic summarizes what each concept represents, the question it helps answer, and a practical example of how it can be applied.

Illustration by the author. Based on Google's SRE framework.

3. Grafana k6: A Tool for Performance Testing

Grafana k6 is an open-source tool developed in Go and maintained by Grafana Labs, designed to perform load and performance testing on applications, APIs, and microservices. Test scenarios are defined using JavaScript, making it easier to automate tests and integrate them into development workflows.

Although we often associate these tools with stress testing, k6 allows us to evaluate how a system behaves under different load conditions, without necessarily pushing it to the point of failure.

Illustration by the author. Based on Grafana k6 documentation.

3.1. Types of Load Tests

Depending on what we want to evaluate, we can define different test scenarios:

• Smoke: verify that the system responds correctly under minimal load.
• Average-load: evaluate system behavior under normal operating conditions.
• Stress: assess how the system responds under high load.
• Soak: identify potential performance degradation when load is sustained over extended periods.
• Spike: evaluate how the system responds to sudden increases in traffic.
• Breakpoint: progressively increase the load to identify the point at which the system stops meeting the expected criteria. These scenarios allow us to analyze different aspects of system performance. We should also consider that an application often depends on multiple components and services, meaning that performance degradation can originate from different parts of its architecture.

3.2. Main Components of k6

To execute these tests, k6 provides several components that allow us to configure load, validate responses, collect metrics, and integrate results with other tools.

Illustration by the author. Based on Grafana k6 documentation.

We can group them into four main components:

• ⚙️ Options: define how a test will be executed. They allow us to configure scenarios, load stages, virtual users (VUs), duration, iterations, and acceptance thresholds. These options enable us to reproduce the different types of tests described earlier.
• ✅ Checks: validate conditions on received responses, such as HTTP status codes or response content. A failed check is recorded in the metrics but does not automatically stop the test execution.
• 📊 Metrics: collect information about system behavior, including response times, request counts, and error rates. k6 provides built-in metrics and supports custom metrics.
• 🔗 Integrations: allow tests to be incorporated into CI/CD pipelines and results to be exported to storage, monitoring, and visualization tools. In our case, we will use CSV and JSON to store the results and subsequently process the metrics included in the final report.

4. Architecture

Our load testing solution builds on the game-catalog-resolve-product Lambda function developed in the previous article, which uses Amazon Bedrock, S3 Vectors, and DynamoDB to identify duplicate products.

In this second part, we expose the function through Amazon API Gateway (HTTP API) with IAM authorization, requiring k6 requests to be signed using AWS credentials.

The testing process is automated using AWS Step Functions, which orchestrates k6 execution on Amazon ECS Fargate and subsequent report generation through an AWS Lambda function. Test scripts and results are stored in Amazon S3, while Amazon ECR stores the container images.

The entire infrastructure is defined using Terraform, and the code is available on GitHub, making the solution reproducible.

Architecture diagram by the author.

5. Use Case: Automated Load Testing

In this use case, we will run different load testing scenarios against our API. To do this, we use Grafana k6 running inside a Docker container to send requests and collect performance metrics. Once the tests are complete, a Python-based process calculates the results and generates a PDF report that includes an interpretation provided by Amazon Bedrock.

Illustration by the author.

5.1. The Repository

The project is available on GitHub: game-catalog-k6.

game-catalog-k6/
├── Dockerfile, entrypoint.sh
├── search.js, sigv4.js
├── scenarios/
├── queries.json
├── report/
│   └── config/
├── terraform/
└── build-and-push.sh

The goal is to make this solution reusable for testing other APIs without modifying the underlying infrastructure or the report generation process. To do this, we only need to configure three elements:

• 〰️ The API we want to test
• 〰️ The requests we want to send
• 〰️ The load testing scenarios

5.2. Configuring Tests for Another API

1. Identify the API: api_id

In our case, we use Amazon API Gateway (HTTP API) with IAM authorization. The service URL follows this structure:

https://<api-id>.execute-api.us-east-1.amazonaws.com

The API identifier can be found in the AWS Management Console under API Gateway → Your API → API details.
To configure the project, we specify it in terraform/terraform.tfvars:

api_id = "<api-id>"

Terraform uses this identifier to construct the API URL and configure the necessary permissions for the k6 container to invoke the API using IAM-signed requests.

2. Define the Requests: queries.json

Once we have identified the API, we need to define which requests we want to send.
Our API accepts text searches through POST /search. To define the requests executed during testing, we use the queries.json file, which contains different video game searches in English and Spanish.

[
  {"query": "fifa 14 ps3 digital code", "lang": "en"},
  {"query": "fifa 14 ps3 codigo digital", "lang": "es"},
  {"query": "gta vice city ps2", "lang": "en"},
  {"query": "el mario kart de la wii", "lang": "es"}
]

If the API expects a different request structure, we will also need to adapt the request body in search.js.

3. Configure the Load: scenarios

Once we have defined what to request, we need to establish how the load will be applied. For this, we use JSON files in the scenarios/ directory, each with its own configuration.

Each JSON file defines a different load testing scenario:

• smoke.json — Smoke Test: 1 virtual user for 1 minute to verify basic functionality.
• load.json — Load Test: 5 virtual users for 5 minutes to evaluate system behavior under steady load.
• stress.json — Stress Test: gradually increases the load from 5 to 40 virtual users over 6 minutes to identify performance degradation.
• spike.json — Spike Test: increases the load from 0 to 50 virtual users in 30 seconds to evaluate how the system responds to sudden traffic spikes. Each scenario can be customized by adjusting the number of virtual users, test duration, load pattern, and performance thresholds.

Each file allows us to configure virtual users, test duration, load patterns, and performance thresholds.
For example, smoke.json contains:

{
  "name": "smoke",
  "description": "1 user for 1 minute: checks that the API works before adding load.",
  "queries_file": "queries.json",
  "endpoint": "/search",
  "think_time_seconds": 1,
  "executor": {
    "executor": "constant-vus",
    "vus": 1,
    "duration": "1m"
  },
  "thresholds": {
    "http_req_failed": ["rate<0.01"],
    "http_req_duration": ["p(95)<5000"]
  }
}

The main parameters are:

• endpoint: the API path to which requests will be sent.
• think_time_seconds: the pause between requests made by each virtual user.
• executor: defines the load pattern, number of virtual users, and test duration.
• thresholds: allows us to define acceptance criteria for the metrics. In this example, an error rate below 1% and a p95 response time below 5 seconds. These thresholds allow us to translate our performance objectives into measurable acceptance criteria during testing, following the SLI and SLO concepts introduced earlier. We can modify existing scenarios or create new JSON files to evaluate different load conditions while reusing the same k6 code.

5.3. Deploying the Infrastructure with Terraform

Once the configuration files are ready, we can deploy the solution using AWS CloudShell, with the tools needed to run Terraform and build Docker images.

1. Install Terraform (First Time Only)

AWS CloudShell does not include Terraform by default. We can download it to our home directory, which persists across sessions:

curl -sLo /tmp/terraform.zip https://releases.hashicorp.com/terraform/1.9.8/terraform_1.9.8_linux_amd64.zip
mkdir -p ~/bin && unzip -o /tmp/terraform.zip -d ~/bin
echo 'export PATH="HOME/bin:$PATH"' >> ~/.bashrc && source ~/.bashrc
terraform version

2. Create the Infrastructure

git clone https://github.com/RominaElenaMendezEscobar/game-catalog-k6.git
cd game-catalog-k6/terraform
echo 'api_id = "<api-id>"' > terraform.tfvars
terraform init
terraform apply \
  -target=aws_ecr_repository.k6 \
  -target=aws_ecr_repository.report
../build-and-push.sh

terraform apply

The deployment is performed in three stages: first, we create the Amazon ECR repositories; next, we build and push the container images; and finally, we deploy the remaining resources.

Terraform provisions the required infrastructure, including Amazon ECS Fargate, AWS Step Functions, AWS Lambda, Amazon S3, and IAM roles and permissions. It also uploads the test scenarios and queries that k6 will use to Amazon S3.

5.4. Running a Load Test

With the infrastructure deployed, we can start a test execution using AWS Step Functions, specifying the scenario we want to run.
For example, to execute load.json:

aws stepfunctions start-execution \
  --state-machine-arn "$(terraform output -raw state_machine_arn)" \
  --input '{"scenario": "load", "label": "baseline"}'

Screenshot by the author. AWS Management Console.

The state machine orchestrates three steps:

• PrepareRun: takes the scenario and label provided in the input and generates a unique identifier based on the date and time (run_id). This identifier is used in the following two steps to ensure that the results and report are associated with the same test execution.
• RunLoadTest: starts the k6 container on Amazon ECS Fargate, executes the selected scenario, and stores the results in Amazon S3.
• GenerateReport: processes the results using an AWS Lambda function, calculates the metrics, and generates a PDF report with an interpretation provided by Amazon Bedrock. Each execution is stored in a separate folder, following a structure such as runs/<scenario>/<date>/, along with the configuration used for the test.

Screenshot by the author. Amazon S3 bucket showing the files and results generated by the automated load testing pipeline.

Generated by k6 (RunLoadTest):

• k6-summary.json: a summary of the test, including aggregated metric values (averages, percentiles, error rates) and threshold evaluation results.
• k6-results.csv: detailed data for each request, including when it was sent, how long it took, and whether it failed. This data is used to generate time-series charts.
• run-info.json: execution metadata, including the scenario, label, timestamps, k6 version, and container image.
• scripts/: an exact copy of the scripts and scenarios used, allowing us to reproduce the test configuration even if the original files are modified later. #### Generated by the Report Lambda (GenerateReport):
• metrics.json: metrics calculated by the code using the files generated during the test.
• ai-analysis.json: the interpretation generated by Amazon Bedrock, along with the results of the numerical validation.
• report.pdf: the final report, combining all the information above. To run a different test scenario, we only need to change the scenario value in the Step Functions execution input.

5.5. Report Generation

Once the test is complete, the pipeline processes the k6 results and generates a PDF report that combines the calculated metrics with an interpretation provided by Amazon Bedrock.
For this stage, we use an AWS Lambda function deployed as a container image, as we need several Python libraries: pandas for data processing, matplotlib for generating charts, and fpdf2 for creating the PDF document.

Unlike traditional ZIP-based deployments, which have a 250 MB uncompressed size limit, AWS Lambda supports container images of up to 10 GB. This makes it easier to package the code and its dependencies into a reproducible environment while maintaining a serverless execution model.

The following infographic summarizes the main benefits of this approach.

Illustration by the author. Based on AWS Lambda documentation.

The container image is built using a Dockerfile and pushed to Amazon ECR. Terraform then configures the AWS Lambda function to use this image, as illustrated in the following workflow.

Illustration by the author. Based on AWS documentation.

We can view an example of the generated PDF report at the following link:
github.com report.pdf

Report generated by the author using Python, Grafana k6, and Amazon Bedrock.

The report consists of several pages organized into the following sections:

• Test Context: the executed scenario (load, duration, and thresholds) and the test environment (k6 version, container image, and location of the results).
• Results and Metrics: requests, throughput, error rates, p50, p95, and p99 latencies, threshold compliance, and a breakdown of each phase of a request.
• Performance Charts: response times over time, latency distribution, percentiles, request time breakdown, throughput, errors, and virtual users.
• AI-Powered Analysis: an interpretation of the metrics generated by Amazon Bedrock, along with validation of the numerical values referenced in the analysis.
• Metrics Glossary: definitions of each metric, how it is calculated, an example, and why it matters.

It is important to emphasize that all metrics are calculated using Python code based on the k6 results. Amazon Bedrock is used only to interpret these values and generate the narrative analysis, with an additional validation step to verify the numerical values mentioned in the generated text.

6. Conclusions

In this article, we started with a question: how does an API behave as concurrency increases? To answer it, we implemented an automated load testing pipeline that allows us to evaluate API performance under different load scenarios and analyze the results.

This exercise allowed us to put some Site Reliability Engineering (SRE) concepts into practice: defining which aspects of the service we want to measure (SLIs), establishing what service levels we consider acceptable (SLOs), and using Grafana k6 to translate those objectives into measurable thresholds during testing.

Beyond running load tests, we also need to interpret a considerable amount of performance data. To address this, we incorporated generative AI to facilitate the analysis and generate a report that summarizes the key findings without requiring an in-depth understanding of every metric. It is important to emphasize that the metrics are calculated by code, while AI is used only to interpret them, maintaining a clear separation between the collected data and the generated conclusions.

Another important aspect is the solution's reusability. By modifying the API, queries, and load testing scenarios, we can use the same pipeline to evaluate other services. In addition, preserving the configuration and results of each execution provides traceability and facilitates future comparisons.

However, this MVP has some limitations and opportunities for improvement:

• Geographic Coverage: tests are executed from a single AWS Region, so they do not necessarily reflect the experience of geographically distributed users.
• Test Duration: short test scenarios may produce limited sample sizes, which can affect the stability of percentiles such as p95 and p99.
• AI Interpretation: the analysis depends on the model and prompt used. It would be interesting to evaluate different models and refine the instructions based on the type of insights we want to obtain.
• CI/CD Integration and Observability: the pipeline is not yet integrated into a CI/CD process. As a future improvement, we could automate testing across application versions, compare results to detect performance regressions, and incorporate Amazon CloudWatch metrics for AWS Lambda and Amazon Bedrock.

Ultimately, the goal is not simply to determine how many requests an API can handle, but to establish a reproducible process that allows us to measure its behavior, identify potential performance degradation, and evaluate how its performance evolves as we introduce changes.

References

• Beyer, B. et al. (2016). Site Reliability Engineering, capítulo 4: Service Level Objectives. https://sre.google/sre-book/service-level-objectives/
• Grafana Labs. (s. f.). Grafana k6. https://grafana.com/oss/k6/
• Grafana Labs. (s. f.). Load test types. Documentación de Grafana k6. https://grafana.com/docs/k6/latest/testing-guides/test-types/
• Amazon Web Services. (s. f.). Lambda quotas. Documentación de AWS Lambda. https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
• Méndez Escobar, R. E. (2026). Identical or Similar? AI-Powered Product Matching with Amazon Bedrock and S3 Vectors. DEV Community. https://dev.to/aws-builders/identical-or-similar-ai-powered-product-matching-with-amazon-bedrock-and-s3-vectors-3pm