Migrating a High-Throughput Service to AWS Graviton

Moving a service from x86 to Arm can look like a routine infrastructure change: rebuild the image, change the instance family, deploy, and collect the savings.

That framing is too narrow. A processor migration changes the performance envelope of the whole service—application code, runtimes, sidecars, network calls, logging, metrics, and scaling policy. The useful question is not “Is this CPU faster?” It is:

How much successful work can this system deliver, within its latency and reliability limits, for each dollar spent?

I used that question to guide an AWS Graviton migration for a latency-sensitive, containerized request-serving workload. In controlled tests, the migrated configuration increased sustainable throughput from 250 to 500 transactions per second (TPS) per host while keeping tail latency broadly flat. Combined with a lower effective host price, that translated to an estimated 75% reduction in compute cost per request.

Those numbers belong to one workload, not to Graviton in general. The more transferable result was the method: define success from the customer’s perspective, test the entire stack to its failure boundary, and make the decision using unit economics rather than instance price.

Start with a service contract, not an instance type

AWS positions Graviton-based EC2 instances as offering up to 40% better price-performance than comparable x86-based instances. That is a useful hypothesis, but it is not a business case for your service. Your application, dependencies, traffic shape, and operational overhead determine the result.

Before choosing candidate instances, write down what must remain true after the migration. For a synchronous request path, the contract might look like this:

Dimension	Example acceptance criterion
Correctness	Identical functional behavior and no architecture-specific failures
Availability	No regression in success rate or error rate
Latency	p50, p90, and p99 stay within the existing service-level objectives
Capacity	Sustainable throughput per host is equal or higher
Saturation	CPU, memory, network, and disk retain safe operating headroom
Operations	Logging, metrics, tracing, and deployment behavior remain healthy
Economics	Cost per successful request decreases

This prevents a common benchmarking mistake: declaring success because a CPU microbenchmark improved while the customer-facing system became less reliable.

AWS’s performance-testing guidance recommends an outside-in view—throughput, latency, and error rate—correlated with inside-out resource metrics such as CPU, memory, disk, and network. Both views matter. The first tells you whether customers are safe; the second helps explain why the result changed.

Inventory the architecture boundary

Most modern application code is relatively portable. The surprises tend to live around it.

Before building an Arm64 artifact, inventory every component that crosses the architecture boundary:

base container images;
native libraries and language extensions;
agents for logging, metrics, tracing, and security;
sidecars and service-mesh components;
runtime and JDK versions;
build tools and CI runners;
downloaded binaries or vendor packages;
assumptions about CPU count, concurrency, or instruction sets.

Container images are architecture-specific even when their Dockerfiles look identical. Building a multi-architecture image under one manifest is usually the cleanest long-term design: the deployment can pull the appropriate linux/amd64 or linux/arm64 image without maintaining different release semantics.

Compatibility should also be separated from performance. First make the Arm64 build pass the same unit, integration, and functional suites as x86. If a runtime or dependency must be upgraded, test that upgrade on x86 before changing the processor architecture. Changing one variable at a time makes failures diagnosable.

Choose candidates from the workload’s constraint

“Same vCPU count” is a starting point, not a guarantee of equivalence. Graviton maps each vCPU to a physical core and does not use simultaneous multithreading, so a numerically equal comparison can still represent different execution resources.

Start with the workload’s bounding resource:

Compute-bound: evaluate compute-optimized families, but verify that memory and network remain sufficient.
Memory-bound: begin with general-purpose or memory-optimized families.
Network-bound: compare network bandwidth and packet performance, not just CPU and RAM.
Mixed or uncertain: test more than one family. The benchmark should decide.

In my case, the smallest host in the existing fleet constrained how containers were sized. The first baseline tests showed that memory pressure in an observability sidecar—not the main application—was the earliest failure. That evidence changed the candidate choice: a memory-richer general-purpose Graviton shape was more appropriate than the compute-optimized option that initially appeared closest on paper.

This is an important lesson: select the new fleet for the bottleneck you actually have, not the one implied by the application’s name.

Benchmark the service to saturation

A fixed-load comparison answers, “Can both configurations handle today’s traffic?” A saturation test answers the more valuable question, “Where does each configuration stop being safe?”

I used a single-host setup for each candidate and held the rest of the environment as constant as practical:

the same application revision and configuration;
equivalent container CPU and memory reservations;
the same request dataset and traffic distribution;
the same downstream behavior;
the same warm-up period and test duration;
the same telemetry and success thresholds.

Then I increased load in steps. Each run was classified by the service contract rather than by whether the process stayed alive.

A run was unsuccessful if any hard boundary was crossed—for example:

tail latency exceeded the target;
errors or timeouts increased materially;
a container restarted or ran out of memory;
CPU stayed near saturation without recovery headroom;
memory grew continuously;
the logging or metrics pipeline fell behind.

The result for each configuration was a pair of points:

the highest successful load, and
the lowest unsuccessful load.

That pair is more honest than a single peak-throughput number. It shows both the validated operating range and the next observed failure mode.

What the tests revealed

On the legacy x86 configuration, 250 TPS per host was sustainable. At the next load step, the logging sidecar’s memory kept growing until the container failed. Host CPU was not yet the limiting resource.

To isolate the application, I repeated the experiment without that sidecar. The service handled substantially more traffic before application CPU and latency became the constraint. This established two separate facts:

the full production stack was limited by logging memory; and
after removing that limit, the application still had a higher CPU-bound ceiling.

On the selected Graviton configuration, the complete stack sustained 500 TPS per host with p99 latency close to the x86 baseline. At the next step, CPU approached saturation and tail latency rose sharply, so that run was rejected.

The migration therefore did more than move the same bottleneck. It shifted the first failure from sidecar memory to application CPU and roughly doubled the usable capacity of a host.

It also exposed follow-up optimization opportunities in metrics emission, request concurrency, and logging. A processor migration will not repair inefficient code or an overloaded sidecar, but it can reveal those limits more clearly by moving the performance boundary.

Use cost per successful request

Comparing hourly instance prices ignores fleet size. Comparing throughput ignores price. The decision metric needs both:

requests per host-hour = sustainable requests/second × 3,600

cost per million requests =
  hourly host cost ÷ requests per host-hour × 1,000,000

The same calculation works for jobs, events, images, or gigabytes processed. Choose a unit that represents useful business output, and count only work completed within the service contract.

For a simplified version of the real result, normalize the old configuration to an hourly cost of 1.0 and capacity of 1.0:

Configuration	Relative hourly cost	Relative sustainable throughput	Relative cost per request
Legacy x86	1.00	1.00	1.00
Graviton	0.50	2.00	0.25

The new host was roughly half the effective price and delivered roughly twice the sustainable throughput. The combined effect was a cost per request near one quarter of the original—about a 75% improvement.

This multiplicative effect is why unit economics matter. Even a modest instance discount can become a much larger fleet-level saving when each host also performs more useful work. The reverse is also true: a cheaper host that requires many more replicas may be a regression.

Roll out as a reversible change

Passing a synthetic load test is necessary, not sufficient. Synthetic traffic can miss endpoint mix, payload diversity, cache behavior, and dependency variance found in production.

Treat the rollout as a controlled experiment:

Build multi-architecture artifacts. Keep the application version identical across architectures.
Create parallel capacity. Use a separate target group, service, node group, or blue/green environment.
Start with a small traffic percentage. Compare latency, errors, throughput, saturation, and business-level success metrics by architecture.
Hold and observe. Include peak periods and enough time to detect memory growth or background-job effects.
Increase traffic gradually. Continue only while the predeclared criteria pass.
Keep rollback boring. Preserve the x86 path until the new fleet has proved stable under representative traffic.
Revisit scaling thresholds. A higher-capacity host may make the old CPU target or minimum fleet size unnecessarily conservative.

AWS recommends canary or blue/green deployment for the transition, and public Graviton case studies show the same pattern: run both architectures in parallel, measure them with the same application-level metrics, and remove the old capacity only after confidence is earned.

Mistakes that weaken the result

Benchmarking at only one load level

Two configurations can look identical at 20% utilization and diverge completely near saturation. Find the knee of the curve, not just a comfortable point on it.

Watching averages

Average latency hides the experience of the slowest requests. Track at least p50, p90 or p95, and p99 alongside error and timeout rates.

Measuring only the main container

Sidecars consume the same host resources and can fail first. Include log volume, telemetry backpressure, agent health, container restarts, and memory growth.

Changing too many variables

A new architecture, runtime, application version, and traffic model in one experiment produce an impressive chart and a weak explanation. Stage changes so each comparison has a clear interpretation.

Using list price as the conclusion

Model your real purchase options, discounts, utilization, and scaling behavior. Then verify the estimate against the bill after rollout.

Generalizing one workload’s win

My service doubled per-host throughput; yours may not. Native dependencies, single-threaded code, memory bandwidth, vector instructions, and network behavior can all change the outcome. Publish the test conditions with the number.

A practical migration checklist

Before migration:

Define latency, reliability, capacity, and cost success criteria.
Inventory images, native dependencies, agents, and sidecars.
Build and functionally validate Arm64 or multi-architecture artifacts.
Identify the current bottleneck from production evidence.
Select multiple plausible Graviton shapes.

During benchmarking:

Keep application and environment variables controlled.
Use a representative request mix and adequate warm-up duration.
Increase load until each candidate crosses a hard boundary.
Record customer, application, container, and host metrics together.
Calculate cost per successful unit of work.

During rollout:

Deploy parallel, reversible capacity.
Route a small percentage of traffic first.
Compare architecture-specific dashboards and business metrics.
Expand gradually through representative peaks.
Validate realized cost after the fleet stabilizes.

The real migration is a measurement system

The most valuable artifact from this project was not an Arm64 image or a new instance configuration. It was a repeatable way to reason about infrastructure changes.

Define what customers need. Measure the whole stack. Push each option to a clearly stated boundary. Translate the result into cost per useful outcome. Roll it out gradually and keep the old path available until production evidence agrees with the lab.

Do that, and a Graviton migration becomes more than a hardware refresh. It becomes a disciplined exercise in making a service faster, cheaper, and better understood.