How AI Is Changing Performance Testing: From Script Generation to Faster Insights

How AI Is Changing Performance Testing: From Script Generation to Faster Insights

How AI Is Changing Performance Testing: From Script Generation to Faster Insights

There’s a slightly unpopular opinion in my mind. The greatest hurdles with performance testing aren’t around load. They’re around time.

There’s hardly a team that doesn’t understand the need to load test prior to a large release. The issue is finding the four or five day window to write the scripts, patch them after last week’s API change, and then wrestle with a 40-page report to understand why checkout slowed down at 2000 users. Finding time to interpret the test results is almost as large a hurdle as writing the scripts.

AI performance testing is starting to fill this gap. It’s not about making your servers faster. It’s about taking the slow, uninteresting parts of the job and letting computers do it.

There are real-world consequences of not performing performance tests. On November 15, 2022, Ticketmaster opened a presale for Taylor Swift’s Eras tour. There was anticipated high demand. They said they received over 3.5 billion system requests in the first hour. Their site was overwhelmed, and the public sale was later cancelled. Knowing that demand would be “high” and testing to see if your site can handle 4x your record are two completely different things.

Short Answer

AI can do parts of load testing. It writes load tests from your API definitions and repairs load tests when your API changes. It runs load tests in your CI/CD pipeline and points to the likely cause when response times increase. People decide what gets fixed first.

Key Takeaways

– AI helps with time consuming parts of load testing first.

– Self healing scripts means performance testing no longer needs to be done once a quarter.

– Thousands of errors are often grouped together and a likely cause for a slow response can be determined by AI in a short time.

– Humans make the final decision to ship.

– A good place to start is with the most profitable API flow.

Why performance testing keeps getting skipped

A performance engineer opens JMeter or k6 and builds a script for a login-then-checkout flow. Logging in to the site returns a session token. All following requests need to include the session token, so the test engineer needs to write “correlation” logic to capture the session token and send it along for the subsequent requests. Thinking about thousands of concurrent users, the test engineer creates 500 users, each with a different cart, to ensure the requests are not done by the same user and to avoid caching.

It’s tedious work and breaks easily.

A developer changes the name of a response field from access_token to token in a release on Tuesday. The performance team is not notified. On Wednesday’s test runs, 50% of the requests come back with a 401 Unauthorized response, and you have wasted an hour measuring how quickly your server says no. Then a developer loses a morning tracking down the issue. Repeat this 30 times within a 2 week sprint, and you understand the need for load testing. Load testing becomes necessary to counteract the number of unexpected issues caused by a deployment.

Step 1: Use the generated script to load test

Now, consider the same tool for load testing. Instead of spending a week just getting JMeter to integrate with your app the way you want, you can give the AI tool a Postman collection or Swagger file, or even just a few cURL commands. The AI reads the auth flow and request bodies, generates a JMeter or k6 script, and creates a token and test data to use for the tests.

The load testing time for a complex flow with 6 requests can be reduced from multiple days to just a few minutes.

Of course, there are limitations. The AI is only as good as the instructions given, so editing and reviewing the script is necessary. A good way to get a feel for the number of requests and the user mix, and if the payment request is hitting a production gateway, is to simply review the script. The AI may have the payment assertion checking if the response is a 200, which is useless, so adjusting the assertions to check something useful would be necessary. I’d rather spend 20 minutes reviewing a draft than 2 days writing a script from scratch.

We built PeakoQ to solve this problem. It takes the API collections in your code and generates both JMeter and k6 scripts. This way, your engineers can work with pre-written test scripts, instead of starting from scratch.

Step 2: Scripts that fix themselves

You likely remember the access_token rename. Our AI looks for inconsistencies like that, and will suggest fixing the script if it’s the reason the test case is failing.

That improvements the team’s velocity, because tests that normally died after every release, stay running.

A line is crossed though. A healer should fix the plumbing, never the verdict. If the checkout process used to take 2 seconds and now takes 5, the tool should not “heal” the assertion to 2 seconds, to make the test case “green”. That’s a performance bug and a dev should see that.

Step 3: continuous performance tests

We believe the cost of creating and maintaining scripts will go down, allowing you to integrate load tests into your CI/CD pipeline.

A setup that works well for a lot of teams looks like this: a small smoke load test runs on every merge to main, perhaps 50 virtual users for 5 minutes. A full load test runs nightly. Stress and soak tests (soak means a steady load for hours to catch memory leaks) are run before a major release. Each is compared to a baseline.

That baseline is the value. For example, if the p95 response time for /search was 420 ms last week and it’s 610 ms after Thursday’s commit, you know which commit, you know which endpoint. Nobody needs to wait for a customer to complain.

Step 4: faster insights, the most underrated part

Generating the script gets the headlines. Reading the results is where AI does the heavy lifting. Interpreting results used to mean one senior engineer, a spreadsheet and a long afternoon. Now AI can group similar errors, line up the moment things went wrong, and explain it in layman’s terms, for example, “Errors start at 1,800 virtual users. 92% are timeouts on /payments. The database connection pool hits its limit of 100 in the same minute.”

This isn’t a real world example, but it gives you an idea of what kind of results AI is giving engineers. It reduces complex, lengthy reports to quick, easy to understand summaries.

There are both opportunities and potential pitfalls associated with using generative AI in quality engineering. The World Quality Report 2025-26 found that 89% of survey respondents have either started or launched initiatives to use generative AI in their quality engineering. Of those, 37% said they have deployed generative AI in quality engineering. Only 15% have scaled the use of generative AI enterprise-wide, and the average productivity gain was 19%. My take on the productivity gap is plenty of teams have integrated AI into a broken testing process and are puzzled as to why it didn’t improve testing significantly.

AI has its limitations in performance testing

There are aspects of performance testing that can’t be fully delegated to AI. For instance, someone has to tell AI what real world load and response time look like. AI may not be able to determine if something is high or low traffic based on website data. Does it mean the same thing as high or low traffic in the real world? Someone has to set those goals. For performance testing, people can develop a hunch about goal setting. For instance, a hunch may develop that 95% of the time the page should load in less than 800ms at 5,000 users. Someone needs to decide if a page load time on a reporting page should be on the same level of importance as a page load time on the checkout page. A person also has to decide if it should be tested as critical, or if it is more important to test availability. A person has to make the call to ship the app or hold the app.

Ticketmaster is a good example again. AI would also have to ask, “what if this is much worse than we think?” That question is also a human job.

How Quarks approaches AI performance testing

Here’s how the work splits between AI and people when we run performance testing with PeakoQ as part of our quality engineering services.

Stage The old way With PeakoQ Who decides
Scripting Engineer hand-builds JMeter or k6 scripts, often over several days Scripts generated from Postman, Swagger or cURL Engineer reviews every draft
Maintenance Scripts break quietly after API changes Auto-healing diagnoses the failure and repairs the script Engineer approves fixes; assertions stay locked
Running Once before launch, if there’s time Load, stress and soak tests inside CI/CD Team sets the gates and targets
Analysis Manual digging through reports and graphs Real-time analytics with grouped errors People decide what to fix and when

Quarks reports 95% faster scripting, a first test in under 24 hours and 40-60% fewer production incidents for teams using PeakoQ.

A real example: from 2 seconds to about 100 milliseconds

One client, a publicly listed lender with over 120 branches, had a loan processing platform that was slow at the worst possible times. Disbursals took too long and payment tracking and customer data update were not done in real time.

Our team used Datadog to determine where time was spent across the various servers and the database, added proper performance metrics and reporting, and enhanced code quality with SonarQube. Improvements moved response and log times from about 2 seconds to 100-105 milliseconds. The process of loan approval was totally moved online, and the client added more than 35,000 customers that year.

While that project also did not use AI-generated scripts, we find that the area of focus where the greatest time reduction was realized also happens to be the area where AI makes the greatest impact.

What I would do in Week 1

To any QA lead considering this, I would suggest you don’t start with your full platform. Instead, focus on the flow that makes money, likely the flows covering login and checkout or payment. Export a Postman collection for one of these flows, generate a script for it, and have your best performance engineer read the script line by line. Reach an agreement on a performance goal before you run the script to test against, for example, a 95th percentile response time at or under 800ms at your expected peak. Be prepared to defend your goal choice. Once an agreement is reached, run the test and the script nightly for two weeks, and only after that block the release. A flaky gate gets switched off quickly. Write down the expected peak traffic from last year and test for 2x the expected traffic. Ticketmaster would say testing for 4x the expected traffic would be more prudent.

If you want a second opinion on how to get started, contact our experts.

AI Performance Testing FAQ

What is AI performance testing?

It is load testing where AI helps write the scripts. It edits the scripts when your API definition changes, and it summarizes and interprets the results for you to help track down why your system is running slowly. Your team is still in charge of setting the goals and choosing what to address. AI is still many years away from fully replacing a performance engineer. Think of it more like a performance testing assistant.

Can AI write JMeter or k6 scripts?

In the case of API scripts, I believe AI is already writing better JMeter or k6 scripts than most engineers. With a Postman collection or OpenAPI and a set of cURL commands, AI can create and fill a script with token and test data. Front end scripts with heavy browser usage are hard for it still. Generally speaking, you should always check scripts for accuracy before taking test result stats at face value.

Will AI take the jobs of performance engineers?

I don’t think so. It will remove the part of the job most don’t like. Building effective traffic models still takes experience. What changes is that one engineer can cover a larger part of the system.

How is this different from APM tools like Datadog?

They actually work great together. APM tools are great for analyzing production issues in real time. With AI powered performance testing you can create a load test before releasing code and see where your limitations are. They actually work great paired together. In an ideal world, you’d pair AI powered performance testing with Datadog to make a test that hits an endpoint that has high server load.

How long does it take to get started?

Faster than most expect. If you already have Swagger files or Postman collections, you can have a first script in a day. The hard part is agreeing on realistic targets, which is a discussion. Give yourself 2 weeks of nightly runs before taking the results seriously and using them to block a release.

Is it Dangerous to Let AI Edit Failing Test Scripts?

For plumbing, yes. Renaming fields and endpoints and expiring tokens are ok. For pass/fail rules, no. If a healer quietly loosens an assertion to allow a slow response to “pass”, you’ve hidden a real bug. Lock assertions and make all AI changes visible and easy to review.

Want to see PeakoQ, AI performance testing in action?

Bring a Postman collection or Swagger file and we’ll show you PeakoQ generating, running and explaining a load test on it. Book a Demo

About Quarks

Quarks is an AI-led digital engineering, data and enterprise solutions company with 1000+ engineers, 225+ clients and 400+ projects delivered. Our quality engineering practice builds its own accelerators, including AutonixQ for functional test automation and PeakoQ for AI performance testing. We’re ISO and CMMI certified. Learn more about Quarks.

Want to turn your ideas into brilliant applications?

Talk to our Experts
Quarks

Want to give wings to your career?

Apply Now

Stay up to date with insights from Quarks!

    How can I help you?