API load testing with Locust for ASP.NET Core

Load testing means recreating peak-day traffic with simulated users before the day arrives, and seeing where the API slows down or starts failing. Locust describes user behavior in Python code, so you can run a realistic flow such as signing in, reading and submitting against an API written with ASP.NET Core. The options in this guide were checked against the Locust 2.46 documentation; current Locust versions require Python 3.11 or later.

Why test before a known peak?

Some peaks can be seen coming: exam day, the opening of registration, a campaign. At Gozine2, before exam peaks of 45,000 to 70,000 concurrent users, I used Locust load tests to find the bottlenecks and then tuned SQL Server queries, caching paths and the busiest APIs. A problem found a week before an exam leaves time to fix it and test again; on exam day there is no such time.

Before writing any code, settle two things:

• Target load. Derive the number of concurrent users and each endpoint's share from real data: registration numbers or the logs of the previous peak.
• Pass criteria. For example, "at the target load, the 95th percentile of every endpoint stays under X milliseconds and the error rate under Y percent". Without criteria, any result can be read as good or bad.

The data volume in the test environment should also be close to production; a query that is fast on ten thousand rows may not be on ten million.

Writing a realistic locustfile

A locustfile is an ordinary Python module. Every class that inherits from HttpUser is a user type, and Locust creates one instance of it for each simulated user. The example below is a student who signs in, fetches questions, submits answers and now and then checks the results. The endpoints and numbers are examples:

import csv
import itertools
import os
import random

from locust import HttpUser, between, task
from locust.exception import StopUser

with open("test_users.csv", newline="", encoding="utf-8") as f:
    ACCOUNTS = itertools.cycle(list(csv.DictReader(f)))

EXAM_IDS = [1201, 1202, 1203]


class Student(HttpUser):
    wait_time = between(3, 12)

    def on_start(self):
        account = next(ACCOUNTS)
        with self.client.post("/api/auth/login", name="login", catch_response=True,
                              json={"userName": account["user"],
                                    "password": os.environ["LOADTEST_PASSWORD"]}) as resp:
            if resp.status_code != 200:
                resp.failure(f"login: HTTP {resp.status_code}")
                raise StopUser()
            token = resp.json()["accessToken"]
        self.client.headers["Authorization"] = f"Bearer {token}"
        self.exam_id = random.choice(EXAM_IDS)

    @task(6)
    def next_question(self):
        self.client.get(f"/api/exams/{self.exam_id}/questions/next",
                        name="/api/exams/[id]/questions/next")

    @task(3)
    def submit_answer(self):
        with self.client.post(f"/api/exams/{self.exam_id}/answers",
                              name="/api/exams/[id]/answers", catch_response=True,
                              json={"choice": random.randint(1, 4)}) as resp:
            if resp.status_code == 200 and resp.json().get("saved") is not True:
                resp.failure("200 but not saved")

    @task(1)
    def my_results(self):
        self.client.get("/api/me/results")

Task weights and wait time

Methods marked with @task are tasks. After each task, a user picks the next one at random, and the number in parentheses is its weight: here fetching a question is picked six times as often as viewing results and twice as often as submitting an answer.

between(3, 12) means each user pauses for 3 to 12 seconds after every task, like a student reading the question. For a fixed rate there are also constant_throughput and constant_pacing. These functions apply to tasks, not requests, and they only cap the rate: if responses slow down, they do not create extra users.

Signing in with on_start

Locust calls on_start once when each user starts; this is the place to sign in and get a token. Each user's client is an HttpSession, which inherits from requests.Session, so a header set on self.client.headers is sent with all of that user's later requests. Test accounts are read from a file so that not everyone signs in with the same account, and their password comes from an environment variable, not from the code. If sign-in fails, the sign-in request itself is recorded as a failure and StopUser stops that user, so the statistics do not fill up with 401 errors. Locust logs this with a traceback and does not start a replacement user, so the number of active users drops.

Grouping with name, and failure rules with catch_response

Locust aggregates statistics by request name, which defaults to the URL; if the path contains an ID, every ID gets its own row. The name argument groups all of those requests under one label such as /api/exams/[id]/answers.

By default any response with a status code below 400 counts as a success, and 4xx and 5xx as failures. But an API can return 200 without having done the work. With catch_response=True and a with block you make the call yourself: resp.failure() records the request as a failure, and resp.success() counts even an error code as a success. Failures are grouped by message text, so write precise messages. If some other exception is raised inside the block, for example because the response body is not JSON, that request is recorded only in the exceptions list, not in the request statistics; so catch the JSON error and record it with resp.failure().

Running without the web UI, and reports

The web UI is convenient for trying out a scenario, but for repeatable runs use --headless with fixed parameters so that results can be compared:

locust -f exam_day.py --headless -H https://staging.example.com \
       -u 3000 -r 50 -t 30m --csv exam_day --html exam_day.html

• -u is the peak number of concurrent users and -r the number of users added per second; here it takes one minute to reach 3000 users. For an exam that everyone starts at once, that steep ramp is part of the scenario.
• -t is the run time, counted from the start of the test, not from the end of the ramp-up.
• --csv exam_day writes exam_day_stats.csv (overall statistics per endpoint), exam_day_failures.csv, exam_day_exceptions.csv and exam_day_stats_history.csv. The history file gains new rows during the run: by default only for the aggregated total, and with --csv-full-history for every endpoint.
• --html writes an HTML report with charts and tables, handy for attaching to a ticket or comparing two runs.

If even one request fails, Locust exits with code 1 (configurable with --exit-code-on-error), which makes it easy to run the test in CI/CD. --reset-stats resets the statistics once, after all users have been spawned, so the ramp-up period does not end up in the final result (in a distributed run, pass it to both the master and the workers). For an exam scenario, look at that period separately in the history file.

When one machine is not enough: distributed runs

Because of the GIL, each Python process effectively uses one CPU core. If the Locust process's CPU usage goes above 90 percent, it logs a CPU usage above 90% warning; from then on, the numbers say more about the limits of the load generator than about the behavior of the API.

In distributed mode, a master process coordinates the test and aggregates the statistics, and worker processes run the users. On a single machine, locust --processes 4 starts a master and four workers, and --processes -1 starts one worker per logical core; this option relies on fork() and does not work on Windows. For several machines, the first command runs on the master and the second on each of the two load-generating machines:

locust -f exam_day.py --master --headless --expect-workers 8 \
       -H https://staging.example.com -u 12000 -r 200 -t 30m --csv exam_day

locust -f - --worker --master-host 10.0.0.10 --processes 4

• --expect-workers counts worker processes, not machines: two machines with four processes each make 8. The master waits until all of them have connected.
• -f - fetches the locustfile from the master, but only a single file; data files such as test_users.csv must be present on every machine. Each process reads the account list from the beginning, so if duplicate accounts are a problem, split the accounts across worker processes, not just across machines.
• By default, workers connect to the master on port 5557.
• Before adding machines, try FastHttpUser: according to the Locust documentation it uses much less CPU time. It does not make responses faster, it only makes the load generator lighter. Its API is similar but not identical; for example, it has no self.client.headers, so the token header has to be sent with each request.

Reading the results: RPS, median, percentiles and failures

For each endpoint, the Locust table shows the number of requests and failures, the average, minimum, maximum and median, and requests per second (RPS), and at the end it adds a table of percentiles. RPS is the actual throughput; if you add users and RPS does not rise, either the system or the load generator itself has hit its ceiling. The median is what a typical request experiences, and the 95th and 99th percentiles show the slow tail: 5 and 1 percent of requests are slower than them. None of these numbers mean much without the failure rate next to them.

The average is misleading, because a few very slow requests drag it up without showing the shape of the distribution. For example, if 90 out of 100 requests take 20 milliseconds and 10 take 2 seconds, the average is 218 milliseconds, a number that no request actually experienced. The median is 20 milliseconds and the 95th percentile two seconds; the real problem shows up only in the percentiles.

Two finer points. To compute percentiles, Locust rounds response times above 100 milliseconds to roughly two significant digits (147 ms is recorded as 150), so its percentiles are approximate. And Locust's load model is closed: a user does not send its next request until the response arrives, so when the server slows down, the send rate drops too, and the slow period is represented in the statistics by fewer samples. This effect is called coordinated omission. If your real users arrive independently and do not wait for each other's responses, Locust's percentiles may look more optimistic than what those users experience.

The Locust documentation describes the saturation pattern like this: users keep being added, but RPS stops rising and response times climb quickly. When you report results, write the conditions next to the numbers. A Locust benchmark of the watch-time event ingest service (a minimal API on .NET 10) on a 12-core workstation, with 10,000 events and 50 concurrent users, showed about 2,450 requests per second, a median of 13 ms and a 95th percentile of 21 ms, and the service's memory peaked at about 108 MB. Numbers like these are meaningful for comparing two versions of the same code on the same machine, not for predicting the capacity of a production server.

Finding the bottleneck: look at the server side too

Locust tells you which endpoint got slow, not why. My method is short: raise the load step by step until you reach saturation, and at that point see which resource has run out. If the API process's CPU is near 100 percent, the bottleneck is probably in the application's own processing; if CPU is low and response times are high, requests are waiting for something. Then change one thing only, rerun the same scenario with the same data, and put the CSV files of the two runs side by side.

On the .NET side, dotnet-counters shows a process's live counters without any code changes. The first command finds the process ID (here 4312):

dotnet-counters ps
dotnet-counters monitor -p 4312 --counters System.Runtime,Microsoft.AspNetCore.Hosting,Microsoft.Data.SqlClient.EventSource

• Thread pool. On .NET 9 and later, watch dotnet.thread_pool.thread.count and dotnet.thread_pool.queue.length (on .NET 8 they are called ThreadPool Thread Count and ThreadPool Queue Length). According to Microsoft's documentation, the sign of thread pool starvation is a slow, steady rise in the thread count, about one or two threads per second, while CPU stays well below 100 percent. The most common cause is sync-over-async code such as .Result or .Wait().
• Connection pool. The SqlClient counters (available since version 3.0 of the Microsoft.Data.SqlClient library) show the pool's active and free connections (number-of-active-connections and number-of-free-connections). The default Max Pool Size for each pool is 100; when all connections are in use, opening a connection waits for up to Connect Timeout (15 seconds by default) and then fails. A rule of thumb, not proof: if you have not changed Connect Timeout and a cluster of responses took about 15 seconds, suspect pool exhaustion and look for the timeout message in the application log. Before raising the limit, check for connection leaks, slow queries and blocked transactions.
• SQL Server. During the test, sys.dm_exec_requests shows what each request is waiting for (wait_type) and which session is blocking it (blocking_session_id). If Query Store is on, SQL Server 2017 and later record each query's waits by category. sys.dm_os_wait_stats accumulates from the last restart, so take a snapshot before and after the test. LCK_M_* means waiting for a lock, PAGEIOLATCH_* waiting for a read from disk, and ASYNC_NETWORK_IO means SQL Server is waiting for the application to consume the results; in that case the bottleneck is sometimes in the application itself. There is more detail in the SQL Server performance guide.
• Redis. SLOWLOG GET lists slow commands (execution time on the server only, without the network). A StackExchange.Redis timeout message also includes thread pool statistics (the WORKER section with Busy and Min); if Busy is higher than Min, new threads are added with a delay, and the root cause of the timeout may be in the application rather than in Redis.

Common mistakes

• Running Locust on the API machine in a capacity test. The load generator and the API compete for CPU, and the real network path and load balancer are bypassed. For comparing two versions of the same code this is fine, as long as you report it that way and do not draw conclusions about production capacity from it.
• Cold start. Right after a deployment, many methods have not been JIT-compiled yet, caches are empty and pools are only starting to grow. If you are measuring steady-state behavior, do a short low-load run first; if you are measuring the rush itself, such as the start of an exam, the cold start is part of the scenario.
• Unrealistic data. If everyone signs in with one account or opens the same exam, either everything comes from the cache or everyone queues for a lock on the same row. The spread of IDs and the data volume should be close to production.
• Ignoring 4xx and 5xx. An error response is usually cheaper than a correct one, so RPS looks high and response times look low while users get nothing. Common cases: 401 when the token expires during a long test; the ASP.NET Core rate limiter, which returns 503 by default and, if it is partitioned by IP, puts all Locust users, who come from a handful of IPs, into a single quota; and a deliberate 503, like the watch-time service rejecting requests when its in-memory channel is full. For that last case, set a separate failure message with catch_response.
• No wait time. Without wait_time, the next task runs immediately, and "3000 users" can no longer be compared with 3000 real students.

The exam-peak high availability case study gives the background to the pre-peak tests, and the details of the benchmark above are in the watch-time note. If you want to get a system ready before a traffic peak, the Work with me page explains how I can help.

Frequently asked questions

What is Locust?

Locust is an open-source load testing tool that describes user behavior in Python code. Each simulated user runs its own scenario, and for every endpoint Locust reports the number of requests, failures, RPS and response-time percentiles.

Can I use Locust to test an API written in .NET?

Yes. Locust talks to the API from the outside over HTTP, so the server-side language and framework make no difference to it. Only the scenario itself is written in Python.

What does spawn rate mean in Locust?

It is the number of users Locust adds per second until it reaches the target count. It does not change the final load, only how quickly you get there; for an event where everyone starts at the same time, a steep ramp is part of the scenario.

Why look at the 95th percentile instead of the average response time?

A few very slow requests pull the average up and hide the shape of the distribution. The 95th percentile shows the value that the slowest 5 percent of requests exceed, and that slow tail is what users feel.

When should Locust run distributed?

When Locust warns that CPU usage is above 90 percent, or a single process cannot generate the load you need. Each Locust process effectively uses one CPU core, so the usual approach is one worker per core, adding machines when needed.

More in this section

• Caching in ASP.NET Core: IMemoryCache, Redis and HybridCache
• RabbitMQ in .NET: retries, duplicate messages and the outbox pattern
• SQL Server performance tuning for high-traffic APIs