Test Data for Load Testing: Realistic, Skewed, Relational Datasets

Test data for load testing quietly decides whether your run tells the truth. Point k6 at ten thousand requests per second backed by a table of user1 through user500, each with the same three orders and a near-identical payload, and the database will happily serve it all from a warm cache and one narrow slice of an index. Green dashboards, low p99, ship it — then production buckles at half that load, because real traffic never had the tidy uniformity your dataset did. The fix isn't more virtual users; it's a dataset whose cardinality and skew look like production's.

What is test data for load testing?

It's the data your performance run actually exercises: the rows preloaded into the database and the values stuffed into the request bodies your load tool sends. It is not the traffic itself. A load test has two separable halves — the generator (k6, JMeter, Gatling, Locust) that produces requests, and the dataset those requests read and write against. JsonFabrica lives entirely on the dataset side. It generates the customers, products, and orders; it sends zero requests and runs no test. Getting the generator right but the dataset wrong is the most common way a load test ends up measuring the wrong thing.

Why not just use random data for a load test?

Because "random" almost always means uniform, and uniform data is the enemy of a truthful load test. When every customer has roughly the same number of orders and every payload is the same size, the entire workload collapses into a small, cache-friendly hot set. That masks the exact failure modes a load test exists to find:

• Cache hit-rate cliffs. A working set that fits in cache at test scale but won't at production scale never reveals the cliff. Uniform data keeps the working set artificially tiny.
• Hot-partition and hot-row contention. Real traffic concentrates on a few popular rows. Spread writes evenly across identical rows and you never see the lock contention that a handful of hot accounts cause in prod.
• Index selectivity problems. A query plan that looks fine against low-cardinality columns can flip to a table scan once the real distribution of values shows up. Near-identical data hides the flip.
• N+1 query blowups. A parent with three children per row barely registers; a parent with a long tail of children — hundreds on a few rows — turns an N+1 into a visible latency spike.

This is the same trap covered in Random Isn't Realistic: field-by-field random generation ignores the distribution and the relationships that real bugs hide in. For load testing specifically, the distribution is the test.

Realistic means skewed and relational

Production traffic follows a power law. A small number of customers hold most of the orders; most customers have one or two or none. Popular products dominate reads; the long tail barely moves. Realistic data for stress testing has to reproduce that skew, because the skew is what drives hot rows, cache pressure, and the join paths your real query plans take.

Skew alone isn't enough — the data also has to be relational. An order that points at a customer who actually exists forces the load to traverse the same foreign-key joins production does. Independently generated tables with random IDs that never line up will short-circuit those joins and, again, flatter your numbers. JsonFabrica's batch generation wires a child's field to a real parent value, so the join paths under load are the ones you ship.

Walk the examples repo from fixture-sized to load-sized

The jsonfabrica-examples repo has a ready-made relational scenario to start from. Clone it — it runs with just Node 18+ and an API key, no npm install — and open scenarios/ecommerce/batch.json. At fixture scale it generates 20 customers, 15 products, and 60 orders, with the orders wired to real customers and products:

{
  "seed": 20260903,
  "sequenceNamespace": "examples-ecommerce",
  "documents": [
    { "template": "templates/customer.json", "alias": "customer", "count": 20 },
    { "template": "templates/product.json",  "alias": "product",  "count": 15 },
    {
      "template": "templates/order.json",
      "alias": "order",
      "count": 60,
      "relations": {
        "customerId": { "from": "customer.id", "strategy": "round-robin" },
        "productId":  { "from": "product.id",  "strategy": "round-robin" }
      }
    }
  ]
}

Each template is a path to a local template file, not a templateId. npm run generate -- ecommerce reads this file, creates each template with POST /v1/templates, then submits the batch with the returned templateIds. round-robin is the only relation strategy, and it's exactly what you want: every order gets a real customerId and productId drawn from the customers and products generated in the same batch. To turn this into a load test data set, keep the relations and the template paths untouched and just raise the count values:

{
  "seed": 20260903,
  "sequenceNamespace": "examples-ecommerce",
  "documents": [
    { "template": "templates/customer.json", "alias": "customer", "count": 5000 },
    { "template": "templates/product.json",  "alias": "product",  "count": 2000 },
    {
      "template": "templates/order.json",
      "alias": "order",
      "count": 40000,
      "relations": {
        "customerId": { "from": "customer.id", "strategy": "round-robin" },
        "productId":  { "from": "product.id",  "strategy": "round-robin" }
      }
    }
  ]
}

How do I generate a large relational dataset?

The examples repo wraps the whole flow in one command:

npm run generate -- ecommerce

That reads scenarios/ecommerce/batch.json, creates each referenced template with POST /v1/templates, and submits the batch with the returned templateIds. If you'd rather drive the API directly, first create your templates (POST /v1/templates returns a templateId for each), then post a batch spec that references those ids — note it uses templateId, not the template file paths from the repo file:

curl -s -X POST https://api.jsonfabrica.com/v1/batches \
  -H 'Authorization: Bearer sk_...' \
  -H 'content-type: application/json' \
  -d '{
    "seed": 20260903,
    "sequenceNamespace": "examples-ecommerce",
    "documents": [
      { "templateId": "tpl_customer", "alias": "customer", "count": 5000 },
      { "templateId": "tpl_product",  "alias": "product",  "count": 2000 },
      {
        "templateId": "tpl_order",
        "alias": "order",
        "count": 40000,
        "relations": {
          "customerId": { "from": "customer.id", "strategy": "round-robin" },
          "productId":  { "from": "product.id",  "strategy": "round-robin" }
        }
      }
    ]
  }'

Small batches come back synchronously (200 with a results array), but a load-sized request crosses the threshold and is queued instead — you get a 202 with a job handle:

{ "batchId": "batch_7d2e4f10", "status": "queued", "seed": 20260903 }

Then poll GET /v1/batches/{batchId} until status is completed, and read the generated documents off the response. documents is a flat array of per-document envelopes, each tagged with its alias and carrying the generated record under result:

curl -s https://api.jsonfabrica.com/v1/batches/batch_7d2e4f10 \
  -H 'Authorization: Bearer sk_...'

{
  "batchId": "batch_7d2e4f10",
  "seed": 20260903,
  "status": "completed",
  "documents": [
    {
      "alias": "customer",
      "seqNo": 0,
      "templateId": "tpl_customer",
      "status": "completed",
      "result": { "id": "cus_0001", "name": "Dana Ruiz" },
      "documentSeed": 811234
    },
    {
      "alias": "order",
      "seqNo": 0,
      "templateId": "tpl_order",
      "status": "completed",
      "result": { "id": "ord_0001", "customerId": "cus_0001", "productId": "prod_0007" },
      "documentSeed": 907651
    }
  ]
}

The repo's generate.mjs regroups those envelopes by alias into scenarios/ecommerce/out.json — { "customer": [ ... ], "product": [ ... ], "order": [ ... ] } — which is the tidy shape you load from, but it's a post-processing step, not the raw API response.

From there you load the documents into your database before the run, or hand them to your load tool as the pool of request bodies it draws from. See the batches API reference for the full request and response shapes.

There is a ceiling. The async batch is not "generate ten million rows in one call" — very large datasets are produced across multiple batches, paginating the generation itself, rather than one giant request. Inside a single template, loops are capped at 10,000 total iterations, so a template can't unroll into an unbounded array. Plan large load sets as a handful of batches you concatenate, not a single monster call.

Pin the seed, or don't — on purpose

The batch request takes a seed, and which way you set it is a deliberate choice for load testing:

• Pin the seed for a repeatable load profile. A before-and-after performance comparison — did my index change help? — is only valid if the data underneath is byte-for-byte identical across runs. A fixed seed guarantees that, so the only variable is your code.
• Drop the seed for cache-busting variety between runs. If you're measuring cold-cache behavior, a fresh dataset each run stops the database from serving a working set it already warmed on the previous identical run.

Both are the same one-field change. For how seeds interact with sequences and counters when you need stable, collision-free IDs across separate runs, see Unique Test Data Generation.

How much test data do I need for a load test?

Match the shape before you chase a row count. If production has a few high-volume accounts and a long tail of near-idle ones, a small uniform sample won't trigger the same query plans no matter how many requests per second you drive. Reproduce the cardinality and the skew first; then scale the row count until the working set stops fitting comfortably in cache, because that's the regime where the interesting failures live. A precisely skewed hundred-thousand-row set will out-test a flat ten-million-row one every time.

What JsonFabrica does not do

To be blunt about the boundaries: JsonFabrica is not a load generator. It generates the dataset and nothing else — it produces no traffic, opens no connections, and runs no test. You still bring k6, JMeter, Gatling, or Locust to actually apply load. It also doesn't insert into your database; it returns JSON over HTTP, and loading that into a table or a request pool is your test harness's job. And a schema change is still a manual template edit — the same effort as editing a fixture — not something inferred from your database automatically.

FAQ

What is test data for load testing? It's the dataset a load or performance test runs against: the customers, orders, and other records preloaded into the database or fed into the request bodies your load tool sends. Good load test data mirrors production cardinality and skew, so that cache hit rates, index selectivity, and query plans behave the way they will under real traffic rather than being flattered by uniform, near-identical rows.

Does JsonFabrica run the load test or generate traffic? No. JsonFabrica generates the dataset only. It produces zero requests, opens zero connections, and runs no test. You feed its output into a real load generator like k6, JMeter, Gatling, or Locust, or preload it into your database before the run. It's the data source that makes the load realistic, not the load generator itself.

Why does uniform random test data make a load test lie? When every customer has the same number of orders and payloads are near-identical, the workload sits in a tiny hot set that caches perfectly and hits one narrow slice of the index. That hides cache hit-rate cliffs, hot-partition contention, poor index selectivity, and N+1 blowups that only appear with realistic cardinality and skew. The test passes; production falls over at the same load.

How do I generate a large relational dataset for stress testing? Define one template per entity, then submit a single batch that generates all of them together with a relations map wiring each child's foreign key to a parent via round-robin. Raise the count values to load size and the API queues the job asynchronously, returning a batchId you poll until its status is completed. Very large sets are split across multiple batches rather than one giant call.

Should I pin the seed for load test data? Pin the seed when you want the exact same dataset across runs, so a before-and-after performance comparison changes only your code, not the data underneath it. Drop the seed when you want fresh values each run to defeat caches that would otherwise be warmed by a repeated identical dataset. Both are one field on the batch request.

How much test data do I need for a load test? Enough to reproduce production's cardinality and distribution, not a round number of rows. If production has a few high-volume accounts and a long tail of near-idle ones, a small uniform sample won't trigger the same query plans no matter how many requests per second you throw at it. Match the shape first, then scale the row count until the working set no longer fits comfortably in cache.

JsonFabrica gives you the realistic, skewed, relational dataset a load test needs — generated once from a single batch definition and handed straight to k6, JMeter, Gatling, or your database preload step. It won't run the test for you; it makes sure the test measures the right thing.