PHPUnit Data Provider From JSON: Generated Laravel Test Data
A PHPUnit data provider is the natural home for Laravel test data that comes in rows: the same request, sent a dozen times with different bodies, each expecting a different response. In most suites those rows are PHP arrays typed into the provider by hand. This post moves them into one committed JSON fixture instead. A seeded template generates the request bodies, valid and invalid, each with the status and validation error it should produce. A short PHP script regenerates the file from a seed, and a Laravel feature test runs every row as its own named data set.
JsonFabrica is the generator, and it's only a JSON-over-HTTP API.
There's no Composer package, Laravel plugin or PHPUnit extension. Your
script calls the API and commits the result, and PHPUnit reads the file
with file_get_contents. Test runs never touch the network.
Everything below was run on Laravel 13.42 with PHPUnit 12.5 (the version the Laravel 13 skeleton installs) and PHP 8.5. Version-specific behavior is called out where it differs.
The Laravel endpoint under test
The example is a ticket order endpoint, POST /api/orders. An order
has an email, a ticket type, a quantity and an optional coupon code
that must match a row in the coupons table:
// app/Http/Controllers/OrderController.php
namespace App\Http\Controllers;
use App\Models\Order;
use Illuminate\Http\Request;
use Illuminate\Validation\Rule;
class OrderController extends Controller
{
public function store(Request $request)
{
$data = $request->validate([
'email' => ['required', 'email'],
'ticket' => ['required', Rule::in(['general', 'vip', 'student'])],
'quantity' => ['required', 'integer', 'between:1,10'],
'coupon' => ['nullable', 'string', 'exists:coupons,code'],
]);
return response()->json(Order::create($data), 201);
}
}
// routes/api.php
use App\Http\Controllers\OrderController;
use Illuminate\Support\Facades\Route;
Route::post('/orders', [OrderController::class, 'store']);
On Laravel 11 and later, routes/api.php isn't there by default;
php artisan install:api creates it, registers it under the /api
prefix and installs Sanctum. Order and Coupon are plain Eloquent models with $fillable
set to their columns (email, ticket, quantity, coupon and
code, percent_off), and Order uses the HasFactory trait.
When validation fails on a JSON request, Laravel returns a 422 with the messages keyed by field:
{
"message": "The quantity field must be between 1 and 10.",
"errors": {
"quantity": ["The quantity field must be between 1 and 10."]
}
}
That errors object is what the tests assert against.
A template for Laravel test data with planted invalid rows
The template produces the whole fixture: one coupon to insert before each test, and a table of request cases. Rows 1 to 7 are planted cases, chosen by loop index. Every row after that is an ordinary valid order.
<setVar('coupon', toUpperCase(getRandomSurname()))>{
"coupon": {
"code": "<getVar('coupon')>",
"percent_off": <getRandomNumber(5, 30)>
},
"cases": [
<for(i, 1, getParam('cases', 12))><if(getVar('i') > 1)>,<endIf>
{
"id": "case-<appendBefore('0', getVar('i'), 2)>",
"case": <if(getVar('i') == 1)>"malformed-email"
<elseIf(getVar('i') == 2)>"unknown-ticket"
<elseIf(getVar('i') == 3)>"zero-quantity"
<elseIf(getVar('i') == 4)>"max-quantity"
<elseIf(getVar('i') == 5)>"too-many"
<elseIf(getVar('i') == 6)>"unknown-coupon"
<elseIf(getVar('i') == 7)>"with-coupon"
<else>"valid"<endIf>,
"body": {
"email": <if(getVar('i') == 1)>
"<toLowerCase(getRandomName())>.example.com"
<else>"<getRandomEmail('example.com')>"<endIf>,
"ticket": <if(getVar('i') == 2)>"backstage"
<else>"<getRandomElement('general', 'vip', 'student')>"<endIf>,
"quantity": <if(getVar('i') == 3)>0
<elseIf(getVar('i') == 4)>10
<elseIf(getVar('i') == 5)>11
<else><getRandomNumber(1, 6)><endIf>,
"coupon": <if(getVar('i') == 6)>"NOPE-<getRandomNumber(100, 999)>"
<elseIf(getVar('i') == 7)>"<getVar('coupon')>"
<else>null<endIf>
},
"status": <if(getVar('i') == 4 || getVar('i') >= 7)>201<else>422<endIf>,
"errors": <if(getVar('i') == 1)>
{ "email": "must be a valid email address" }
<elseIf(getVar('i') == 2)>{ "ticket": "selected ticket is invalid" }
<elseIf(getVar('i') == 3 || getVar('i') == 5)>
{ "quantity": "must be between 1 and 10" }
<elseIf(getVar('i') == 6)>{ "coupon": "selected coupon is invalid" }
<else>null<endIf>
}<end_for>
]
}
What each part does:
couponis the only variable, because two fields read it.setVarstores one code, an upper-cased surname fromtoUpperCase. It becomes both the coupon row the test inserts and the body of case 7, so theexists:coupons,coderule has a real match on every render. Everything else is inline.- Each planted row breaks one rule. Case 1 has no
@, built fromtoLowerCaseand a random first name. Case 2 asks for a ticket type that doesn't exist, and case 6 sends a code with a dash and digits, which an all-letters surname code can never equal. The other fields in each row are ordinary valid values, so the row can only fail for one reason. - Two rows sit on the quantity limits. Case 4 sends 10, the upper
limit itself, and should pass. Case 5 sends 11 and should fail. An
off-by-one in the
betweenrule breaks one of them. - Every row carries its expected outcome.
statusanderrorscome from the same loop index as the defect, so the data and the expectation can't drift apart.errorsuses the same shape as Laravel's response, field name to message, so the test can pass it straight to the assertion. - The ids are zero-padded.
appendBeforeturns the loop index intocase-01,case-02and so on. They sort correctly and stay the same from one render to the next, which matters once they become data set names.
Template expressions support comparisons (==, !=, <, <=, >,
>=) and &&/||, but no arithmetic, which is why the planted rows
are picked by index rather than computed. for bounds are inclusive,
the loop bound reads the cases param inline with
getParam, and the
<if(getVar('i') > 1)>,<endIf> guard puts a comma before every item
except the first. The control flow docs cover
for and if/elseIf/else. Keep cases at 7 or more, or some
planted rows won't be generated.
The expected messages are fragments of Laravel's default English
validation messages. If you publish and edit lang/en/validation.php,
or test in another locale, change the fragments to match.
What seed 1117 produces
Rendered with seed 1117 and the default 12 cases, the fixture starts
like this, trimmed to the coupon and cases 1 and 7:
{
"seed": 1117,
"coupon": {
"code": "WOODSHAW",
"percent_off": 28
},
"cases": [
{
"id": "case-01",
"case": "malformed-email",
"body": {
"email": "kali.example.com",
"ticket": "vip",
"quantity": 4,
"coupon": null
},
"status": 422,
"errors": {
"email": "must be a valid email address"
}
},
{
"id": "case-07",
"case": "with-coupon",
"body": {
"email": "[email protected]",
"ticket": "vip",
"quantity": 4,
"coupon": "WOODSHAW"
},
"status": 201,
"errors": null
}
]
}
The seed key isn't part of the template. The regen script adds it
from the API response, so the file records how to rebuild itself.
Regenerating the JSON fixture from a seed
The regen script is the only code that talks to JsonFabrica. It sends
the raw template to
POST /v1/templates/generate, which
renders a template without saving it, and writes the result to
tests/Fixtures/orders.json:
<?php
// Regenerates tests/Fixtures/orders.json from scripts/orders.tmpl.
// php scripts/regen-orders.php # pinned seed, no diff
// php scripts/regen-orders.php --seed=4711 # rebuild a logged seed
// php scripts/regen-orders.php --random # API picks the seed
// Review the diff and commit. Tests never call the API.
const API = 'https://api.jsonfabrica.com/v1/templates/generate';
const ROOT = __DIR__.'/..';
const OUT = ROOT.'/tests/Fixtures/orders.json';
$opts = getopt('', ['seed:', 'random']);
$seed = isset($opts['random']) ? null : (int) ($opts['seed'] ?? 1117);
$payload = array_filter([
'body' => file_get_contents(ROOT.'/scripts/orders.tmpl'),
'params' => ['cases' => 12],
'seed' => $seed,
], fn ($v) => $v !== null);
$ch = curl_init(API);
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => json_encode($payload, JSON_THROW_ON_ERROR),
CURLOPT_HTTPHEADER => [
'Content-Type: application/json',
'Authorization: Bearer '.getenv('JSONFABRICA_API_KEY'),
],
CURLOPT_RETURNTRANSFER => true,
]);
$body = curl_exec($ch);
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
// errors arrive as {"error": {"code", "message", "details"}}
if ($status !== 200) {
fwrite(STDERR, "$status: $body\n");
exit(1);
}
$result = json_decode($body, true, flags: JSON_THROW_ON_ERROR);
$fixture = ['seed' => $result['meta']['seed']] + $result['data'];
$flags = JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR;
file_put_contents(OUT, json_encode($fixture, $flags)."\n");
echo "seed {$fixture['seed']}: ".count($fixture['cases'])." cases\n";
It needs only PHP's curl extension, which most installs have, and no Composer dependency. Things worth knowing about the request and response:
- The response is
{ data, meta }.datais the generated document.meta.seedis the seed that was used, either the one you sent or, with--random, the one the API picked.meta.generatedAtis left out of the file because it changes on every call. - Same template, seed and params give the same file. Running the
script with the pinned seed produces no diff. Editing the template or
changing
casesshifts the random draws, so commit the template and the fixture together. - The template lives in
scripts/, next to the script, not intests/Fixtures/, so nothing mistakes it for test data.
JSONFABRICA_API_KEY=... php scripts/regen-orders.php
vendor/bin/phpunit
git add scripts/orders.tmpl tests/Fixtures/orders.json
If you'd rather not write PHP for this, the same request is a jq -n
payload piped to curl, with a second jq call to merge meta.seed
into data. What matters is that the generated file is committed and
the tests read the file.
A PHPUnit data provider from JSON, with named data sets
The feature test reads the fixture in a data provider and yields one named data set per case:
// tests/Feature/CreateOrderTest.php
namespace Tests\Feature;
use App\Models\Coupon;
use Illuminate\Foundation\Testing\RefreshDatabase;
use PHPUnit\Framework\Attributes\DataProvider;
use Tests\TestCase;
class CreateOrderTest extends TestCase
{
use RefreshDatabase;
// Data providers run before the Laravel app boots, so no base_path().
private const FIXTURE = __DIR__.'/../Fixtures/orders.json';
private static function fixture(): array
{
static $fixture = null;
return $fixture ??= json_decode(
file_get_contents(self::FIXTURE), true, flags: JSON_THROW_ON_ERROR
);
}
public static function orderCases(): iterable
{
foreach (self::fixture()['cases'] as $c) {
yield "{$c['id']} {$c['case']}" => [
$c['body'], $c['status'], $c['errors'],
];
}
}
protected function setUp(): void
{
parent::setUp();
Coupon::create(self::fixture()['coupon']);
}
#[DataProvider('orderCases')]
public function test_post_orders(
array $body, int $status, ?array $errors
): void {
$response = $this->postJson('/api/orders', $body);
$response->assertStatus($status);
if ($errors !== null) {
// the one field this row was built to break, and no other
$response->assertOnlyJsonValidationErrors($errors);
$this->assertDatabaseCount('orders', 0);
} else {
$response->assertJson($body);
$this->assertDatabaseHas('orders', ['email' => $body['email']]);
}
}
}
How the provider side fits together:
- The provider is
public static. PHPUnit 10 and later expect data providers to be public static methods. PHPUnit 12 reports anything else as an invalid data provider and runs none of that test's data sets, with errors likeData Provider method ...::orderCases() is not static(oris not public). The rest of the suite still runs. - String keys name the data sets. Each
yieldkey, such ascase-05 too-many, becomes the data set's name in PHPUnit's output, in failure messages and in--filter. Without keys you'd getdata set #4, and you'd have to count rows to find it. - Each data set is a list of arguments. PHPUnit passes the three
values to
test_post_ordersin order, so the test method's signature documents what a row contains, and a malformed row fails with a type error instead of an undefined index. - The fixture is decoded once. The
staticvariable caches the parsed file, so the provider and the 12setUpcalls share onejson_decode.
--testdox prints the result as the table of cases (cases 9 to 11
trimmed):
Create Order (Tests\Feature\CreateOrder)
✔ Post orders with data set "case-01 malformed-email"
✔ Post orders with data set "case-02 unknown-ticket"
✔ Post orders with data set "case-03 zero-quantity"
✔ Post orders with data set "case-04 max-quantity"
✔ Post orders with data set "case-05 too-many"
✔ Post orders with data set "case-06 unknown-coupon"
✔ Post orders with data set "case-07 with-coupon"
✔ Post orders with data set "case-08 valid"
...
✔ Post orders with data set "case-12 valid"
OK (12 tests, 46 assertions)
Every test runner has its own name for this shape. It's parametrize
in the pytest version, test.each in the
Jest and Vitest post, table-driven
subtests in the Go version, xUnit theories in the
.NET post, @MethodSource in the
Spring Boot post, and a plain loop in
the RSpec version. In PHPUnit it's a data
provider.
Why the data provider can't call Laravel helpers
PHPUnit calls every data provider while it builds the test suite,
before the first setUp runs. In a Laravel test, setUp is where the
application gets created, so inside the provider there's no app yet.
Swap self::FIXTURE for base_path('tests/Fixtures/orders.json') and
PHPUnit reports the provider as invalid and runs none of that test's
data sets. Other test classes still run:
The data provider specified for
Tests\Feature\CreateOrderTest::test_post_orders is invalid
Call to undefined method Illuminate\Container\Container::basePath()
That's PHPUnit's message with the first line wrapped to fit. The same
applies to config(), factories and the database. fake() is the
exception: it falls back to a default en_US generator, so it runs,
but it's unseeded and ignores app.faker_locale, so the data sets
change on every run. That's why
the provider reads the file with __DIR__ and plain PHP, and why the
coupon is inserted in setUp, where the app and the database exist.
RefreshDatabase gives every data set a clean database, so each one
starts with exactly one coupon and no orders.
The @dataProvider annotation and PHPUnit versions
Older tests declare the provider in a docblock, /** @dataProvider orderCases */. PHPUnit 10 added the #[DataProvider] attribute and
still reads the annotation. PHPUnit 11 deprecates doc-comment
annotations, and PHPUnit 12 no longer reads them. On PHPUnit 12.5 the
annotation is silently ignored and the test fails with ArgumentCountError: Too few arguments, because PHPUnit calls it with no data. If a suite
breaks that way after an upgrade, switch the annotations to attributes.
PHPUnit 11 prints a deprecation for each test method (and class) that still uses doc-comment annotations.
Asserting with postJson and assertJsonValidationErrors
The test method body is mostly Laravel's HTTP testing helpers:
postJsonsends a real JSON body. It encodes the array as JSON and sets theAccept: application/jsonheader, so a failed validation returns the 422 JSON shown earlier instead of a redirect. Integers stay integers andnullstaysnull, exactly as a client would send them.assertStatus($status)comes first. If a row that should fail comes back 201, the test stops with "Expected response status code [422] but received 201", which is more useful than a missing-key error from the next line.assertJsonValidationErrors($errors)checks field and message. With['quantity' => 'must be between 1 and 10'], it asserts thaterrors.quantityexists and that one of its messages contains that text. A bare list,['quantity'], would only check the key.assertOnlyJsonValidationErrorsalso rejects extra errors.assertJsonValidationErrorspasses if other fields fail too. TheOnlyvariant runs it and then fails with "Response has unexpected validation errors" if any other key is present, so the row has to fail on exactly the field it was built to break. Laravel 12 and 13 (and 11.x releases from February 2025 on) have it. On older versions, callassertJsonValidationErrors($errors)and add$this->assertCount(count($errors), $response->json('errors')).- Valid rows check the response and the database.
assertJsonpasses if the response contains the request body as a subset, andassertDatabaseHasconfirms the order was stored.
Checking the message, not just the field, earns its keep. Raise the
limit to between:1,20 in the controller and two rows fail, for two
different reasons. This is the real output, shortened with ... and
wrapped to fit:
1) ...\CreateOrderTest::test_post_orders@case-03 zero-quantity with data
(['[email protected]', 'student', 0, null], 422, [...])
Failed to find a validation error in the response for key and message:
'quantity' => 'must be between 1 and 10'
Response has the following JSON validation errors:
{
"quantity": [
"The quantity field must be between 1 and 20."
]
}
2) ...\CreateOrderTest::test_post_orders@case-05 too-many with data
(['[email protected]', 'general', 11, null], 422, [...])
Expected response status code [422] but received 201.
Failed asserting that 201 is identical to 422.
Case 5 is the obvious failure, since 11 is now allowed. Case 3 still returns a 422, because 0 is below both limits, so a test that only checked the status or the field name would have passed it. The message fragment shows that the rule behind the error changed.
Running one data set and reproducing a failure from its seed
To rerun a single data set, pass the method name, @ and the data set
name to --filter:
vendor/bin/phpunit --filter 'test_post_orders@case-05.*'
The filter is a regular expression, and the trailing .* matches the
rest of the name after the id, so you don't have to type the label.
php artisan test passes the same option through to PHPUnit. Because
ids come from the loop index, case-05 is the too-many row in every
regeneration.
The pinned seed keeps everyday runs identical. You can also have a scheduled CI job try a fresh seed each time, to vary the random parts of the data: the emails, ticket types, quantities and coupon code. Each planted row keeps its defect and its expected outcome at every seed.
php scripts/regen-orders.php --random # prints "seed 4172046628: ..."
vendor/bin/phpunit tests/Feature/CreateOrderTest.php
The job doesn't commit anything. The regen script prints meta.seed
before PHPUnit starts, so the seed is in the job log next to the
failure, and it's also the seed key of the fixture file the job
tested with. To reproduce a failure locally, rebuild that exact file
and run the failing data set:
php scripts/regen-orders.php --seed=4172046628
vendor/bin/phpunit --filter 'test_post_orders@case-09.*'
Regenerating with the same seed gives a byte-identical file, so the local run sees exactly the data CI saw. Once you've found the bug, either commit that seed's file or add the input that broke things as a new planted row. Then run the regen script with no arguments to put the pinned fixture back.
Laravel factories: alternative or companion?
Laravel's model factories are the default way to build test data, and for many tests they're the right tool:
// database/factories/OrderFactory.php
namespace Database\Factories;
use App\Models\Order;
use Illuminate\Database\Eloquent\Factories\Factory;
/**
* @extends Factory<Order>
*/
class OrderFactory extends Factory
{
public function definition(): array
{
return [
'email' => fake()->unique()->safeEmail(),
'ticket' => fake()->randomElement(['general', 'vip', 'student']),
'quantity' => fake()->numberBetween(1, 10),
'coupon' => null,
];
}
public function vip(): static
{
return $this->state(['ticket' => 'vip']);
}
}
// tests/Feature/OrderFactoryTest.php
namespace Tests\Feature;
use App\Models\Order;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Tests\TestCase;
class OrderFactoryTest extends TestCase
{
use RefreshDatabase;
protected function setUp(): void
{
parent::setUp();
fake()->seed(1117); // same seed, same factory values, every run
}
public function test_rejects_more_than_ten_tickets(): void
{
$body = Order::factory()->raw(['quantity' => 11]);
$this->postJson('/api/orders', $body)
->assertUnprocessable()
->assertJsonValidationErrors(['quantity' => 'between 1 and 10']);
}
public function test_stores_a_vip_order(): void
{
$body = Order::factory()->vip()->raw();
$this->postJson('/api/orders', $body)->assertCreated();
$this->assertDatabaseHas('orders', $body);
}
}
The test states the one attribute it cares about right where it's
used. raw() returns the attributes as an array, which makes a good
request body, while make() builds an unsaved model and create()
saves one. States like vip(), relationships through has() and
for(), and sequences cover most of what a model test needs. There's
no template, script or API key involved, and a factory evolves with
the model in the same pull request.
Factory values are random unless you seed Faker. fake()->seed(1117)
in setUp is enough: Faker's seed() calls PHP's mt_srand, which
seeds the generator that factory definitions draw from. Each test
seeds it again, so a test produces the same values whether it runs
alone or in the full suite. In this suite, the too-many test posts
[email protected] with a general ticket on every run, and changing
the seed changes the values. Two caveats: inserting a field into a
definition shifts every later draw, and upgrading fakerphp/faker can
change the generated values, since the seed is replayed against
whatever data lists the installed version ships.
So where does the JSON fixture earn its extra machinery?
- The cases are reviewable as a table. Each row and its expected status and error sit in one diffable file, instead of being spread across test methods and factory overrides.
- The data doesn't change until you regenerate it. A committed file isn't affected by a Faker upgrade or a reordered definition.
- Rows can depend on each other. The coupon code that has to exist in the database and appear in exactly one request body is one variable in the template.
- Other tools can read it. The same file can feed a frontend mock server, a Postman collection or a service written in another language.
It costs you a regen script, an API key for whoever regenerates, and a second place to update when the validation rules change. Factories win for tests that build a few models with specific attributes or relationships. The fixture wins for the table of requests an endpoint must accept or reject. Most suites end up using both: factories for model and policy tests, and the fixture behind the data provider.
FAQ
How do I use a data provider in PHPUnit 10, 11 or 12?
Write a public static method that returns an array or yields data sets,
and put the #[DataProvider('methodName')] attribute from
PHPUnit\Framework\Attributes on the test method. Each data set is a
list of arguments passed to the test method in order. Give the data
sets string keys and PHPUnit uses them as the data set names in its
output and in --filter.
Why can't I use Laravel helpers in a PHPUnit data provider?
PHPUnit calls data providers while it builds the test suite, before any
test's setUp method runs, so the Laravel application hasn't been
created yet. Helpers like base_path() and config() fail there, for
example with Call to undefined method Illuminate\Container\Container::basePath(). fake() still runs, but
it is unseeded and ignores app.faker_locale, so its values change on
every run. Read files with plain PHP
and __DIR__ in the provider, and do anything that needs the app or
the database in setUp or the test itself.
How do I run a single data set in PHPUnit?
Pass the test method name, an @ sign and the data set name to
--filter, for example
vendor/bin/phpunit --filter 'test_post_orders@case-05.*'. The filter
is a regular expression, so the trailing .* matches the rest of the
name and you only have to type its start. Unnamed data sets are numbered instead, and
you select them with a # and the index.
How do I make Faker data reproducible in Laravel tests?
Call fake()->seed(1234) in your test's setUp method, after
parent::setUp(). Faker's seed() seeds PHP's Mersenne Twister
generator, so factory definitions that use fake() return the same
values on every run, whether the test runs alone or in the full suite.
Values can still change when you edit a factory definition or upgrade
fakerphp/faker.
Should I use Laravel factories or JSON fixtures for test data?
Use model factories when a test needs a few Eloquent models with specific attributes, relationships or states, created through the model the way your app creates them. Use a JSON fixture when many tests share one table of cases, when each case should carry its expected result and be reviewable in a pull request, or when other tools and languages need the same data. Many Laravel suites use both.
Is there a JsonFabrica PHPUnit or Laravel package?
No. JsonFabrica is a plain JSON-over-HTTP API, with no Composer package, Laravel plugin, PHPUnit extension, SDK or database connector. You write a small regen script, in PHP with ext-curl or with curl and jq, that sends a template and a seed to the API and writes the generated JSON into your repository. PHPUnit reads that committed file, and test runs never call the API.
A PHPUnit data provider backed by a generated fixture gives you one
committed file of request cases, with planted invalid rows and the
response each one should get, rebuilt from a single seed. Rendering a
raw template with a seed and params is one request to the JsonFabrica
API; the templates API reference
documents the request body and the { data, meta } response.
Generate realistic test data with JsonFabrica
Describe the shape of your data once, then generate as many fresh, realistic JSON documents as you need via a simple API call.