RSpec Test Data: Generated JSON Fixtures for Rails Request Specs
RSpec test data for a Rails API usually lives in one of two places:
FactoryBot calls scattered through the specs, or request bodies typed
out by hand inside each it block. Either way, nobody can see the full
list of inputs the endpoint is tested with. This post puts that list in
one committed JSON fixture. A seeded template generates the request
bodies, valid and invalid, each with the status and validation error it
should produce. A short Ruby script regenerates the file from a seed,
and the spec turns every row into its own RSpec example.
JsonFabrica is the generator, and it's only a JSON-over-HTTP API. There is no gem or Rails plugin. Your script calls the API and commits the result, and your specs read the file with plain Ruby. Test runs never touch the network.
The endpoint under test
The example is a sign-up endpoint, POST /accounts, backed by one
model:
# app/models/account.rb
class Account < ApplicationRecord
PLANS = %w[free pro team].freeze
validates :name, presence: true
validates :email, format: { with: URI::MailTo::EMAIL_REGEXP },
uniqueness: true
validates :plan, inclusion: { in: PLANS }
validates :seats, numericality: {
only_integer: true,
greater_than_or_equal_to: 1,
less_than_or_equal_to: 50
}
end
The controller returns the new account with a 201, or a 422 listing each error as its attribute and its error type:
# app/controllers/accounts_controller.rb
class AccountsController < ApplicationController
def create
account = Account.new(account_params)
if account.save
render json: account.slice(:id, :email, :plan, :seats), status: 201
else
errors = account.errors.map { |e| { field: e.attribute, code: e.type } }
render json: { errors: errors }, status: 422
end
end
private
def account_params
params.require(:account).permit(:name, :email, :plan, :seats)
end
end
Error types such as :taken and :inclusion don't change with the
locale or with custom messages, so they're more stable to assert on
than message strings. Clients send a flat JSON body. Rails' params
wrapper, on by default for JSON in new apps, nests it under account
for params.require.
A template for RSpec test data with planted invalid rows
The template produces the whole fixture: one existing account for the uniqueness check, and a table of request cases. Rows 1 to 6 are planted edge cases, chosen by loop index. Every row after that is an ordinary valid sign-up.
<setVar('taken', getRandomEmail('example.com'))>{
"existing": {
"name": "<getRandomFullName()>",
"email": "<getVar('taken')>",
"plan": "pro",
"seats": <getRandomNumber(2, 10)>
},
"cases": [
<for(i, 1, getParam('cases', 12))><if(getVar('i') > 1)>,<endIf>
{
"id": "case-<appendBefore('0', getVar('i'), 2)>",
"case": <if(getVar('i') == 1)>"malformed-email"
<elseIf(getVar('i') == 2)>"duplicate-email"
<elseIf(getVar('i') == 3)>"unknown-plan"
<elseIf(getVar('i') == 4)>"zero-seats"
<elseIf(getVar('i') == 5)>"max-seats"
<elseIf(getVar('i') == 6)>"too-many-seats"
<else>"valid"<endIf>,
"body": {
"name": "<getRandomFullName()>",
"email":
<if(getVar('i') == 1)>"<toLowerCase(getRandomName())>.example.com"
<elseIf(getVar('i') == 2)>"<getVar('taken')>"
<else>"<getRandomEmail('example.com')>"<endIf>,
"plan": <if(getVar('i') == 3)>"enterprise"
<else>"<getRandomElement('free', 'pro', 'team')>"<endIf>,
"seats": <if(getVar('i') == 4)>0
<elseIf(getVar('i') == 5)>50
<elseIf(getVar('i') == 6)>51
<else><getRandomNumber(1, 10)><endIf>
},
"status": <if(getVar('i') == 5 || getVar('i') > 6)>201<else>422<endIf>,
"field": <if(getVar('i') <= 2)>"email"
<elseIf(getVar('i') == 3)>"plan"
<elseIf(getVar('i') == 4 || getVar('i') == 6)>"seats"
<else>null<endIf>,
"code": <if(getVar('i') == 1)>"invalid"
<elseIf(getVar('i') == 2)>"taken"
<elseIf(getVar('i') == 3)>"inclusion"
<elseIf(getVar('i') == 4)>"greater_than_or_equal_to"
<elseIf(getVar('i') == 6)>"less_than_or_equal_to"
<else>null<endIf>
}<end_for>
]
}
What each part does:
takenis the only variable, because two fields read it.setVarstores one random email, which becomes both the existing account's address and the request body of the duplicate-email case. That keeps case 2 a real duplicate on every render. Everything else is inline.- Each planted row breaks one rule. Case 1 has no
@, built fromtoLowerCaseand a random first name. Case 3 asks for a plan that doesn't exist, and cases 4 and 6 sit one step outside the seat limits. Each row's other fields are ordinary valid values, so it can only fail for one reason. - Case 5 is an edge that should pass. 50 seats is the upper limit itself. Off-by-one bugs at the upper limit fail either case 5 or case 6.
- Every row carries its expected outcome.
status,fieldandcodeare derived from the same loop index as the defect, so the data and the expectation can't drift apart. - The ids are zero-padded.
appendBeforeturns the loop index intocase-01,case-02and so on, so the ids sort correctly and match from one render to the next.
Template expressions support comparisons (==, !=, <, <=, >,
>=) and &&/||, but no arithmetic, which is why the planted rows
are picked by index rather than computed. for bounds are inclusive,
the loop bound reads the cases param inline with
getParam, and the
<if(getVar('i') > 1)>,<endIf> guard puts a comma before every item
except the first. The control flow docs cover
for and if/elseIf/else. Keep cases at 6 or more, or some
planted rows won't be generated.
What seed 1109 produces
Rendered with seed 1109 and the default 12 cases, the fixture starts
like this, trimmed to the existing account and the first two cases:
{
"seed": 1109,
"existing": {
"name": "George Dalham",
"email": "[email protected]",
"plan": "pro",
"seats": 2
},
"cases": [
{
"id": "case-01",
"case": "malformed-email",
"body": {
"name": "Penelope Yorkshaw",
"email": "memphis.example.com",
"plan": "team",
"seats": 6
},
"status": 422,
"field": "email",
"code": "invalid"
},
{
"id": "case-02",
"case": "duplicate-email",
"body": {
"name": "Caroline Pemble",
"email": "[email protected]",
"plan": "pro",
"seats": 5
},
"status": 422,
"field": "email",
"code": "taken"
}
]
}
The seed key isn't part of the template. The regen script adds it
from the API response, so the file records how to rebuild itself.
Regenerating the RSpec JSON fixtures from a seed
The regen script is the only code that talks to JsonFabrica. It sends
the raw template to
POST /v1/templates/generate, which
renders a template without saving it, and writes the result to
spec/fixtures/files/signups.json:
#!/usr/bin/env ruby
# Regenerates spec/fixtures/files/signups.json from script/signups.tmpl.
# ruby script/regen_signups.rb # pinned seed, no diff
# ruby script/regen_signups.rb --seed 4711 # rebuild a logged seed
# ruby script/regen_signups.rb --random # API picks the seed
# Review the diff and commit. Specs never call the API.
require "json"
require "net/http"
require "optparse"
API = URI("https://api.jsonfabrica.com/v1/templates/generate")
ROOT = File.expand_path("..", __dir__)
OUT = File.join(ROOT, "spec/fixtures/files/signups.json")
seed = 1109 # change only when you mean to replace the data
OptionParser.new do |opts|
opts.on("--seed N", Integer) { |n| seed = n }
opts.on("--random") { seed = nil }
end.parse!
payload = {
body: File.read(File.join(ROOT, "script/signups.tmpl")),
params: { cases: 12 },
seed: seed
}.compact
response = Net::HTTP.post(
API, payload.to_json,
"Content-Type" => "application/json",
"Authorization" => "Bearer #{ENV.fetch("JSONFABRICA_API_KEY")}"
)
# errors arrive as {"error": {"code", "message", "details"}}
abort "#{response.code}: #{response.body}" unless response.code == "200"
result = JSON.parse(response.body)
fixture = { seed: result["meta"]["seed"] }.merge(result["data"])
File.write(OUT, JSON.pretty_generate(fixture) + "\n")
puts "seed #{fixture[:seed]}: #{fixture["cases"].size} cases"
It uses only the standard library, so it doesn't need to go in your Gemfile. Things worth knowing about the request and response:
- The response is
{ data, meta }.datais the generated document.meta.seedis the seed that was used, either the one you sent or, with--random, the one the API picked.meta.generatedAtis left out of the file because it changes on every call. - Same template, seed and params give the same file. Running the
script with the pinned seed produces no diff. Editing the template or
changing
casesshifts the random draws, so commit the template and the fixture together. - The template lives in
script/, next to the script, not inspec/fixtures/, so nothing mistakes it for a fixture.
JSONFABRICA_API_KEY=... ruby script/regen_signups.rb
bundle exec rspec
git add script/signups.tmpl spec/fixtures/files/signups.json
Generating RSpec examples from the fixture's rows
The request spec reads the fixture once and defines one example per case, with the case's id, label and expected outcome in its description:
# spec/requests/accounts_spec.rb
require "rails_helper"
# Examples are defined when this file loads, before any example runs,
# so read the fixture by path here: file_fixture only works inside one.
signups = JSON.parse(
Rails.root.join("spec/fixtures/files/signups.json").read,
symbolize_names: true
)
RSpec.describe "POST /accounts (seed #{signups[:seed]})", type: :request do
before { Account.create!(signups[:existing]) }
signups[:cases].each do |c|
expected = c[:code] ? "#{c[:status]} #{c[:field]} #{c[:code]}" : "201"
it "#{c[:id]} #{c[:case]} -> #{expected}" do
post "/accounts", params: c[:body], as: :json
expect(response).to have_http_status(c[:status])
json = JSON.parse(response.body, symbolize_names: true)
if c[:code]
# exactly one error, on the field this row was built to break
expect(json[:errors]).to eq([{ field: c[:field], code: c[:code] }])
else
expect(json).to include(c[:body].slice(:email, :plan, :seats))
end
end
end
end
How the pieces fit:
- The loop runs at describe level. RSpec evaluates the
describeblock when it loads the file, so theeachdefines 12 separate examples. Each one reports its own pass or fail, and one bad row doesn't hide the others. symbolize_names: truemakes the rows look like Ruby. You writec[:body]instead ofc["body"], andc[:body]can be passed straight topostas the params hash.as: :jsonsends a real JSON body. Integers stay integers andnullstaysnull, the same as a client would send them, instead of being form-encoded into strings.- Invalid rows must fail for the right reason. Comparing the whole
errorsarray to one expected entry catches a row that fails twice, or fails on a different field. A spec that only checked for 422 would pass if the seat check fired on the duplicate-email row. - The existing account is created per example. With
use_transactional_fixtureson, as in the generatedrails_helper.rb, each example runs in a transaction that's rolled back, so thebeforehook doesn't pile up duplicate rows.
Run with the documentation formatter, the output reads as the table of cases:
POST /accounts (seed 1109)
case-01 malformed-email -> 422 email invalid
case-02 duplicate-email -> 422 email taken
case-03 unknown-plan -> 422 plan inclusion
case-04 zero-seats -> 422 seats greater_than_or_equal_to
case-05 max-seats -> 201
case-06 too-many-seats -> 422 seats less_than_or_equal_to
case-07 valid -> 201
...
case-12 valid -> 201
12 examples, 0 failures
Every test runner has its own name for this shape. It's
parametrize in the pytest version, test.each
in the JavaScript post, subtests in
the Go version, xUnit theories in the
.NET post and @MethodSource in the
Spring Boot post. In RSpec it's just a
loop, because a describe block is ordinary Ruby.
Rerunning one generated example
All 12 examples come from the same it line, so
rspec spec/requests/accounts_spec.rb:17 runs all of them. When an
example fails, RSpec lists it by example id instead. This output is
from the same spec with the seat limit raised to 100 in the model. The
long descriptions are cut short with ... to fit:
1) POST /accounts (seed 1109) case-06 too-many-seats -> 422 seats ...
Failure/Error: expect(response).to have_http_status(c[:status])
expected the response to have status code 422 but it was 201
Failed examples:
rspec ./spec/requests/accounts_spec.rb[1:6] # POST /accounts (seed ...
[1:6] means the sixth example in the first top-level group, and it
reruns exactly that example. bundle exec rspec -e case-06 does the
same by name. Because ids come from the loop index, case-06 is the
too-many-seats row in every regeneration.
Loading Rails file_fixture JSON inside examples
rspec-rails includes Rails' file_fixture helper. It returns a
Pathname under spec/fixtures/files, which is the default value of
the file_fixture_path setting. The catch is that it's an instance
method. It works inside it, let and before, but calling it in a
describe block raises an error when the file loads, before any
example runs:
`file_fixture` is not available on an example group (e.g. a `describe`
or `context` block). It is only available from within individual
examples (e.g. `it` blocks) or from constructs that run in the scope of
an example (e.g. `before`, `let`, etc).
That's the RSpec error message, wrapped to fit. It's why the request
spec reads the file by path: it needs the rows at load time, to define
examples. Inside examples, file_fixture is the natural fit. A good
use for it is a small spec that checks the fixture's own invariants:
# spec/data/signups_fixture_spec.rb
require "rails_helper"
RSpec.describe "signups.json" do
let(:signups) do
JSON.parse(file_fixture("signups.json").read, symbolize_names: true)
end
let(:cases) { signups[:cases] }
it "plants each error code exactly once" do
expect(cases.filter_map { |c| c[:code] }).to contain_exactly(
"invalid", "taken", "inclusion",
"greater_than_or_equal_to", "less_than_or_equal_to"
)
end
it "reuses the existing email only in the duplicate case" do
taken = cases.select { |c| c[:body][:email] == signups[:existing][:email] }
expect(taken.map { |c| c[:case] }).to eq(["duplicate-email"])
end
it "gives every case a unique email" do
emails = cases.map { |c| c[:body][:email] }
expect(emails).to eq(emails.uniq)
end
end
If someone lowers cases below 6 or edits the template so that a
random valid email collides with the taken one, these specs fail with a
clear message about the data, instead of a request spec failing in a
confusing way. If you keep fixture files somewhere else, set
config.file_fixture_path in rails_helper.rb.
Reproducing a failure from the logged seed
The pinned seed keeps everyday runs identical. You can also have a scheduled CI job try a fresh seed each time, to vary the random parts of the data: the names, emails, plans and seat counts of the valid rows. Each planted row keeps its defect and its expected outcome at every seed.
ruby script/regen_signups.rb --random # prints the seed it got
bundle exec rspec spec/requests spec/data
The job doesn't commit anything. If it fails, the seed is in the script's output, and in the name of every failing request example, because that describe label reads it from the file. To reproduce the failure locally:
ruby script/regen_signups.rb --seed 662544252
bundle exec rspec "spec/requests/accounts_spec.rb[1:9]"
Example ids are positions, and the number and order of cases don't
depend on the seed, so [1:9] is the same case at any seed (for a
failure in spec/data, run bundle exec rspec spec/data instead). Once
you've found the bug, either commit that seed's file or add the input
that broke things as a new planted row.
Don't confuse this seed with the one RSpec prints as "Randomized with
seed 4242" when config.order = :random is on. That seed controls the
order examples run in, and rspec --seed 4242 replays the order, but
it has no effect on the JSON file. The
guide to replaying a failure from its seed
covers capturing seeds in CI in more depth.
Rails YAML fixtures and a generated JSON file do different jobs
Rails already has fixtures, and they're good at what they do. A YAML
file per table in spec/fixtures (or test/fixtures under Minitest)
defines named rows:
# spec/fixtures/accounts.yml
acme:
name: Acme Admin
email: [email protected]
plan: team
seats: 12
Declare fixtures :accounts in a group, or set
config.global_fixtures = :all, and Rails inserts the rows before the
examples run. You get them back by label with accounts(:acme). Ids
are derived from the label, associations can refer to other rows by
label, and ERB works inside the file. The location comes from
config.fixture_paths. Newer rspec-rails versions generate that
setting, while older ones use the singular fixture_path, so check
your version's docs. YAML fixtures are deterministic, readable and
reviewed in pull requests already, so the JSON file isn't there to fix
any of that.
The difference is what the rows are. A YAML fixture describes database
rows, inserted through ActiveRecord::FixtureSet without running
validations or callbacks. A row with seats: 0 and an invalid email
loads without complaint, and from then on it's part of every example
that uses the table, without testing anything. The generated file
describes requests: input a client sends, together with the response
it should get. Most of it never reaches the database. Some of it, like
the malformed email, is meant to be rejected before it gets there.
So the two complement each other. Keep stable reference rows in YAML,
such as an admin account or the plans table, and keep the table of
request cases, with its planted invalid rows and expected outcomes, in
the generated JSON. The Django post
draws the same line between loaddata rows and invalid form input.
If you'd rather generate the YAML side as well, that's possible,
because ERB can read the JSON file, but hand-written YAML is usually
clearer for a few reference rows.
FactoryBot alternative or complement?
FactoryBot is the default way to build test data in most Rails suites, and for a lot of specs it's the right choice:
# spec/factories/accounts.rb
FactoryBot.define do
factory :account do
name { Faker::Name.name }
sequence(:email) { |n| "user#{n}@example.com" }
plan { "pro" }
seats { 3 }
trait :team do
plan { "team" }
seats { 12 }
end
end
end
# spec/models/account_spec.rb
require "rails_helper"
RSpec.describe Account do
it "rejects a second account with the same email" do
existing = create(:account)
expect(build(:account, email: existing.email)).not_to be_valid
end
it "allows a team account at the seat limit" do
expect(build(:account, :team, seats: 50)).to be_valid
end
end
The spec states the one attribute it cares about right where it's
used. create goes through ActiveRecord, so validations and callbacks
run as in production, while build and build_stubbed don't save the
record at all. There's no template, script or API key involved. In
a request spec, attributes_for(:account, seats: 51) makes a perfectly
good request body. When a spec needs two or three records with specific
attributes, factories are simpler than a fixture file.
Faker values are random unless you seed them. Faker draws from
Faker::Config.random, so set it in a hook, using RSpec's own seed:
# spec/rails_helper.rb, inside RSpec.configure
config.include FactoryBot::Syntax::Methods
config.before do
Faker::Config.random = Random.new(config.seed)
end
With config.order = :random enabled (it's commented out in the
generated spec_helper.rb), RSpec prints the seed, and
rspec --seed 4242 replays both the example order and every Faker
value. Resetting the generator before every example means each example
draws the same values whether it runs alone or in the full suite, so
rerunning one failure by id reproduces it.
A generated JSON fixture is worth the extra machinery when the list of
cases is the thing you want to review. Each row and its expected
outcome sit in one diffable file, instead of being spread across it
blocks. It also suits rows that depend on each other, like the
duplicate email that has to match the existing account, and data that
other tools need too, such as a frontend's mock server or a Postman
collection. The two mix well: factories for the records a model spec
builds, and the fixture for the table of requests an endpoint must
accept or reject.
FAQ
How do I load a JSON fixture file in RSpec?
In rspec-rails, put the file under spec/fixtures/files and call
file_fixture("signups.json").read inside an example, a let or a
before hook, then parse it with
JSON.parse(text, symbolize_names: true). The directory is the
file_fixture_path setting, which defaults to spec/fixtures/files.
Outside Rails, read the file by path with File.read.
Can I use file_fixture in a describe block?
No. file_fixture is an instance helper, so it only works inside
examples and constructs that run in an example's scope, such as
before and let. Calling it directly in a describe or context
block raises an error when the spec file loads. To generate examples
from a file, read it by path at the top of the spec file, for example
with Rails.root.join("spec/fixtures/files/signups.json").read.
How do I generate RSpec examples from an array of test cases?
Loop over the array in the describe block and call it once per
item, with a description built from the item, such as its id and the
expected result. Each item becomes its own example with its own pass
or fail line. All generated examples share one line number, so rerun a
single one by its example id, like
"spec/requests/accounts_spec.rb[1:6]" (quoted, so zsh doesn't treat
the brackets as a glob), or by name with -e.
How do I make Faker data reproducible in RSpec?
Set Faker::Config.random = Random.new(seed) before each example, for
example in a config.before hook in rails_helper.rb. Using RSpec's
own config.seed as the seed means that, with config.order = :random
enabled (it's commented out in the generated spec_helper.rb), RSpec
prints the seed and rspec --seed 4242 replays both the example order
and the Faker values. Resetting the generator before every example
means each example draws the same values whether it runs alone or in
the full suite, so rerunning one failure by id reproduces it.
Should I use FactoryBot or fixtures in RSpec?
Use FactoryBot when each example builds a few records with specific attributes and you want them created through ActiveRecord, with validations and callbacks running. Use a fixture file when many examples share one data set, when the cases should be reviewable as a table in a pull request, or when other tools need the same data. Many suites use both.
Is there a JsonFabrica gem or plugin for RSpec or Rails?
No. JsonFabrica is a plain JSON-over-HTTP API, with no gem, Rails
plugin, SDK or database connector. Write a small regen script, in Ruby
with Net::HTTP or with curl, that sends a template and a seed to the
API and writes the generated JSON into your repository. RSpec reads
that committed file, and test runs never call the API.
Generated RSpec test data gives you one committed file of request
cases, with planted invalid rows and the outcome each one should
produce, rebuilt from a single seed. Rendering a raw template with a
seed and params is one request to the JsonFabrica API; the
templates API reference documents the
request body and the { data, meta } response.
Generate realistic test data with JsonFabrica
Describe the shape of your data once, then generate as many fresh, realistic JSON documents as you need via a simple API call.