The Lie
“No, the problem isn’t you. Fixtures suck, and you were lied to.”
– Francis Hwang
That is correct. This is exactly what the Rails Core team is going for:

Poor John Locke, protector of the island, pushing a damn button.
I do agree with Francis on about everything else though: Fixtures do suck, and testing is really fucking hard. By the time you realize you’re getting into Fixture Hell, moving your monstrosity of an application off fixtures is a daunting task. My solution (and currently the ENTP way): Machinist.
The problem I’ve had with fixtures, is that having one large set means that tiny tweaks can break tests all over the place. Also, working with YAML makes me a sad monkey. Fixture Scenarios helped solve the first problem, but not the second.
I then worked on Model Stubbing. My idea was that I could introduce stubbing definitions (groups of sample data) with a rubyish syntax. You could use a single stubbing definition for the app, or create one for a specific test case (from scratch, or by inheriting from an existing definition). This flopped too. Despite my efforts, this too dragged a project into Model Stubbing Hell.
I’m not even going to get too much into the rspec mocking. When I see code that stubs find or save, I cringe. Though after talking with David a bit, this was more a case of something being lost in translation. You should be replacing your mocks with live code once it’s implemented. It’s a tool for ping-pong programming, not making your tests fast and fragile.
So, this bring us to Machinist. It doesn’t do anything magical, it just creates valid records based on a blueprint that you define:
Post.blueprint do
title "Example Post"
body "Lorem ipsum dolor sit amet"
end
Comment.blueprint do
post
author { Sham.name }
body "Lorem ipsum dolor sit amet"
end
The problem here, though, is that factories and manual record creations tend to slow down your tests greatly. I try to use before(:all) to help, but I definitely miss a lot of the work that fixtures go through to keep tests speedy.
What can Rails do to help the situation? Personally, I’d support the notion of dropping fixtures all together. But, as someone that hasn’t used fixtures since before foxy fixtures were introduced, I’m not at all bothered by having to ignore them. Though there is one solution:
Time Travel :) (linked for spoiler prevention)
Updated: Removed the season 5 spoiler… I figured any LOST junkie would’ve seen it already, but apparently some of you are more patient than I am.
Comments
Comments are closed.

by Pat Nakajima on 09 Feb 13:06
Does Machinist still only allow you to create records? That was my issue when I first looked at it. Most of your domain logic should be expressable and testable with unsaved records. I don’t like getting the database involved unless I absolutely need to.
by Ash on 09 Feb 13:32
One thing I have found useful in DataMapper is Sweatshop. You can generate random data for your fixtures. For testing, its nice because of things like this:
User.fixture {{ :username => (username = /\w+/.gen), :email => "#{username}@example.com", :password => (password = /\w+/.gen), :pasword_confirmation => password # The /\w+/.gen notation is part of the randexp gem: # http://github.com/benburkert/randexp/ }} before :all do @a_user = User.gen # endYou can also give the fixtures a tag, like
Person.fixture(:valid) {{ :first_name => %w(Michael Adam Guiseppe)[rand(3)], :last_name => %w(Smith Black White)[rand(3)], :email => "#{/\w{10}/.gen}@somedomain.info", :password_salt => (salt = /\w{20}/.gen), :password_hash => Digest::SHA1.hexdigest("#{salt}@--,-`--secret") }}So to use that one you’d need to call the Person.gen :valid
Random data for testing = yay, Maybe someone should look into something like this for AR
by Pat Nakajima on 09 Feb 13:35
Ash,
I like to use the Faker gem for random data: http://faker.rubyforge.org
by Ruck on 09 Feb 13:43
I use the faker gem with machinist. Dm-sweatshops model generation is interesting, but I’d prefer a pure ruby solution not tied to an ORM. I’m sure that wouldn’t be tough at all (and probably already available). Machinist just added a #plan method for returning unsaved records.
by Adam Meehan on 09 Feb 14:01
@Pat: Yes Machinist now allows Comment.make_unsaved, which also only initialises any association record without saving them.
There is also the new plan method eg. Post.plan. It generates the attributes hash with the values you explicitly set in the blueprint. Its for use in controller tests.
by Mike Moore on 09 Feb 14:37
First you said your Jon reference is the last Lost reference, but then reference the frozen wheel! First you lie about fixtures, and then you lie about the number of Lost references. How are we to trust you?
Second, we haven’t ever seen time travel fix anything in the past. If you did turn the frozen wheel you’d most likely jump ahead to Rails 4 when it merges with the MagLev VM and fixtures are replaced with an object database. Just sayin…
by Mike Moore on 09 Feb 14:38
Crap. s/Jon/Jin
by Paul Horsfall on 09 Feb 15:19
Nice article. I’m really interested in your comment about fragile specs that stub find and save. I agree with you, but the idea that this is a problem doesn’t seem to be a widely held opinion in my experience. I don’t think any of the documentation/articles I’ve read have suggested replacing the stubs once the code is implemented; is there any? (Sorry for bringing up the thing you don’t want to get into!)
by rick on 09 Feb 17:30
Mike: I uh also misspelled my name (damn iphone!).
And, I removed the lie about it being my last reference. The lie has been covered up. I just need that MIB flashy thing to use against google and everyone’s feed readers and the coverup is complete.
by Lawrence Pit on 09 Feb 18:19
technoweenie, while you conclude that your model_stubbing project flopped, we’ve been using it for many months in a fairly large project with rspec, and imho it’s still one of the best and unknown solutions out there.
Having said that, in a different project I actually opted for using Test::Unit/factorygirl/shoulda/Matchy/Pending combination (I just can’t stand the slowness of rspec anymore, and its codebase is just horribly unreadable cq too magical to my liking) and use Machinst’s sham with faker. Even though Machinist markets itself as having a better philosophy and a better syntax compared to factorygirl (by claiming the opposite about factorygirl), I actually find their arguments dubious: I don’t like Machinist including itself in my models for one thing, I don’t like any “Gotchas” (and Machinist has a few of them, they only mention one), and I find setting up an object graph in factorygirl slightly easier and actually syntactically better.
This is where model_stubbing shines imo. Within a test file, define your object graph once, have it set up in your database once, and then all the tests within that file will use that data. It’s as if you have fixtures.
I think that if you marry the simplicity of machinist/factorygirl with modelstubbing’s ability to set up a fixture-like situation (based on blueprints) exactly once for a group of tests would be the winner. Rails can help by providing a better API to make this possible (modelstubbings way was a bit hackish in that respect I felt, due to the given API I assumed back then).
by rick on 09 Feb 20:35
Some of the ideas behind Model Stubbing are nice. I think a big part of where its speed benefits come from is skipping validations and callbacks. Faster, but then you have to remember to manually create any dependent objects that your callbacks would automatically perform. Everything’s a tradeoff though… and you could enable validations and/or callbacks with model stubbing.
by grosser on 09 Feb 23:01
i use valid_attributes, one set of valid fixtures goes into a yaml file and is usable in all specs
http://github.com/grosser/valid_attributes/tree/master
by justin on 09 Feb 23:58
Use rails-test-serving to speed up machinist
by Gabe da Silveira on 10 Feb 00:08
I know everyone has loved to hate fixtures for at least 2 or 3 years now. Fixtures’ suckiness is well-established at this point. However it’s a bit disturbing to hear a core member talk about “dropping fixtures altogether”.
With all the development in testing methodologies, Ruby/Rails is a pretty cool place to be. There are a ton of sweet ideas floating around waiting to be cherry-picked for a project. Unfortunately there’s also a lot of hype, idealism, obsessive-compulsiveness over syntax minutiae. I think replacing fixtures is one of these areas where there are clearly better ways to do things a lot of the time, so people bend over backwards to get rid of them entirely, but end up creating a bigger mess in the process.
Replacing fixtures with factories works pretty well in unit tests where models tend to be more isolated. Writing out a little more model generation code here is an acceptable tradeoff for increased readability and maintainability. When it comes to functional tests, though, I haven’t seen an approach other than fixtures that I like.
The way I see it, fixtures should represent a base state for the application. There should be no more than 5-10 fixtures for each model, and they should represent the significant data variations that you expect in your application. For testing edge cases, some kind of factory helper is definitely the way to go, but with fixtures you have a great sanity check that extends to all your functional tests without a ton of boilerplate model construction/stubbing.
Sure fixtures are unmaintainable in large quantities, but you don’t need many to get a lot of test coverage. They’re easy to dislike because they aren’t conceptually pure. Rather they are a pragmatic way to simulate a live environment with low-overhead. When I think of the amount of factory code I’d have to write to replace fixtures in my functional tests, and how much agonizing I’d have to do about how to acceptably DRY it up, I just don’t see how it’s worth the trouble.
by Justin on 10 Feb 17:06
Gabe that’s what I do. Fixtures for functional, machinist for unit.
by Scott Taylor on 26 Feb 20:18
Rick,
What do you think Machinist does better than FixtureReplacement?
I was trying to show you the way over two years ago…