Preparing test data for Automated Testing

Question

So i've ran into a bit of a problem, I have a set of automated tests created for a site, were looking into really moving towards a CI based system for testing (Running the test suite after pushing features/etc...)

However in this site and a lot of my automated tests involve creating records in the database (through the site). Which is fine and everything, I mean these are features I WANT to exercise.

However....I can't just keep running the tests on the same database, the test data would just keep filling up! I could Delete the records from the database using the Websites ability to delete things, but that's insanely brittle and relies me going to an exact "Delete" button on a list to do that....seems like a poor idea. Plus if that automated test doesn't work, we get extra data...not good.

So how do I get around this?

Most of our sites are rails sites, i've looked into using Factories, but that doesn't really seem like it replicates doing a true integration test. So I could just recreate a database and reseed with test data before every automation suite run, or is this bad practice?

How do I get a good system running for test data in this scenario + moving forward?

• Niels van Reijmersdal: Have a look at my strategy I answered in another question: sqa.stackexchange.com/a/12225/3201
• Michael Durrant: The main concept to get used to is having an empty database before EACH individual test. Only create the reference data you need for that test and delete it after that test. For examples for zip code lookup we have 20,000 records but for testing we need ONE - 90210 - so we just create that one record for the test and delete it afterwards. This approach with no seed data is a big change from the past. Some folks need to see it to really believe it. It came about 'cos we learned that all the other approaches don't scale or maintain well.

Answer 1

Think about the three steps for every test

• setup
• execute
• teardown

Ideally this is done for every test and the database strategy used between tests is truncation.

Frequently this is deemed not possible and seed data is used which would slow down the tests too much if deleted each time. In those cases each test should use the database strategy of rolling back the transactions it create.

For rails look at this section in spec/rails_helper.rb
The following is an example from our environment.

config.prepend_before(:each) do |example|
 if truncate?(example)
 DatabaseCleaner[:active_record].strategy = :truncation, {pre_count: true, reset_ids: false}
 else
 DatabaseCleaner[:active_record].strategy = :transaction
 end 
 DatabaseCleaner.start
end 

• Mercfh: I guess how would I truncate between tests? Im not familiar with anything that does that
• Michael Durrant: I added the relevant section from spec/rails_helper.rb that deals with the strategies.
• Mercfh: I tried database cleaner, use the capybara database_cleaner.rb file found all over the web....and while I don't get any errors (i've included require database_cleaner) in my test_helper file....it just doesn't seem to remove the record. Is there any way for it to show if it's working at all log-wise?