Sequence of Integration Test in Build Process

Question

We are developing a Web API (ASP.net, C#) project and use Bamboo CI for our continuous Integration. We run the Unit test before the deployment task so that deployment does not happen if test fails. We deploy to a development server. My questions are

1. When should I run Integration test in build process before or after deployment?
2. Should it depend on Real DB or use In memory option in Entity Framework?
3. Should I self host web api using owin or hit the hosted web api in the dev environment?

Answer 1

Test as much as possible before deployment

Deployment isn't cheap. Even if the deployment is fully automated and pushed automatically from the commit it still takes time to deploy, it prevents any use of the development environment while deploying and if you deploy a broken application the development environment is stuck until things get fixed.

This depends on the kind of integration tests you have, obviously. A standalone component (for example, a message parser that stores things in a database) does not have to be deployed for integration tests to run. If you are doing UI-tests you obviously DO need to deploy to some environment to run those tests.

Run as much tests as you can during the build, and only deploy to run those tests that need the real system deployed to be able to run.

In-memory databases are great for unittests, not so much for integration tests

The advantage of in-memory databases is that they can be set up and torn down quickly and have a high rate of turnaround in your tests. A lot of people favor testing things like ORM-mapping and repositories with queries on in-memory databases and run integration tests on actual databases. I think that's a very sensible approach, but it's a very personal thing. If your tests are fast, running on an actual server is better (because the closer your tests are to the actual production environment, the better they are).

• Ajay Bhasy: As part of this build only the web api is tested, so no need of UI tests. If I opt to test with the development server, any changes in business logic will need to be deployed first, for me to test it against the server. Won't it be? So to follow your approach of Test before deployment shouldn't I be using self hosting and in memory DB? Thanks