Magento 2: How long should Unit, Integration and Framework Tests take to run on an unmodified site;

Question

Magento 2: How long should Unit, Integration and Framework Tests take to run on an unmodified site.

I am experiencing 10 - 15 minutes for PHPUnit tests

Integration tests run for hours and hours... I gave up at 7 hours in.

Haven't tried framework tests.

I'm think that I have configuration issues with my Docker based system that I need to get a dev-ops person to investigate, but before I do I'd like to know how long a well configured system would take?

• Wayne Theisinger: For anyone interested in why my docker set-up was so slow - it was IO I'd setup my test database on the same db container as my dev db which used a volume, I've an old fashioned spinning disk rather than solid state which didn't help. Anyway I've rewritten my docker-compose to include a new container with tmpfs: /var/lib/mysql:rw so that the database is now held in memory rather than being written to disc - persistence not being an issue.

Answer 1

My laptop configuration:

• Ubuntu 16.04.1 LTS
• PHP 7.0.14 with Xdebug 2.4.0
• Magento2 CE develop branch

Unit tests:

Time: 1.91 minutes, Memory: 666.00MB
OK, but incomplete, skipped, or risky tests!
Tests: 12518, Assertions: 38189, Incomplete: 38, Skipped: 64.

Integration tests require more time (we have Bamboo builds, and all integration tests for CE and EE versions take about 25-30 min), on my laptop one integration test runs about 280-300 msec (depends on the test).

Functional tests - are heavier, one test runs about 2-3 min, depends on variations and scenarios (for example, place order flow requires more time because uses shipping carriers and payment solutions).

Answer 2

For reference it's helpful to look at the Travis builds.

At this time, this is the result for the latest commit in the develop branch:

https://travis-ci.org/magento/magento2/builds/203371592

travis screenshot

• unit tests 4 minutes on PHP 7.0, 9 minutes on PHP 5.6
• integration tests 46 minutes on PHP 7.0, 87 minutes on PHP 5.6

So your unit test execution time is still normal, but integration tests running for 7 hours needs investigation.