Drupal.org Testing Infrastructure

The Project Dependency module is deployed and enabled on drupal.org, with initial dependencies built! Now we have to get to the next step, deploying the PIFT changes that go with it.

IRC: #drupal-testing

• Our key #1 goal for the day is to get dependencies stable and deployed, see Project Dependency.
  • Fix additional project dependency issues with memory limit.

I'd love to do a code sprint on testing Friday at Drupalcon London.

The most important outcome would be deployment of Project Dependency and the related PIFT, PIFR, and Project module changes.

What we need to do to accomplish this:

1. Set up a test environment (PIFT, QA.scratch + testbot) demonstrating dependencies working.
2. Get the new PIFR version deployed, assuming it can be done without a PIFT change.
3. Get the deploy issue RTBC'd and perhaps deployed.

I wanted to point possible contributors to a new low-level testing infrastructure doc page, http://drupal.org/automated-testing - Although this is incomplete and probably incorrect, it's a starting place for documenting our testing infrastructure. I invite you to improve it.

This wiki page has been set up to continue the 'Automated Coder' discussion initially started at http://groups.drupal.org/node/155084#comment-517764.

I want to make sure that everybody in these groups knows about the fantastic simpletest + selenium integration work that ygerasimov has been doing. I just tested out his Selenium module (7.x) and it worked fine for me.

Here's what I did:

1. Install Selenium module and start Selenium Server as described in the INSTALL.txt.
2. Installed the Selenium Test Example sandbox for sample tests.

I posted about the testbots being on the OSUOSL Supercell over at http://randyfay.com/node/99.

-Randy

One of the problems we've always had that delays deployment of PIFR and PIFT code is that it has been hard to test. But now we have a workable test environment, consisting of three parts:

• A drupal.org clone, http://rfay.redesign.devdrupal.org, housed on stagingvm.drupal.org. (This is one of the new tricked out development sites that are now available to people who want to improve drupal.org)
• http://qa.scratch.drupal.org, housed on stagingvm.
• Testbots as needed can be spun up either as local VMs and configured to pair with qa.

I was trying to gauge what the best setup for a PIFR node is, and I noticed that this page (http://qa.drupal.org/node/62) seemed way out of date, so I ran some tests of my own. Here I was trying to figure out a couple of things:
- Which concurrency setting is best?
- How does performance improve with number of cores

For testing, I used two separate PIFR nodes running in KVM virtual machines with the following specs:

CPUs are Opteron 6172, 2.1 Ghz
2 GB RAM
Local disk (10K SAS)
Running Debian Squeeze 64-bit, installed PIFR using DamZ's puppet setup

My results were as follows:

We have a test environment available now to test the new code that is ready to be deployed and solves the problems with 6.x core patches and a number of contrib problems. It does not solve the problem with loading contrib dependencies. That's our next task.

The drupal.org copy is at http://rfay.redesign.devdrupal.org. Follow the instructions on the basic auth popup to get in (drupal/drupal).

You'll need a password to get in there unfortunately (to log into the drupal.org copy itself). Please contact me on IRC or via the contact page and I'll give you your account password.

boombatower, DamZ, and I talked tonight about how to catch up to HEAD/master on PIFT and PIFR.

boombatower says there are protocol changes that have been implemented on the PIFR side but not PIFT, and probably HEAD of each won't talk to each other.

The plan:

1. rfay will create an experimental branch and try to merge all the 6.x-3.x work (git migration, since 6.x-2.2) into the former HEAD (master).

I noticed that issues with queued patches were not being updated with test results, so naturally, I visited qa.drupal.org to see what was the problem.

Not only is the statistics page (still) returning a 500 server error, but the most recent "News" item is over a year old.

I'm planning to sprint Friday on PIFR/PIFT (our d.o testing infrastructure) and would love to have everybody involved (or who wants to be involved) in there.

Here are things I think we could do:

1. Merge boombatower's work on HEAD with the 3.x work done for the Git migration.
2. Make testing work for the 8.x branch

The steps involved in taking forward the Selenium test for Drupal.org initiative:

Link to drupal.org issue queue: http://drupal.org/node/991040

• Identifying a groups of people interested in building Selenium test-cases
• Working on Identifying critical tests
• Writing tests
• Working with the infrastructure team to run these tests

If you are interested in building Selenium tests and supporting the same, please add you name and the area of support.

Implementers:

We are currently seeking students interested in learning about our AWS testing implementations. The primary objective is to create a curriculum that will help educate others understand the methods utilized by the Drupal infrastructure to support it's automated testing in the Amazon cloud. This is a prime opportunity for you to become involved in Drupal and help create positive learning materials to help others understand the benefits of Amazon cloud services as they apply to Drupal.

http://aws.amazon.com/free/

Could this be useful for testbot instances?

I relaunched qa.scratch.drupal.org, which had been idle for awhile and had gotten stale in its code, but it works now. We can use it to test out new environment or other testing scenarios without running the risk of messing up qa.d.o.

There may already be a list going on this, but I didn't fine one in the issue queue. What we list here can be turned into issues, but I think we need a higher-level discussion first.

• Notes/description for each testing client on qa.drupal.org
• Notification if a PIFR test client is not contacted within 24 hours (when they just disappear and stop checking in)
• Abiliity to mark an environment as "advisory"
• Ability to test on multiple environments and require all to pass
• Fix queue length discrepancy
• Show list of deferred queues