Showing posts with label phpunit. Show all posts
Showing posts with label phpunit. Show all posts

Sunday, November 13, 2011

Managing multiple job configurations for Jenkins

If you are in the same boat as I am, you find you have too many packages to look after with Jenkins.

The beauty of Jenkins is the simplicity at setting up a job with the web frontend - but once you get over a certain level of complexity this is actually one of the bigger drawbacks.

Sure, we've got some templates, but how far can you really stretch it?

In my situation, I need to:

  1. Trawl SVN/other version control for all packages available - several hundred
  2. Only if the package has tests, add an entry to the CI suite
  3. Adapt to packages which require E_ALL & ~E_STRICT to run happily under that
  4. Packages which require dependencies, but can't be installed, still need a mechanism to install said dependencies
  5. And some which need to be invoked with the legacy AllTests.php
  6. Detect when a package has migrated to github
  7. ... and update an existing build/job with a new tool when required
I had tackled part 1 with pear's "packages-all" SVN link, which pointed to the trunk branches of all relevant code, and written some scripts for cruisecontrol to find all directories with a /tests/, but I find myself in need of something more.

So, my code is on github for now, and you can see the current CI system where those scripts have installed new jobs.

I'm quite sure that pyrus and a local installation will deal with the dependencies; as they are all described with PEAR's package.xml format. Also; detecting when a package has shifted to github should be fairly easy to tackle, as there is much work underway to deal with migration.

The one area I need to explore is manipulating jenkins jobs via xpath, to understand what parts of a job are already present and what need updating - basically number seven in the above list.

I'm curious who's done this sort of thing before, regardless of language, and if there are any libraries which make it easier to do this sort of thing.


Tuesday, February 10, 2009

PEAR bug day roundup - Feb 7th 2009

Here's a quick list of things done at/around the last bug triage day.

Accomplishments:
* Triaged the latest 50 bugs - doconnor
* Knocked off parse error related bugs - doconnor
* Updated unit tests to PHPUnit 3 for I18Nv2 - doconnor
* More unit tests fixed in PEAR 1.8 - dufuz
* Added Image_JpegXmpReader into CVS - doconnor
* Added Validate_HU into CVS, marked as unmaintained, removed 2x releases - doconnor
* Math_Finance got added to CVS - doconnor
* Validate got a new release - amir, davidc
* HTML_Page2 got a new release - doconnor
* Crypt_Rc4 bug fixes - kguest
* pearweb password bug - cweiske
* pearweb deployment and regression - cweiske / dufuz / doconnor
Related articles by Zemanta


Tuesday, July 29, 2008

PHPUnit Custom Assertations

Raphael has a nice quick article on custom assertions in PHPUnit.

Worth a read, it's blindingly obvious and simple, but the most clever things always are.

Even better, there's a link to xunitpatterns.com.

Monday, December 03, 2007

Unit testing your email...

Problem: I have code which uses PEAR's mail class to send email.

Every now and then, someone breaks something rather important when doing a release. Additionally, the continuously running unit tests keep sending me email. What shall I do?

The answer, most of you are probably crying, is use a stub/mock class.

Take a look at this suggestion.

You could provide a method in the test which checks sent mail against an array of expected mail, or simply fetch the number of sent emails and check its correct.

It's come in handy in a few places on my work project; have a play and see if it comes in handy for you.

Update: this will be available in the next release.

Wednesday, July 04, 2007

Making Useful PHPUnit results

So you have about eleven billion test results, and you need a pretty way to quickly see failures.
What you need is a continous integration environment, and if you don't want to have to stick things into a database, I have the answer for you.

PHPUnit 3.0.6 provides (the first working) version of test-results as JSON.
Basically, you run:
phpunit --log-json results.js MyTest.php


And out comes JSON.

The next step is pretty simple - you just need a bit of Prototype magic, some css, and some love.
What you end up with is:
Our Tests

Here's the javascript, php, and css behind it. I'll leave it up to you to find nice icons.

<?php
$file = dirname(__FILE__) . '/results.js';
if (!file_exists($file)) {
 die("No unit tests have been run?");
}
$json = file_get_contents($file);

//Ugh
$json = '[' . str_replace("}{", "},{", $json) . ']';

?>
<html>
<head>
<script src="/prototype.js"></script>
</head>
<body>
<style>
table {
 border: 1px solid rgb(240, 240, 240);
}

.error {
 border: 1px solid orange;
}

.fail {
 border: 1px solid red;
}

.pass {
 border: 1px solid green;
}

.cell {
 width: 200px !important;
 text-align: center;
 padding: 0.5em;
}
.cell img {
 display: block;
 margin: auto;
}

h4 {
 font-weight: normal !important;
 font-style: italic;
}
</style>
<script type="text/javascript">
var total_passes = 0;
var total_failures = 0;
var total_errors = 0;
var total_skipped = 0;
var total_incomplete= 0;

function renderSuiteStart(item) {
 var h2 = document.createElement("H2");
 h2.appendChild(document.createTextNode(item.suite));

 var p = document.createElement("p");
 p.appendChild(document.createTextNode(item.tests + " tests in suite"));

 var div = document.createElement("div");
 var table = document.createElement("table");

 div.id = item.suite;
 table.id = item.suite + "_table";
 table.className = '';

 div.appendChild(h2);
 div.appendChild(p);
 div.appendChild(table);

 $('main').appendChild(div);
}

function renderTest(item) {
 var h3 = document.createElement("H3");
 var h4 = document.createElement("H4");
 h3.appendChild(document.createTextNode(item.test));

 var p = document.createElement("p");
 p.appendChild(document.createTextNode(item.time + " tests in suite"));

 var tr = document.createElement("TR");
 var th = document.createElement("TH");
 var message = document.createElement("TD");
 var detail = document.createElement("TD");
 var time = document.createElement("TD");
 var img = document.createElement("IMG");
 var backtrace = document.createElement("OL");


 time.appendChild(document.createTextNode(item.time));


 img.alt = item.status;
 message.className = item.status + " cell";

 switch (item.status) {
 case 'error':
 if (item.message == "Skipped Test") {
 message.className = "skipped cell";
 total_skipped++;
 } else if (item.message == "Incomplete Test") {
 message.className = "incomplete cell";
 total_incomplete++;
 } else {
 total_errors++;
 }
 img.src = '/presentation/common/img/emblem-important-small.gif';
 break;
 case 'fail':
 total_failures++;
 img.src = '/presentation/common/img/cross.gif';
 break;
 default:
 total_passes++;
 img.src = '/presentation/common/img/check.gif';
 break;
 }

 item.trace.each(
 function(trace) {
 var li = document.createElement("LI");
 var text = trace.file + "(line " + trace.line + "): " + trace.class + trace.type + trace.function + "()";
 li.appendChild(document.createTextNode(text));

 backtrace.appendChild(li);
 }
 );

 h4.appendChild(document.createTextNode(item.test));
 detail.appendChild(h4);
 detail.appendChild(backtrace);

 message.appendChild(img);
 message.appendChild(document.createTextNode(item.message));
 message.style.width = "300px";

 tr.appendChild(message);
 tr.appendChild(detail);


 $(item.suite + "_table").appendChild(tr);
}

function renderResults() {
 var data = $A(<?php print $json; ?>);

 data.each(function(item) {
 switch (item.event) {
 case 'suiteStart':
 renderSuiteStart(item);
 break;
 case 'test':
 renderTest(item);
 }
 });

 renderTOC();
}

function renderTOC() {
 var fail = $A(document.getElementsByClassName('fail'));
 var pass = $A(document.getElementsByClassName('pass'));
 var error= $A(document.getElementsByClassName('error'));
 var skipped= $A(document.getElementsByClassName('skipped'));
 var incomplete= $A(document.getElementsByClassName('incomplete'));

 var menu = document.createElement('p');
 var checkbox_pass = document.createElement('input');
 var checkbox_fail = document.createElement('input');
 var checkbox_error = document.createElement('input');
 var checkbox_skipped = document.createElement('input');
 var checkbox_incomplete = document.createElement('input');

 var label_pass = document.createElement('label');
 var label_fail = document.createElement('label');
 var label_error = document.createElement('label');
 var label_skipped = document.createElement('label');
 var label_incomplete = document.createElement('label');
 var status_image;

 menu.style.position = 'fixed';
 menu.style.bottom = 0;
 menu.style.right = "1.5em";
 menu.style.backgroundColor = 'rgb(240, 240, 240)';
 menu.style.padding = "1em;";
 menu.style.fontWeight = 'bolder';

 checkbox_pass.type = "checkbox";
 checkbox_fail.type = "checkbox";
 checkbox_error.type = "checkbox";
 checkbox_skipped.type = "checkbox";
 checkbox_incomplete.type = "checkbox";

 checkbox_pass.checked = "checked";
 checkbox_fail.checked = "checked";
 checkbox_error.checked = "checked";
 checkbox_skipped.checked = "checked";
 checkbox_incomplete.checked = "checked";


 checkbox_pass.onclick=function(){toggle(pass)};
 checkbox_fail.onclick=function(){toggle(fail)};
 checkbox_error.onclick=function(){toggle(error)};
 checkbox_skipped.onclick=function(){toggle(skipped)};
 checkbox_incomplete.onclick=function(){toggle(incomplete)};

 label_pass.appendChild(checkbox_pass);
 label_fail.appendChild(checkbox_fail);
 label_error.appendChild(checkbox_error);
 label_skipped.appendChild(checkbox_skipped);
 label_incomplete.appendChild(checkbox_incomplete);

 label_pass.appendChild(document.createTextNode("Passed (" + total_passes + ")"));
 label_fail.appendChild(document.createTextNode("Failed (" + total_failures + ")"));
 label_error.appendChild(document.createTextNode("Error (" + total_errors + ")"));
 label_skipped.appendChild(document.createTextNode("Skipped (" + total_skipped + ")"));
 label_incomplete.appendChild(document.createTextNode("Incomplete (" + total_incomplete + ")"));

 label_pass.style.color = "green";
 label_fail.style.color = "red";
 label_error.style.color = "#FF6600";
 label_skipped.style.color = "blue";
 label_incomplete.style.color = "gray";

 status_image = document.createElement("IMG");
 status_image.style.display = "block";
 status_image.style.margin = "auto";
 if (total_failures > 0) {
 status_image.src = "/presentation/common/img/emblem-unreadable.png";
 } else {
 status_image.src = "/presentation/common/img/compliance.png";
 }
 menu.appendChild(status_image);

 menu.appendChild(label_pass);
 menu.appendChild(label_fail);
 menu.appendChild(label_error);
 menu.appendChild(label_skipped);
 menu.appendChild(label_incomplete);

 checkbox_pass.onclick();
 checkbox_pass.checked = "";

 checkbox_skipped.onclick();
 checkbox_skipped.checked = "";

 checkbox_incomplete.onclick();
 checkbox_incomplete.checked = "";

 document.body.appendChild(menu);
}

function toggle(type) {
 type.each(function(td) {
 if (td.parentNode.style.display != 'none') {
 td.parentNode.style.display = 'none';
 } else {
 td.parentNode.style.display = 'table-row';
 }
 });
}

function hide(type) {
 type.each(function(td) {
 td.parentNode.style.display = 'none';
 });
}

function show(type) {
 type.each(function(td) {
 td.parentNode.style.display = 'table-row';
 });
}

window.onload = renderResults;
</script>
<div id="main" class="contentbox">
<h1>Unit tests</h1>
<p><a href="results.txt">Raw results</a> | <a href="docs.html">Agile Documentation</a> | <a href="recent.php">Recent changes</a> | Generated: <?php print date("F j, Y, g:i a", filemtime($file)); ?> </p>
</div>
</body>
</html>

Saturday, May 26, 2007

assertSame() & phpunit

Take a look at enhancing assertSame() error messages for phpunit.

This is one of my most frequent gotchas, usually happening about once a fortnight. It drives me insane.

Basically, I do:

$expected = "Catfish";
$actual = SomeComplicatedConvolutedClass::method(); //returns "Catfish"
$this->assertSame($excepted, $actual);


and get an error message telling me that null isn't the same as Catfish. That's perfectly good, except I haven't clicked that the bug is my typo, not in SomeComplicatedConvolutedClass::method()

What a pain!
Subscribe to: Comments (Atom)

AltStyle によって変換されたページ (->オリジナル) /