how to fully unit test functions and their internal validation

Question

I am just now getting into formal unit testing and have come across an issue in testing separate internal parts of functions.

I have created a base class of data manipulation (i.e.- moving files, chmodding file, etc) and in moveFile() I have multiple levels of validation to pinpoint when a moveFile() fails (i.e.- source file not readable, destination not writeable).

I can't seem to figure out how to force a couple particular validations to fail while not tripping the previous validations.

Example:

I want the copying of a file to fail, but by the time I've gotten to the actual copying, I've checked for everything that can go wrong before copying.

Code Snippit: (Bad code on the fifth line...)

// if the change permissions is set, change the file permissions
if($chmod !== null)
{
 $mod_result = chmod($destination_directory.DIRECTORY_SEPARATOR.$new_filename, $chmod);
 if($mod_result === false || $source_directory.DIRECTORY_SEPARATOR.$source_filename == '/home/k...../file_chmod_failed.qif')
 {
 DataMan::logRawMessage('File permissions update failed on moveFile [ERR0009] - ['.$destination_directory.DIRECTORY_SEPARATOR.$new_filename.' - '.$chmod.']', sfLogger::ALERT);
 return array('success' => false, 'type' => 'Internal Server Error [ERR0009]');
 }
}

So how do I simulate the copy failing. My stop-gap measure was to perform a validation on the filename being copied and if it's absolute path matched my testing file, force the failure. I know this is very bad to put testing code into the actual code that will be used to run on the production server but I'm not sure how else to do it.

Note: I am on PHP 5.2, symfony, using lime_test().

EDIT I am testing the chmodding and ensuring that the array('success' => false, 'type' => ..) is returned

• Job: Mocking comes to mind. If I understood this correctly, you somehow want the ability to change the implementation of `logRawMessage' or whatever other function you are testing to a predictable fail right before you run it. Some languages / frameworks are friendlier than others (for instance in Python you can change things on the fly), while in others you need to add extra code for being able to substitute a different implementation. Perhaps while you test, you should always call a wrapper method, not the method under test directly. Then you instruct a failing behavior right before you test it.
• Job: (Continued) ... for some tests, of course, you would be able to manipulate the file system / files directly and cause the error conditions (if your error checking logic is not tight enough). My concern is this: you currently do NOTHING when something fails. Should not you at least log that somewhere?
• Patrick: I wasn't clear in that I am testing the return from the function, not the logRawMessage. About manipulating the files directly to cause the error, all the validation happens in the same function immediately before the attempted chmod so I don't believe I can do that. I hadn't thought of the wrapper function before that's just for testing. I do log the information when the function fails, that's what the logRawMessage is doing.
• Job: @Patrick, I do not believe in mandates such as "Every function that you write/every bug that you fix must have a corresponding unit test". This starts to sound like theoretical TDD, something that few people actually attempt. Your tests do not have to be limited to unit-level. As long as you have a bunch of tests which exercise this function, you should be ok. The truth is ... chances are that after running this function manually a few times, you will have fixed 99.9% of all bugs that it could cause. Even if you have to use a debugger to pinpoint a problem, as long as you have many good tests
• Job: And ... if you are not happy with the coverage, then you might need to break an already small function into yet smaller parts. This is what some academics advocate, but I do not. Testability is just one factor; time to market and making $$$ is another important one. For inspiration on how to test this hard low-level stuff, you might want to look at how Linux Kernel is being tested :) or what unit tests get shipped with Python/Perl/Ruby/etc IO module. Some tests are best done manually - e.g. make changes to the file system while in the middle of debugging, or temporarily remove safeguard code.

Answer 1

So how do I simulate the copy failing.

You mock the chmod() function so that it can return error status.

• Patrick: how would I go about mocking that in PHP? would it just be making a wrapper function to call instead of chmod()?
• S.Lott: @Patrick. Essentially, yes. You'd be happiest having your own 'chmod' function that has several implementations. One implementation calls the system function directly. The other throws errors for testing purposes. The idea is that your application code doesn't change, only the choice of which chmod to use changes.

Answer 2

Hmm. It's tricky to get PHP's chmod() to fail. I did find one way, though not sure if it helps on your case. I assume an Unix environment, not sure how it would work on Windows.

chmod() will fail on "too many symbolic links", so try to set up a situation where you have a symbolic link pointing to itself. Something like:

ln -s foo.file foo.file

and now PHP's chmod("foo.file") will fail with

Warning: chmod(): Too many levels of symbolic links

So, if the foo.file above passes your other tests, it should fail on chmod().

• Patrick: thanks for the tip, I was at a loss for how to 'break' chmod(). I am on linux