Try Assert Exceptions not failing test in python unittest or better way to loop through failures

Question

Okay I am running Chromedriver test that gathers console logs and then tests an assertion against pythons unittest.

class ConsoleLogsTest(unittest.TestCase):
 @classmethod
 def setUpClass(cls):
 cls.driver = get_browser_driver()
 cls.base = BasePage(cls.driver, BASE_URL)
 cls.page = MultiplePages(cls.driver, BASE_URL)
 cls.verificationErrors = []
 @classmethod
 def tearDownClass(cls):
 cls.driver.quit()
 def test_logs_for_error(self):
 self.page.navigate()
 self.base.wait_for_footer()
 self.verificationErrors = self.page.check_for_severe_console_logs(
 BASE_URL)
 try:
 assert len(self.verificationErrors) == 0
 except AssertionError, e:
 for message in self.verificationErrors:
 print str(message)

check_for_severe_console_logs returns a list of the entries from self.driver.get_log('browser') and I'm trying to loop through my list all the errors (which includes the URI and the console error) as a failure to the test instead of the actual exception # does not equal 0 because that's not very useful. If I run test with nose and include the sdout (e.g. nosetests -verbosity=1 -s) you it will print out my error message for every log. But if you run it without nosetests -verbosity=1 the test isn't actually failing it just says "Ok". I tried adding self.fail after the message loop but that didn't fix it. What am I doing wrong or know a better way to get useful errors?

---

Update: I figured out one way to do this is to use a custom exception so adding that solution here. But will probably go with @alecxe suggestion self.assertEqual(self.verificationErrors, []) unless somebody has something better or how to do that without self. I added a custom exception at the top of the file

class ConsoleException(Exception):
 def __init__(self, messages):
 self.value = ''
 for message in messages:
 self.value += '\r\n' + str(message)
 def __str__(self):
 return str(self.value)

Note that I has to use str instead of repr as suggested in the docs (and I'm worried that might cause problems if it finds lots) to get it to format how I wanted. And then I'm calling it like this:

try:
 assert len(self.verificationErrors) == 0
except AssertionError:
 raise ConsoleException(self.verificationErrors)

Update 2:

If you want this to fail as an assertion error, you could also just make function . to loop through errors.

def loop_thru_messages(messages):
 value = ''
 for message in messages:
 value += '\r\n' + str(message)
 return str(value)

Then run a function at the end of your assert.

assert len(self.verificationErrors) == 0, loop_thru_messages(self.verificationErrors)

• ernie: Why are you trying to match the style of using the built in python assert instead of the assertion methods in your test library (sounds like you're using nose)?
• Cynic: I think using just assert means I'm using Nose's assert and not Python unittest which uses self.assertTrue. I was trying to match somebody else's style and a project I did not setup.
• ernie: Using assert will use the built in python assert statement (linked above, but pasted here again to be explicit: docs.python.org/2/reference/…). Nose has it's own assert methods, and if you wanted to use those, you'd have to call them explicitly, something like nose.tools.assert_equal() (note the PEP8 convention as opposed to the mixed case that pyunit uses). Again, I'd highly suggest figuring out why you're trying to switch to using assert over using assert_* methods in your test library.

Answer 1

I think you just need to re-raise the assertion error after printing out log messages:

try:
 self.assertEqual(len(self.verificationErrors), 0) # no errors
except AssertionError as e:
 for message in self.verificationErrors:
 print(str(message))
 raise # < HERE

Also, if you want to check for severe console messages after every single test, see if it would make sense to do it in a "tear down" function of your test case.

Or, you may also assert self.verificationErrors is an empty list. In case of failure, you would get to see all the messages on a console:

self.assertEqual(self.verificationErrors, []) 

• Cynic: So that does raise the error. But my question is partially of how do I add the console errors to the exception error instead of them just being part of the stdout. Now that you've pointed me in the correct direction it does look like this is possible docs.python.org/2/tutorial/errors.html#user-defined-exceptions. I'm gonna keep going in that direction but if you already know how can you edit you answer?
• alecxe: @Cynic wait, why don't you want to go with a simple self.assertEqual(self.verificationErrors, []) then? Thanks.
• Cynic: I was actually doing it like that but was trying to match a devs style of not using self. before assertions and assert self.verificationErrors == [] doesn't error like self.assertEqual. I got it add it to the exception using a custom exception I'll edit that into my question as an option. But do you know if there is there a way to do it in that style?
• alecxe: @Cynic okay. To be honest, I don't really understand why would you not use unittest.TestCase's built-in assertions, but, have you tried to use pytest as your test runner - you might get a better and more verbose output just because of switching to pytest - not to mention it's configuration options and a huge base of plugins. Thanks.
• Cynic: So kind of funny, if you run assert self.verificationErrors == [] type assertion with the verbose flag pytest -v , it will get me a fail that also contains the log messages. Nose on the other hand just gives you an empty AssertionError: , (but will include the stdout if you leave in the print messages and pass the -s flag). I think the only way to make Nose to log out the errors as part of the failure is the custom assertion i added to the question. Going to mark this as answered as your answer of using unittest assertion or pytest runner instead is maybe a better fix.Thanks for help