A Python Exception Mail Tracker
Tuesday, July 3rd 2018 - Reading time : About 10 minutes
I'd like to thank David Bôle which helped me develop this piece of code back at Quantic Dream and later.
As a technical director, I believe my job is to help artist thrive and excel in their work. With that aim, the end goals are speed and simplicity. As much as we can, we should take the burden of technical complexity to help them consistently deliver great work. As The Zen of Python states : Simple is better than complex.
For that I'd like to discuss a task that I think most artists loathe with good reasons : reporting bugs. If some bugs are legitimately tricky, and in need of a proper description with insights, some are so straight-forward that they shouldn't even need someone to be reported and that's what we're going to improve today - by receiving an email with all the code's detail when an exception occurs.
A traceback is good, a debugger's view is (much) better
The basic trick here is that every Python exception (or at least, every exception that's not caught)
goes through the same built-in function: sys.excepthook()
.
With some good ol' monkey-patch, we can execute our own snippet of code when an exception is raised.
And thanks to some Python introspection, we can produce a detailed log of basically anything we want at the time of the error.
For starters, here's what the Python documentation has to say about this function :

Note that the Python documentation tells you directly that you can customize this handling
by replacing this call ¯\_(ツ)_/¯. So, sys.excepthook()
takes three arguments :
the exception's type, its value and a traceback object.
Those three fit perfectly with the format_exception()
function, from the traceback
module.
This handy function returns the full traceback as a string :
import traceback def _get_traceback_string(exception_type, exception_value, traceback_): """Return the formatted traceback as a string.""" return '\n'.join( traceback.format_exception( exception_type, exception_value, traceback_ ) )
You might notice I'm using a traceback_
var ; that's because we're already
using the traceback
module. So to not overwrite it in our function, we're adding an underscore at the
end of it.
Now for the introspection part - retrieving code variables at the time of the error.
There's a great amount of information in the
inspect
module documentation.
We can find out that our traceback object has a few interesting properties, namely tb_next
, tb_frame
and our target for this part, f_locals
, which is described as the
"local namespace seen by this frame" - that is, the local variables.
def get_exception_variables(traceback_): """Return a dict of variables {name: value} from traceback_.""" # Get latest call stack frame (latest_frame is None with SyntaxErrors) latest_frame = traceback_ while latest_frame and latest_frame.tb_next: latest_frame = latest_frame.tb_next # No latest_frame means a SyntaxError which bears no vars if latest_frame: return latest_frame.tb_frame.f_locals return []
A fun thing I like to add is a code preview, to get a context of the code that failed. That's kind of a large part so I'll skip it for now - I'll probably make a second article for that.
We now have our formatted traceback and variable details ready to be mailed. For that, we enter the realm of HTML and inline CSS since a lot of mail clients don't support CSS styles (looking at you, Gmail). Sadly, that makes the code quite bulky so I'll try to keep it short, without a nice layout or colors. Although you might want to do it on your side, to get a better readability off your report.
Let's format our traceback for now. A basic thing to do is to preserve the traceback's indentation, which we can't do by using spaces since strings are stripped in most mail clients - we'll use CSS margins. We'll also use a monospaced font to preserve a "Python console" look.
def format_traceback(exception_type, exception_value, traceback_): """Format the call's traceback with proper indentation.""" # Get call stack text call_stack = get_traceback_string( exception_type, exception_value, traceback_ ) html = [] traceback_lines = call_stack.split("\n") for line in traceback_lines: # Keep only lines with content line = line.strip() if not line: continue # Apply offsets to preserve indentation, depending on the line if line == traceback_lines[0] or line == traceback_lines[-1]: html.append('<SPAN>{}</SPAN>'.format(line)) elif line.startswith('File "'): html.append('<SPAN style="margin-left: 15px">{}</SPAN>'.format(line)) else: html.append('<SPAN style="margin-left: 30px">{}</SPAN>'.format(line)) # Return traceback lines wrapped in a div with monospace font return '<DIV style="font-family: monospace;">\n{}\n</DIV>\n'.format( "<br />\n".join(html) )
You might have noticed some line breaks - the "\n" - added to the output. They're quite useless in HTML code but come in handy when your formatting breaks down and you need to debug it by looking at the result, you can trust me on that. Let's keep moving and format our variables :
def format_variables(traceback_): """Return a html table of the local variables names and values.""" # Build table with two columns: key/name & value table = '<TABLE>\n' # Add each variable to the table for key, value in sorted(_get_exception_variables(traceback_).items()): table += ' <TR>\n' table += ' <TD>{} :</TD>\n'.format(key) v = str(value).replace('<', '<') table += ' <TD>{}</TD>\n'.format(v) table += ' </TR>\n' table += '</TABLE>\n' return table
This one is pretty much straight-forward except for the str.replace()
on line 10. What happens here is that we're writing HTML content as we go, and HTML
uses the lesser than and greater than symbols in its syntax.
Some Python strings may be formatted as an HTML tag, like for instance when you use
the print function on a module:
>>> import sys >>> print sys "<module 'sys' (built-in)>"
This kind of string would get converted into an HTML tag (though its syntax is broken)
and not being displayed at all. To avoid this, we changed the first lesser than
sign to its HTML code equivalent, <
which displays it correctly without making it
a valid HTML tag.
We got everything we need by now. We'll just wrap all this in a big HTML div and send it.
Here's our final function that we'll use to replace the original sys.excepthook
:
def main(exception_type, exception_value, traceback_): """Python exception hook that mails exception reports.""" # Build mail message = "".join(( '<DIV>', '<H3>Traceback</H3>', format_traceback(exception_type, exception_value, traceback_), '<H3>Variables</H3>', format_variables(traceback_), '</DIV>' )) # Prepare the mail subject = "{}: {}".format(exception_type.__name__, exception_value) your_address = "you@users.com" user_address = "user@users.com" msg = MIMEMultipart('alternative') msg["Subject"] = subject msg["From"] = user_address msg["To"] = your_address msg.attach(MIMEText(mail_contents, 'html')) # Send the message - you'll need to enter your own server here server = smtplib.SMTP('smtp-server.client.com') server.sendmail(user_address, your_address, msg.as_string()) server.quit() # Run the basic python error to print the traceback to the console sys.__excepthook__(exception_type, exception_value, traceback_)
If you'd like to send an email with a Gmail address, I invite you to read this question on StackOverflow.
Our tracker is now complete! I've made a gist with
all of this packed in a class for convenience - with added enable()
and disable()
functions. Those use the sys.__excepthook__()
function which simply backs up
sys.excepthook()
, allowing us to easily reinstate the original hook or switch between the two at will.
If you write your own overwrite someday, just be sure not to overwrite that one.
A few tips if you wish to use this tracker in Maya : maya.utils.formatGuiException()
is called in place of sys.excepthook()
, and it takes an additional fourth keyword argument
("detail") that you'll have to add to your main function. It also necessitate that this function
returns a string, that will be passed to OpenMaya.MGlobal.displayError()
- you can just use
a return on the sys.__excepthook_ call.
Also, some errors occur quite a lot, the plugin not found error in Maya when opening a new file is one of them.
Remember to filter those out - I tend to use regular expressions on the traceback string to do so.
You should probably exclude SyntaxError
as well.
One final thing, there might be
a lot of variables in the list you will send via email - it can be useful to sort them out.
Once again, The inspect
module is your friend here if you want, for instance,
to separate modules from other types of variables using inspect.ismodule()
or
inspect.isclass()
...
Happy tracking !
Something appears to be missing from the format_traceback() function. There is an indent and an un-referenced variable 'line' as written. I initially thought that the line missing was:
for line in call_stack:
but then traceback_lines remains undefined.
Something is missing...
Hi Paul, well you're absolutely right! Looks like I copy-pasted a little too fast... traceback_lines is simply the call_stack string, split by new lines.
You can find the proper code in the gist:
https://gist.github.com/Vimkxi/fcf0d00df0bee7f1831109289aee0df2#file-python_mail_tracker-py-L86
I edited the article to fix the issue. Thanks :)