A Python Exception Mail Tracker

Tuesday, July 3rd 2018 - Reading time : About 10 minutes

I'd like to thank David Bôle which helped me develop this piece of code back at Quantic Dream and later.

As a technical director, I believe my job is to help artist thrive and excel in their work. With that aim, the end goals are speed and simplicity. As much as we can, we should take the burden of technical complexity to help them consistently deliver great work. As The Zen of Python states : Simple is better than complex.

For that I'd like to discuss a task that I think most artists loathe with good reasons : reporting bugs. If some bugs are legitimately tricky, and in need of a proper description with insights, some are so straight-forward that they shouldn't even need someone to be reported and that's what we're going to improve today - by receiving an email with all the code's detail when an exception occurs.

A traceback is good, a debugger's view is (much) better

The basic trick here is that every Python exception (or at least, every exception that's not caught) goes through the same built-in function: sys.excepthook(). With some good ol' monkey-patch, we can execute our own snippet of code when an exception is raised. And thanks to some Python introspection, we can produce a detailed log of basically anything we want at the time of the error. For starters, here's what the Python documentation has to say about this function :

Note that the Python documentation tells you directly that you can customize this handling by replacing this call ¯\_(ツ)_/¯. So, sys.excepthook() takes three arguments : the exception's type, its value and a traceback object. Those three fit perfectly with the format_exception() function, from the traceback module. This handy function returns the full traceback as a string :

import traceback

def _get_traceback_string(exception_type, exception_value, traceback_):
    """Return the formatted traceback as a string."""
    return '\n'.join(
        traceback.format_exception(
            exception_type, exception_value, traceback_
        )
    )

You might notice I'm using a traceback_ var ; that's because we're already using the traceback module. So to not overwrite it in our function, we're adding an underscore at the end of it.

Now for the introspection part - retrieving code variables at the time of the error. There's a great amount of information in the inspect module documentation. We can find out that our traceback object has a few interesting properties, namely tb_next, tb_frame and our target for this part, f_locals, which is described as the "local namespace seen by this frame" - that is, the local variables.

def get_exception_variables(traceback_):
    """Return a dict of variables {name: value} from traceback_."""
    # Get latest call stack frame (latest_frame is None with SyntaxErrors)
    latest_frame = traceback_
    while latest_frame and latest_frame.tb_next:
        latest_frame = latest_frame.tb_next

    # No latest_frame means a SyntaxError which bears no vars
    if latest_frame:
        return latest_frame.tb_frame.f_locals
    return []

A fun thing I like to add is a code preview, to get a context of the code that failed. That's kind of a large part so I'll skip it for now - I'll probably make a second article for that.

We now have our formatted traceback and variable details ready to be mailed. For that, we enter the realm of HTML and inline CSS since a lot of mail clients don't support CSS styles (looking at you, Gmail). Sadly, that makes the code quite bulky so I'll try to keep it short, without a nice layout or colors. Although you might want to do it on your side, to get a better readability off your report.

Let's format our traceback for now. A basic thing to do is to preserve the traceback's indentation, which we can't do by using spaces since strings are stripped in most mail clients - we'll use CSS margins. We'll also use a monospaced font to preserve a "Python console" look.

def format_traceback(exception_type, exception_value, traceback_):
    """Format the call's traceback with proper indentation."""
    # Get call stack text
    call_stack = get_traceback_string(
        exception_type, exception_value, traceback_
    )
    html = []
    traceback_lines = call_stack.split("\n")
    for line in traceback_lines:
        # Keep only lines with content
        line = line.strip()
        if not line:
            continue
        # Apply offsets to preserve indentation, depending on the line
        if line == traceback_lines[0] or line == traceback_lines[-1]:
            html.append('<SPAN>{}</SPAN>'.format(line))
        elif line.startswith('File "'):
            html.append('<SPAN style="margin-left: 15px">{}</SPAN>'.format(line))
        else:
            html.append('<SPAN style="margin-left: 30px">{}</SPAN>'.format(line))

    # Return traceback lines wrapped in a div with monospace font
    return '<DIV style="font-family: monospace;">\n{}\n</DIV>\n'.format(
        "<br />\n".join(html)
    )

You might have noticed some line breaks - the "\n" - added to the output. They're quite useless in HTML code but come in handy when your formatting breaks down and you need to debug it by looking at the result, you can trust me on that. Let's keep moving and format our variables :

def format_variables(traceback_):
    """Return a html table of the local variables names and values."""
    # Build table with two columns: key/name & value
    table = '<TABLE>\n'

    # Add each variable to the table
    for key, value in sorted(_get_exception_variables(traceback_).items()):
        table += '  <TR>\n'
        table += '    <TD>{} :</TD>\n'.format(key)
        v = str(value).replace('<', '&lt;')
        table += '    <TD>{}</TD>\n'.format(v)
        table += '  </TR>\n'

    table += '</TABLE>\n'
    return table

This one is pretty much straight-forward except for the str.replace() on line 10. What happens here is that we're writing HTML content as we go, and HTML uses the lesser than and greater than symbols in its syntax. Some Python strings may be formatted as an HTML tag, like for instance when you use the print function on a module:

>>> import sys
>>> print sys
"<module 'sys' (built-in)>"

This kind of string would get converted into an HTML tag (though its syntax is broken) and not being displayed at all. To avoid this, we changed the first lesser than sign to its HTML code equivalent, < which displays it correctly without making it a valid HTML tag.

We got everything we need by now. We'll just wrap all this in a big HTML div and send it. Here's our final function that we'll use to replace the original sys.excepthook :

def main(exception_type, exception_value, traceback_):
    """Python exception hook that mails exception reports."""

    # Build mail
    message =  "".join((
        '<DIV>',
        '<H3>Traceback</H3>',
        format_traceback(exception_type, exception_value, traceback_),
        '<H3>Variables</H3>',
        format_variables(traceback_),
        '</DIV>'
    ))

    # Prepare the mail
    subject = "{}: {}".format(exception_type.__name__, exception_value)
    your_address = "you@users.com"
    user_address = "user@users.com"

    msg = MIMEMultipart('alternative')
    msg["Subject"] = subject
    msg["From"] = user_address
    msg["To"] = your_address
    msg.attach(MIMEText(mail_contents, 'html'))

    # Send the message - you'll need to enter your own server here
    server = smtplib.SMTP('smtp-server.client.com')
    server.sendmail(user_address, your_address, msg.as_string())
    server.quit()

    # Run the basic python error to print the traceback to the console
    sys.__excepthook__(exception_type, exception_value, traceback_)

If you'd like to send an email with a Gmail address, I invite you to read this question on StackOverflow.

Our tracker is now complete! I've made a gist with all of this packed in a class for convenience - with added enable() and disable() functions. Those use the sys.__excepthook__() function which simply backs up sys.excepthook(), allowing us to easily reinstate the original hook or switch between the two at will. If you write your own overwrite someday, just be sure not to overwrite that one.

A few tips if you wish to use this tracker in Maya : maya.utils.formatGuiException() is called in place of sys.excepthook(), and it takes an additional fourth keyword argument ("detail") that you'll have to add to your main function. It also necessitate that this function returns a string, that will be passed to OpenMaya.MGlobal.displayError() - you can just use a return on the sys.__excepthook_ call. Also, some errors occur quite a lot, the plugin not found error in Maya when opening a new file is one of them. Remember to filter those out - I tend to use regular expressions on the traceback string to do so. You should probably exclude SyntaxError as well.

One final thing, there might be a lot of variables in the list you will send via email - it can be useful to sort them out. Once again, The inspect module is your friend here if you want, for instance, to separate modules from other types of variables using inspect.ismodule() or inspect.isclass()...

Happy tracking !