Python – Source code for evaluated/dynamically generated functions in Python

I’m looking for a way to create a Python function at runtime from a string containing the source code so that the source code is available by inspection.

My current approach is this:

src = 'def foo(x, y):' + '\n\t' + 'return x / y'
g = {numpy: numpy, ...}  # Modules and such required for function
l = {}
exec(src, g, l)
func = l['foo']

It works just fine, but the function doesn’t have the source code/file associated with it. This makes debugging difficult (note that the line where the error occurred in foo() is not displayed):

>>> foo(1, 0)
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-85-9df128c5d862> in <module>()
----> 1 myfunc(3, 0)

<string> in foo(x, y)

ZeroDivisionError: division by zero

If I define a function in the IPython interpreter, I can get the source code using inspect.getsource, and it will be printed in the traceback. inspect.getsourcefile returns something like '<ipython-input-19-8efed6025c6f>' For these types of functions, this is certainly not a real file. Is there a way to do something similar in a non-interactive environment?

So I was able to partially solve this problem by digging into the IPython source code. It uses the built-in module linecache, which contains functions to read source code from files and cache results. Both the inspect and traceback modules use this module to get the source code of the function.

The solution is to create the function in the same way as in the problem, but with compile and a fictitiously unique file name:

source = 'def foo(x, y):' + '\n\t' + 'return x / y'

filename = '<dynamic-123456>'  # Angle brackets may be required?
code = compile(source, filename, 'exec')

g = {numpy: numpy, ...}  # Modules and such required for function
l = {}
exec(src, g, l)
func = l['foo']

Linecache contains the variable cache, which is a dictionary that maps file names to tuples (size, mtime, lines, fullname). You can simply add an entry for the fake file name:

lines = [line + '\n' for line in source.splitlines()]

import linecache
linecache.cache[filename] = (len(source), None, lines, filename)

The function will then work with inspect.getsource(), the ? The /?? syntax is used with IPython backtracking. However, it still doesn’t seem to work with built-in backtracking. That’s enough for me because I almost always work in IPython.

EDIT: See the comment below for user2357112 on how to use backprinting in the built-in interpreter.