Low-level introspection tools and utilities¶

In order to implement its context manager analysis, stackscope includes some fairly arcane bits of Python introspection lore, including bytecode analysis and inspection of raw frame objects using ctypes. The stackscope.lowlevel module provides direct access to these lower-level bits, in case you want to use them for a different purpose. It also collects a few of the utilities used to track code objects for the stackscope.elaborate_frame() customization hook, in the hope that they might find some broader use.

These are supported public APIs just like the rest of the library; their membership in the lowlevel module is primarily because the problems they solve aren’t directly relevant to the typical end user.

Extracting context managers¶

stackscope.lowlevel.contexts_active_in_frame(frame: types.FrameType, origin: Any = None, next_inner: types.FrameType | None = None) → List[Context]¶

Inspects the given frame to try to determine which context managers are currently active; returns a list of stackscope.Context objects describing the active context managers from outermost to innermost.

This is the entry point to the frame-analysis functions in stackscope.lowlevel from the rest of the library. All the others are indirectly called from this one.

There are two implementations of this function with different tradeoffs. By default, the most capable one that appears to work in your environment will be chosen; you can override this choice using set_trickery_enabled(). See the documentation of that function for more information.

If frame is the frame of a generator or coroutine, then you are encouraged to pass that generator or coroutine as the origin parameter. This is required in order to get context manager information on CPython 3.11 and later when using the (fallback/safer) “referents” implementation.

next_inner should be the frame that frame is currently calling, if any. This is necessary to set the stackscope.Context.obj attribute correctly on a context manager that is currently exiting.

stackscope.lowlevel.set_trickery_enabled(enabled: bool | None) → None¶

Choose which of the two available implementations of contexts_active_in_frame() should be used. This is a global setting.

The “trickery” implementation (enabled = True) uses ctypes frame object introspection and bytecode analysis to determine all available information about context managers. It works on currently-executing frames and there are presently no known situations in which it can be fooled.

The “referents” implementation (enabled = False) uses gc.get_referents() to locate __exit__ and __aexit__ methods referenced by the frame. It doesn’t support executing frames on CPython (but does on PyPy, and supports suspended frames on CPython such as in a generator object/coroutine). It can’t tell which line a context manager started on or what name it was assigned to; you just get the context manager object and an is-async flag. It can be fooled by context managers whose __exit__ methods are not implemented by functions that know their name is __exit__, and by frames that keep direct references to methods called __exit__ for reasons unrelated to an active context manager in that frame. In exchange for these limitations, you get increased portability and robustness: it should be impossible by construction to crash the interpreter using this implementation, while with the “trickery” implementation you’re putting more trust in our level of testing and caution.

The default on CPython and (PyPy with the default “incminimark” garbage collector) is to attempt trickery-based analysis of a simple function the first time context managers need to be extracted, and to use trickery as long as that works. On other Python implementations the “referents” implementation is used. You may request a return to this dynamic default by passing enabled = None.

Frame analysis pieces¶

stackscope.lowlevel.analyze_with_blocks(code: types.CodeType) → Dict[int, Context]¶

Analyze the bytecode of the given code object, returning a partially filled-in Context object for each with or async with block.

Each key in the returned mapping uniquely identifies one with or async with block in the function, by specifying the bytecode offset of the WITH_CLEANUP_START (3.8 and earlier), WITH_EXCEPT_START (3.9 and 3.10), or PUSH_EXC_INFO (3.11+) instruction that begins its associated exception handler. The corresponding value is a Context object appropriate to that block, with the is_async, varname, and start_line fields filled in.

stackscope.lowlevel.inspect_frame(frame: types.FrameType) → FrameDetails¶

Return a FrameDetails object describing the exception handlers and evaluation stack for the currently executing or suspended frame frame.

There are three implementations of this function: one for CPython 3.8-3.10, one for CPython 3.11+, and one for PyPy when using the “incminimark” garbage collector. The appropriate one will be chosen automatically.

class stackscope.lowlevel.FrameDetails(blocks: List[FinallyBlock] = <factory>, stack: List[object] = <factory>)¶

A collection of internal interpreter details relating to a currently executing or suspended frame.

class FinallyBlock(handler: int, level: int)¶

Information about a currently active exception-catching context within the frame.

On CPython 3.11+, these are inferred from the “zero-cost exception handling” co_exceptiontable attribute of the code object. On earlier CPython and all PyPy, they are directly tracked at runtime by the frame object.

handler: int¶: The bytecode offset to which control will be transferred if an exception is raised.

level: int¶: The value stack depth at which the exception handler begins execution.

blocks: List[FinallyBlock]¶: Currently active exception-catching contexts in this frame (includes context managers too) in order from outermost to innermost

stack: List[object]¶: All values on this frame’s evaluation stack. This may be truncated at the position where an exception would unwind to, if the frame is currently executing and we don’t know its actual stack depth. Null pointers are rendered as None and local variables (including cellvars/freevars) are not included.

stackscope.lowlevel.currently_exiting_context(frame: types.FrameType) → ExitingContext | None¶

If frame is currently suspended waiting for one of its context managers’ __exit__ or __aexit__ methods to complete, then return an object indicating which context manager is exiting and whether it’s async or not. Otherwise return None.

This function uses some rather involved bytecode introspection, but only via public interfaces, and should always be safe to call even if something is incorrect in its output. It is used in the “referents” mode of contexts_active_in_frame() as well as “trickery” mode, because an exiting context manager is no longer referenced by its frame’s value stack.

For the curious, the implementation of this function contains extensive comments about the bytecode sequences to which context managers compile on different Python versions.

class stackscope.lowlevel.ExitingContext(is_async: bool, cleanup_offset: int)¶

Information about the sync or async context manager that’s currently being exited in a frame.

is_async: bool¶: True for an async context manager, False for a sync context manager.

cleanup_offset: int¶: The bytecode offset of the WITH_CLEANUP_START or WITH_EXCEPT_START instruction that begins the exception handler associated with this context manager.

stackscope.lowlevel.describe_assignment_target(insns: List[Instruction], start_idx: int) → str | None¶: Given that insns[start_idx] and beyond constitute a series of instructions that assign the top-of-stack value somewhere, this function returns a string description of where it’s getting assigned, or None if we can’t figure it out. Understands simple names, attributes, subscripting, unpacking, and positional-only function calls.

Code-object-based dispatch utilities¶

stackscope.lowlevel.get_code(thing: object, *nested_names: str) → types.CodeType¶

Return the code object that implements the behavior of thing or of a function nested inside it.

thing should be a code object or a callable: a function, method, or functools.partial object. If thing is a callable, it will be unwrapped if necessary to yield a function (looking inside methods, following decorator-produced wrappers to the original decorated function, and so forth), and then the function’s code object will be extracted. (If thing is already a code object, it is used as-is at this stage.)

If no nested_names are provided, then the code object obtained through the actions in the previous paragraph is returned directly. Otherwise, each of the nested_names is used to look up a function or class whose definition is nested inside the function we’re working with. The code object that results at the end of this traversal is returned from get_code().

A somewhat contrived example:

def make_calc(mult, add):
    class C:
        def __init__(self, addend):
            self.addend = addend
        def calculate(val):
            return val * mult + self.addend
    return C(add)

calc_code = get_code(make_calc, "C", "calculate")
calc_obj = make_calc(5, 2)
assert calc_obj.calculate.__func__.__code__ is calc_code
assert calc_obj.calculate(3) == (3 * 5) + 2 == 17

stackscope.lowlevel.code_dispatch(code_from_arg: Callable[[T], types.CodeType]) → Callable[[Callable[[Concatenate[T, P]], R]], _CodeDispatcher[T, P, R]]¶

Decorator for a function that should dispatch to different specializations depending on the code object associated with its first argument. Similar to functools.singledispatch, except that singledispatch dispatches on the type of its first argument rather than the implementation of something associated with its first argument.

“Associated with” is determined by the required code_from_arg argument, a callable which will be used to extract a code object from the first argument of the decorated function each time it is called. For example, a function that operates on Python frames, and wants to operate differently depending on what function those frames are executing, might pass lambda frame: frame.f_code as its code_from_arg.

Example use: creating a registry of “should this frame be hidden?” logic:

# The argument to code_dispatch() is used to obtain a code object
# from the first argument of each call to should_hide_frame()
@code_dispatch(lambda frame: frame.f_code)
def should_hide_frame(frame: types.FrameType) -> bool:
    return "__tracebackhide__" in frame.f_locals

# Hide any outcome.capture() or outcome.acapture() frames
@should_hide_frame.register(outcome.capture)
@should_hide_frame.register(outcome.acapture)
def hide_captures(frame: types.FrameType) -> bool:
    return True

As with singledispatch, the decorated function has some additional attributes in addition to being callable:

Use @func.register(target, *names) as a decorator to register specializations. (target, *names) identifies a code object as documented under get_code(). register() also supports invocation as a non-decorator func.register(target, *names, impl).
Use func.dispatch(first_arg) to return the function that will be invoked for calls to func(first_arg, ...).
func.registry is a read-only mapping whose keys are code objects and whose values are the corresponding specializations of the @code_dispatch-decorated function.

class stackscope.lowlevel.IdentityDict(items: Iterable[Tuple[K, V]] = ())¶

Bases: MutableMapping[K, V]

A dict that hashes objects by their identity, not their contents.

We use this to track code objects, since they have an expensive-to-compute hash which is not cached. You can probably think of other uses too.

Single item lookup, assignment, deletion, and setdefault() are thread-safe because they are each implented in terms of a single call to a method of an underlying native dictionary object.