stackscope: unusually detailed Python stack introspection

stackscope is a library that helps you tell what your running Python program is doing and how it got there. It can provide detailed stack traces, similar to what you get in an exception traceback, but without needing to throw an exception first. Compared to standard library facilities such as traceback.extract_stack(), it is far more versatile. It supports async tasks, generators, threads, and greenlets; provides information about active context managers in each stack frame; and includes a customization interface that library authors can use to teach it to improve the stack extraction logic for code that touches their library. (As an example of the latter, the stack of an async task blocked in a run_in_thread() function could be made to cover the code that’s running in the thread as well.)

stackscope is loosely affiliated with the Trio async framework, and shares Trio’s obsessive focus on usability and correctness. The context manager analysis is especially helpful with Trio since you can use it to understand where the nurseries are. You don’t have to use stackscope with Trio, though; it requires only the Python standard library, 3.8 or later, and the ExceptionGroup backport on versions below 3.11.

stackscope is mostly intended as a building block for other debugging and introspection tools. You can use it directly, but there’s only rudimentary support for end-user-facing niceties such as pretty-printed output. On the other hand, the core logic is (believed to be) robust and flexible, exposing customization points that third-party libraries can use to help stackscope make better tracebacks for their code. stackscope ships out of the box with such “glue” for Trio, greenback, and some of their lower-level dependencies.

stackscope requires Python 3.8 or later. It is fully type-annotated and is tested with CPython (every minor version through 3.12) and PyPy, on Linux, Windows, and macOS. It will probably work on other operating systems. Basic features will work on other Python implementations, but the context manager decoding will be less intelligent, and won’t work at all without a usable gc.get_referents().

Quickstart

Call stackscope.extract() to obtain a stackscope.Stack describing the stack of a coroutine object, generator iterator (sync or async), greenlet, or thread. If you want to extract part of the stack that led to the extract() call, then either pass a stackscope.StackSlice or use the convenience aliases extract_since() and extract_until().

Trio users: Try print(stackscope.extract(trio.lowlevel.current_root_task(), recurse_child_tasks=True)) to print the entire task tree of your Trio program.

Once you have a Stack, you can:

• Format it for human consumption: str() obtains a tree view as shown in the example below, or use stack.format() to customize it or stack.format_flat() to get an alternate format that resembles a standard Python traceback.

• Iterate over it (or equivalently, its frames attribute) to obtain a series of stackscope.Frames for programmatic inspection. Each frame represents one function call. In addition to the interpreter-level frame object, it lets you access information about the active context managers in that function (Frame.contexts).

• Look at its leaf attribute to see what’s left once you peel away all the frames. For example, this might be some atomic awaitable such as an asyncio.Future. It will be None if the frames tell the whole story.

• Use its as_stdlib_summary() method to get a standard library traceback.StackSummary object (with some loss of information), which can be pickled or passed to non-stackscope-aware tools.

Example

This code uses a number of context managers:

from contextlib import contextmanager, ExitStack

@contextmanager
def null_context():
    yield

def some_cb(*a, **kw):
    pass

@contextmanager
def inner_context():
    stack = ExitStack()
    with stack:
        stack.enter_context(null_context())
        stack.callback(some_cb, 10, "hi", answer=42)
        yield "inner"

@contextmanager
def outer_context():
    with inner_context() as inner:
        yield "outer"

def example():
    with outer_context():
        yield

def call_example():
    yield from example()

gen = call_example()
next(gen)

You can use stackscope to inspect the state of the partially-consumed generator gen, showing the tree structure of all of those context managers:

$ python3 -i example.py
>>> import stackscope
>>> stack = stackscope.extract(gen)
>>> print(stack)
stackscope.Stack (most recent call last):
╠ call_example in __main__ at [...]/stackscope/example.py:28
║ └ yield from example()
╠ example in __main__ at [...]/stackscope/example.py:25
║ ├ with outer_context():  # _: _GeneratorContextManager (line 24)
║ │ ╠ outer_context in __main__ at [...]/stackscope/example.py:21
║ │ ║ ├ with inner_context() as inner:  # inner: _GeneratorContextManager (line 20)
║ │ ║ │ ╠ inner_context in __main__ at [...]/stackscope/example.py:16
║ │ ║ │ ║ ├ with stack:  # stack: ExitStack (line 13)
║ │ ║ │ ║ ├── stack.enter_context(null_context(...))  # stack[0]: _GeneratorContextManager
║ │ ║ │ ║ │   ╠ null_context in __main__ at [...]/stackscope/example.py:5
║ │ ║ │ ║ │   ║ └ yield
║ │ ║ │ ║ ├── stack.callback(__main__.some_cb, 10, 'hi', answer=42)  # stack[1]: function
║ │ ║ │ ║ └ yield "inner"
║ │ ║ └ yield "outer"
║ └ yield

That full tree structure is exposed for programmatic inspection as well:

>>> print(stack.frames[1].contexts[0].inner_stack.frames[0].contexts[0])
inner_context(...)  # inner: _GeneratorContextManager (line 20)
╠ inner_context in __main__ at /Users/oremanj/dev/stackscope/example.py:16
║ ├ with stack:  # stack: ExitStack (line 13)
║ ├── stack.enter_context(null_context(...))  # stack[0]: _GeneratorContextManager
║ │   ╠ null_context in __main__ at /Users/oremanj/dev/stackscope/example.py:5
║ │   ║ └ yield
║ ├── stack.callback(__main__.some_cb, 10, 'hi', answer=42)  # stack[1]: function
║ └ yield "inner"

Of course, if you just want a “normal” stack trace without the added information, you can get that too:

>>> print("".join(stack.format_flat()))
stackscope.Stack (most recent call last):
  File "/Users/oremanj/dev/stackscope/example.py", line 28, in call_example
    yield from example()
  File "/Users/oremanj/dev/stackscope/example.py", line 25, in example
    yield

Detailed documentation

Extracting and inspecting stacks

Extracting a stack

The main entry point for stackscope is the stackscope.extract() function. It comes in several variants:

• extract() accepts a “stack item”, which is anything stackscope knows how to turn into a series of frame objects. This might be a StackSlice, coroutine object, generator iterator, thread, greenlet, trio.lowlevel.Task, or anything else for which a library author (maybe you) have added support through the unwrap_stackitem() customization hook.

• extract_since() and extract_until() obtain a stack for the callees or callers (respectively) of a currently-executing frame. Use extract_since(None) to get the full stack of the thread that made the extract_since() call, like inspect.stack(). These are aliases for invoking extract() with a StackSlice.

• extract_outermost() returns the outermost Frame that extract() would return, without computing the whole stack.

stackscope.extract(stackitem: StackItem, *, with_contexts: bool = True, recurse_child_tasks: bool = False) → Stack

Extract a Stack from stackitem.

stackitem may be anything that has a stack associated with it. If you want to dump the caller’s stack or the stack starting or ending with some frame, then either pass a StackSlice or use extract_since() or extract_until(), which are shortcuts for passing a StackSlice to extract(). Besides that, stackscope also ships with support for threads, greenlets, generator iterators (sync and async), coroutine objects, and a few more obscure things that might be encountered while traversing those; and libraries may add support for more using the unwrap_stackitem hook.

If with_contexts is True (the default), then each returned Frame will have a contexts attribute specifying the context managers that are currently active in that frame. If you don’t care about this information, then specifying with_contexts=False will substantially simplify the stack extraction process.

Some context managers might logically contain other tasks that each have their own Stack: trio.Nursery, asyncio.TaskGroup, etc. These will be listed as Context.children in the returned Frame.contexts for these context managers. If recurse_child_tasks is False (the default), then these “child tasks” will be rendered as stub Stack objects with only a root (the child task object) but no frames. If recurse_child_tasks is True, then the child stacks will be fully populated, including grandchildren and so on.

extract() tries not to throw exceptions; any exception should be reported as a bug. Errors encountered during stack extraction are reported in the error attribute of the returned object. If multiple errors are encountered, they will be wrapped in an ExceptionGroup.

stackscope.extract_since(outer_frame: types.FrameType | None, *, with_contexts: bool = True, recurse_child_tasks: bool = False) → Stack

Return a Stack reflecting the currently-executing frames that were directly or indirectly called by outer_frame, including outer_frame itself. Equivalent to extract(StackSlice(outer=outer_frame)) with more type checking.

If outer_frame is a frame on the current thread’s stack, the result will start with outer_frame and end with the immediate caller of extract_since().

If outer_frame is a frame on some other thread’s stack, and it remains there throughout the traceback extraction process, the resulting stack will start with outer_frame and end with some frame that was recently the innermost frame on that thread’s stack.

Note

If other_frame is not continuously on the same other thread’s stack during the extraction process, you’re likely to get a one-frame stack, maybe with an error. It’s not possible to prevent thread switching from within Python code, so we can’t do better than this without a C extension.

If outer_frame is None, the result contains all frames on the current thread’s stack, starting with the outermost and ending with the immediate caller of extract_since().

In any other case – if outer_frame belongs to a suspended coroutine, generator, greenlet, or if it starts or stops running on another thread while extract_since() is executing – you will get a Stack containing information only on outer_frame itself; depending on the situation, its error member might describe the reason more information can’t be provided.

stackscope.extract_until(inner_frame: types.FrameType, *, limit: int | types.FrameType | None = None, with_contexts: bool = True, recurse_child_tasks: bool = False) → Stack

Return a Stack reflecting the currently executing frames that are direct or indirect callers of inner_frame, including inner_frame itself.

If inner_frame belongs to a suspended coroutine or generator, or if it otherwise is not linked to other frames via its f_back attribute, then the returned traceback will contain only inner_frame and not any of its callers.

If a limit is specified, only some of the callers of inner_frame will be returned. If the limit is a frame, then it must be an indirect caller of inner_frame and it will be the first frame in the result; any of its callers will be excluded. Otherwise, the limit must be a positive integer, and the traceback will start with the limit’th parent of inner_frame.

Equivalent to extract(StackSlice(outer=outer_frame, limit=limit)) or extract(StackSlice(outer=outer_frame, inner=limit)) depending on the type of limit, except that extract_until() does more checking of its inputs (an exception will be raised if limit has an invalid type or is a frame that isn’t an indirect caller of inner_frame).

stackscope.extract_outermost(stackitem: StackItem, *, with_contexts: bool = True, recurse_child_tasks: bool = False) → Frame

Extract the outermost Frame from stackitem.

extract_outermost() produces the same result as calling extract() and returning the first Frame of the returned stack, but might be faster since it can stop once it’s extracted one frame. If the result has no Frames, an exception will be thrown.

class stackscope.StackSlice(outer: types.FrameType | None = None, inner: types.FrameType | None = None, limit: int | None = None)

Identifies a contiguous series of frames that we want to analyze.

StackSlice has no logic on its own; its only use is as something to pass to extract() or return from an unwrap_stackitem() hook.

This can be used in three different ways:

• If inner is not None, then the StackSlice logically contains currently-executing frames that are direct or indirect callers of inner, ending with inner itself. Iteration begins with inner and proceeds outward via frame.f_back links until the frame outer is reached or limit frames have been extracted. If neither of those is specified, then the stack slice starts with the outermost frame of the thread on which inner is running.

  If inner belongs to a suspended coroutine or generator, or if it otherwise is not linked to other frames via its f_back attribute, then the returned traceback will contain only inner and not any of its callers.

• If inner is None but outer is not, the StackSlice contains outer followed by its currently-executing direct and indirect callees, up to limit frames total.

  If outer is executing on the current thread, then the StackSlice ends with the frame that called stackscope.extract() (unless it is cut off before that by reaching its limit). If it is executing on some other thread, and remains so throughout the stack extraction process, then the StackSlice ends with the innermost frame on that thread. In any other case – if outer belongs to a suspended coroutine, generator, greenlet, or if it starts or stops running on another thread while stackscope.extract() is executing – the returned stack will contain information only on outer itself.

• If inner and outer are both None, the StackSlice contains the entirety of the current thread’s stack, ending with the frame that made the call to stackscope.extract().

outer: types.FrameType | None = None

The outermost frame to extract. If unspecified, start from inner (or the caller of stackscope.extract() if inner is None) and iterate outward until top-of-stack is reached or limit frames have been extracted.

inner: types.FrameType | None = None

The innermost frame to extract. If unspecified, extract all the currently-executing callees of outer, up to limit frames total.

limit: int | None = None

The maximum number of frames to extract. If None, there is no limit.

class stackscope.StackItem

Placeholder used in type hints to mean “anything you can pass to extract().”

exception stackscope.InspectionWarning

Warning raised if something goes awry during frame inspection.

Working with stacks

This section documents the Stack object that extract() returns, as well as the Frame and Context objects that it refers to. All of these are dataclasses. Their primary purpose is to organize the data returned by extract().

class stackscope.Stack(root: object, frames: Sequence[Frame], leaf: object = None, error: Exception | None = None)

Representation of a generalized call stack.

In addition to the attributes described below, you can treat a Stack like an iterable over its frames, and its len() is the number of frames it contains.

frames: Sequence[Frame]

The series of frames (individual calls) in the call stack, from outermost/oldest to innermost/newest.

root: object

The object that was originally passed to extract() to produce this stack trace, or None if the trace was created from a StackSlice (which doesn’t carry any information beyond the frames).

leaf: object = None

An object that provides additional context on what this call stack is doing, after you peel away all the frames.

If this callstack comes from a generator that is yielding from an iterator which is not itself a generator, or comes from an async function that is awaiting an awaitable which is not itself a coroutine, then leaf will be that iterator or awaitable.

Library glue may provide additional semantics for leaf; for example, the call stack of an async task that is waiting on an event might set leaf to that event.

If there is no better option, leaf will be None.

error: Exception | None = None

The error encountered walking the stack, if any. (stackscope does its best to not actually raise exceptions out of extract().)

format(*, ascii_only: bool = False, show_contexts: bool = True, show_hidden_frames: bool = False) → List[str]

Return a list of newline-terminated strings describing this object, which may be printed for human consumption. str(obj) is equivalent to "".join(obj.format()).

Parameters:

• ascii_only – Use only ASCII characters in the output. By default, Unicode line-drawing characters are used.

• show_contexts – Include information about context managers in the output. This is the default; pass False for a shorter stack trace that only includes frames in the main series.

• show_hidden_frames – Include frames in the output even if they are marked as hidden. By default, hidden frames will be suppressed. See Frame.hide for more details.

format_flat(*, show_contexts: bool = False) → List[str]

Return a list of newline-terminated strings providing a flattened representation of this stack.

This will be formatted similarly to a standard Python traceback, such as might be produced if an exception were raised at the point where the stack was extracted. Context manager information is not included by default, but can be requested using the show_contexts parameter.

as_stdlib_summary(*, show_contexts: bool = False, show_hidden_frames: bool = False, capture_locals: bool = False) → StackSummary

Return a representation of this stack as a standard traceback.StackSummary. Unlike the Stack object, a StackSummary can be pickled and will not keep frames alive, at the expense of some loss of information.

If show_contexts is True, then additional frame summaries will be emitted describing the context managers active in each frame. See the documentation of Frame.as_stdlib_summary_with_contexts() for details.

By default, hidden frames (Frame.hide) are not included in the output. You can use the show_hidden_frames parameter to override this.

capture_locals is passed through to the Frame.as_stdlib_summary() calls for each stack frame; see that method’s documentation for details on its semantics.

class stackscope.Frame(pyframe: types.FrameType, lineno: int = -1, origin: StackItem | None = None, contexts: Sequence[Context] = (), hide: bool = False, hide_line: bool = False)

Representation of one call frame within a generalized call stack.

pyframe: types.FrameType

The Python frame object that this frame describes.

lineno: int = -1

A line number in pyframe.

This is the currently executing line number, or the line number at which it will resume execution if currently suspended by a yield statement or greenlet switch, as captured when the Frame object was constructed.

origin: StackItem | None = None

The innermost weak-referenceable thing that we looked inside to find this frame, or None. Frames themselves are not weak-referenceable, but extract_outermost(frame.origin).pyframe will recover the original frame.pyframe. For example, when traversing an async call stack, origin might be a coroutine object or generator iterator.

This is exposed for use in async debuggers, which might want a way to get ahold of a previously-reported frame if it’s still running, without keeping it pinned in memory if it’s finished.

contexts: Sequence[Context] = ()

The series of contexts (with or async with blocks) that are active in this frame, from outermost to innermost. A context is considered “active” for this purpose from the point where its manager’s __enter__ or __aenter__ method returns until the point where its manager’s __exit__ or __aexit__ method returns.

hide: bool = False

If true, this frame relates to library internals that are likely to be more distracting than they are useful to see in a traceback. Analogous to the __tracebackhide__ variable supported by pytest. Hidden frames are suppressed by default when printing stacks, but this can be controlled using the show_hidden_frames argument to format().

hide_line: bool = False

Limited version of hide which by default suppresses display of the executing line, but not of the function information or context managers associated with the frame. As with hide, you can force the hidden information to be displayed by specifying the show_hidden_frames argument to format().

property filename: str

The filename of the Python file from which the code executing in this frame was imported.

property funcname: str

The name of the function executing in this frame.

property clsname: str | None

The name of the class that contains the function executing in this frame, or None if we couldn’t determine one.

This is determined heuristically, based on the executing function having a first argument named self or cls, so it can be fooled.

property modname: str | None

The name of the module that contains the function executing in this frame, or None if we couldn’t determine one.

This is looked up using the __name__ attribute of the frame’s globals namespace. It can be fooled, but usually won’t be. Another option, which is possibly more reliable but definitely much slower, would be to iterate through sys.modules looking for a module whose __file__ matches this frame’s filename.

property linetext: str

The text of the line of source code that this stack entry describes. The result has leading and trailing whitespace stripped, and does not end in a newline.

format(*, ascii_only: bool = False, show_contexts: bool = True, show_hidden_frames: bool = False) → List[str]

Return a list of newline-terminated strings describing this object, which may be printed for human consumption. str(obj) is equivalent to "".join(obj.format()).

Parameters:

• ascii_only – Use only ASCII characters in the output. By default, Unicode line-drawing characters are used.

• show_contexts – Include information about context managers in the output. This is the default; pass False for a shorter stack trace that only includes frames in the main series.

• show_hidden_frames – Include frames in the output even if they are marked as hidden. By default, hidden frames will be suppressed. See Frame.hide for more details.

as_stdlib_summary(*, capture_locals: bool = False) → FrameSummary

Return a representation of this frame entry as a standard traceback.FrameSummary object. Unlike the Frame object, a FrameSummary can be pickled and will not keep frames alive, at the expense of some loss of information.

If capture_locals is True, then the returned FrameSummary will contain the stringified object representations of local variables in the frame, just like passing capture_locals=True to traceback.StackSummary.extract().

for ... in as_stdlib_summary_with_contexts(*, show_hidden_frames: bool = False, capture_locals: bool = False) → Iterator[FrameSummary]

Return a representation of this frame and its context managers as a series of standard traceback.FrameSummary objects.

The last yielded FrameSummary matches what as_stdlib_summary() would return. Before that, one or more FrameSummary objects will be yielded for each of the active contexts in this frame. Each context will get one FrameSummary introducing it (pointing to the start of the with or async with block), followed by zero or more frames containing any relevant substructure, such as elements in an ExitStack or nested context managers within a @contextmanager function. The order of FrameSummary objects is intended to hew as closely as possible to the (reverse) path that an exception would take if it were to propagate up the call stack. That is, the result of as_stdlib_summary_with_contexts() should ideally look pretty similar to what you would see when printing out a traceback after an exception.

A FrameSummary that introduces a context will append some additional information (the type of the context manager and the name that its result was assigned to) to the function name in the returned object, in parentheses after a space. This results in reasonable output from traceback.StackSummary.format().

By default, hidden frames (Frame.hide) encountered during context manager traversal are not included in the output. You can use the show_hidden_frames parameter to override this. The frame on which you called as_stdlib_summary_with_contexts() will be included unconditionally.

If capture_locals is True, then the local reprs will be included in each FrameSummary, as with as_stdlib_summary(). Frame summaries that introduce a context will include the stringified context manager object as a fictitious local called "<context manager>".

class stackscope.Context(obj: object, is_async: bool, is_exiting: bool = False, varname: str | None = None, start_line: int | None = None, description: str | None = None, inner_stack: Stack | None = None, children: Sequence[Context | Stack] = (), hide: bool = False)

Information about a context manager active within a frame.

obj: object

The object that best describes what this context is doing. By default, this is the context manager object (the thing with the __enter__/__aenter__ and __exit__/__aexit__ methods), but library glue may override it to provide something more helpful. For example, an async with trio.open_nursery(): block will put the trio.Nursery object here instead of the context manager that wraps it.

is_async: bool

True for an async context manager, False for a sync context manager.

is_exiting: bool = False

True if this context manager is currently exiting, i.e., the next thing in the traceback is a call to its __exit__ or __aexit__.

varname: str | None = None

The name that the result of the context manager was assigned to. In with foo() as bar:, this is the string "bar". This may be an expression representing any valid assignment target, not just a simple identifier, although a simple identifier is by far the most common case. If the context manager result was not assigned anywhere, or if its assignment target was too complex for us to reconstruct, name will be None.

start_line: int | None = None

The line number on which the with or async with block started, or None if we couldn’t determine it. (In order to determine the corresponding filename, you need to know which Frame this Context is associated with.)

description: str | None = None

A description of the context manager suitable for human-readable output. By default this is None, meaning we don’t know how to do better than repr(obj), but library glue may augment it in some cases, such as to provide the arguments that were passed to a @contextmanager function.

inner_stack: Stack | None = None

The call stack associated with the implementation of this context manager, if applicable. For a @contextmanager function, this will typically contain a single frame, though it might be more if the function uses yield from. In most other cases there are no associated frames so stack will be None.

children: Sequence[Context | Stack] = ()

The other context managers or child task stacks that are logically nested inside this one, if applicable. For example, an ExitStack will have one entry here per thing that was pushed on the stack, and a trio.Nursery will have one entry per child task running in the nursery.

hide: bool = False

If true, this context manager relates to library internals that are likely to be more distracting than they are useful to see in a traceback. Analogous to the __tracebackhide__ variable supported by pytest. Hidden context managers are suppressed by default when printing stacks, but this can be controlled using the show_hidden_frames argument to format().

format(*, ascii_only: bool = False, show_contexts: bool = True, show_hidden_frames: bool = False) → List[str]

Return a list of newline-terminated strings describing this object, which may be printed for human consumption. str(obj) is equivalent to "".join(obj.format()).

Parameters:

• ascii_only – Use only ASCII characters in the output. By default, Unicode line-drawing characters are used.

• show_contexts – Include information about context managers in the output. This is the default; pass False for a shorter stack trace that only includes frames in the main series.

• show_hidden_frames – Include frames in the output even if they are marked as hidden. By default, hidden frames will be suppressed. See Frame.hide for more details.

Customizing stackscope for your library

stackscope contains several customization hooks that allow it to be adapted to provide good stack traces for context managers, awaitables, and control-flow primitives that it doesn’t natively know anything about. These are implemented either as @functools.singledispatch functions (which dispatch to a different implementation depending on the type of their first argument) or as @stackscope.lowlevel.code_dispatch functions (which dispatch to a different implementation depending on the identity of the Python code object associated with their first argument). Implementations of these hooks that support a particular library are referred to as stackscope “glue” for that library.

If you’re working with a library that could be better-supported by stackscope, you have two options for implementing that support:

• If you maintain the library that the customizations are intended to support, then define a function named _stackscope_install_glue_ at top level in any of your library’s module(s), which takes no arguments and returns None. The body of the function should register customization hooks appropriate to your library, using the stackscope APIs described in the rest of this section. As long as you only write import stackscope inside the body of the glue installation function, this won’t require that users of your library install stackscope, but they will benefit from your glue if they do.

• If you’re contributing glue for a library you don’t maintain, you can put the glue in stackscope instead. We have glue for several modules and a system that avoids registering it unless the module has been imported. See stackscope/_glue.py, and feel free to submit a PR.

If the same module has glue implemented using both of these methods, then the glue provided by the module will be used; the glue shipped with stackscope is ignored. This allows for a module’s glue to start out being shipped with stackscope and later “graduate” to being maintained upstream.

Overview of customization hooks

In order to understand the available customization hooks, it’s helpful to know how stackscope’s stack extraction works internally. There are separate systems for frames and for context managers, and they can interact with each other recursively.

Frames

There are two hooks that are relevant in determining the frames in the returned stack: unwrap_stackitem() and elaborate_frame().

unwrap_stackitem() receives a “stack item”, which may have been passed to stackscope.extract() or returned from an elaborate_frame() or unwrap_stackitem() hook. It is dispatched based on the Python type of the stack item. Abstractly speaking, a stack item should be something that logically has Python frame objects associated with it, and the job of unwrap_stackitem() is to turn it into something that is closer to those frame objects. unwrap_stackitem() may return a single stack item, a sequence thereof, or None if it can’t do any unwrapping. Each returned stack item is recursively unwrapped in the same way until no further unwrapping can be done. The resulting frame objects become the basis for the Stack.frames and any non-frame-objects go in Stack.leaf.

Example: built-in glue for unwrapping a generator iterator:

@unwrap_stackitem.register(types.GeneratorType)
def unwrap_geniter(gen: types.GeneratorType[Any, Any, Any]) -> Any:
    if gen.gi_running:
        return StackSlice(outer=gen.gi_frame)
    return (gen.gi_frame, gen.gi_yieldfrom)

elaborate_frame() operates after unwrapping is complete, and is dispatched based on the code object identity of the executing frame, so it’s useful for customizations that are specific to a particular function. (You get approximately one code object per source file location of a function definition.) It receives the frame it is elaborating as well as the next inner frame-or-leaf for context. It can customize the Frame object, such as by setting the Frame.hide attribute or modifying the Frame.contexts. It can also redirect the rest of the stack extraction, by returning a stack item or sequence of stack items that will be used in place of the otherwise-next frame and all of its callees. (If it returns a sequence that ends with the otherwise-next frame, then the preceding elements are inserted before the rest of the stack trace rather than replacing it.)

Example: glue for elaborating greenback.await_():

@elaborate_frame.register(greenback.await_)
def elaborate_greenback_await(
    frame: Frame, next_inner: object
) -> object:
    frame.hide = True

    if (
        isinstance(next_inner, Frame)
        and next_inner.pyframe.f_code.co_name != "switch"
    ):
        # await_ that's not suspended at greenlet.switch() requires
        # no special handling
        return None

    # Greenback-mediated await of async function from sync land.
    # If we have a coroutine to descend into, do so;
    # otherwise the traceback will unhelpfully stop here.
    # This works whether the coro is running or not.
    # (The only way to get coro=None is if we're taking
    # the traceback in the early part of await_() before
    # coro is assigned.)
    return frame.pyframe.f_locals.get("coro")

Contexts

There are three hooks relevant in determining the context managers in the returned stack: unwrap_context(), unwrap_context_generator(), and elaborate_context(). They are less complex than the frame hooks since they only operate on one context manager at a time.

unwrap_context() handles context managers that wrap other context managers. It receives the “outer” context manager and returns the “inner” one, or returns None to indicate no further unwrapping is needed. It is dispatched based on the type of the outer context manager. When using unwrap_context(), the “outer” context manager is totally lost; it appears to stackscope, and to clients of extract(), as though only the “inner” one ever existed. If you prefer to include both, you can assign to the Context.children attribute in elaborate_context() instead.

Example: glue for elaborating greenback.async_context objects:

@unwrap_context.register(greenback.async_context)
def unwrap_greenback_async_context(manager: Any) -> Any:
    return manager._cm

unwrap_context_generator() is a specialization of unwrap_context() for generator-based context managers (@contextmanager and @asynccontextmanager). It works exactly like unwrap_context() except that it takes as its first argument the generator’s Stack rather than the context manager object. Like elaborate_frame(), unwrap_context_generator() is dispatched based on the code object identity of the function that implements the context manager, so you can unwrap different generator-based context managers in different ways even though their context manager objects all have the same type.

elaborate_context() is called once for each context manager before trying to unwrap it, and again after each successful unwrapping. It is dispatched based on the context manager type and fills in attributes of the Context object, such as Context.description, Context.children, and Context.inner_stack. These Context attributes might be filled out using calls to fill_context() or extract(), which will recursively execute context/frame hooks as needed. elaborate_context() can also change the Context.obj which may influence further unwrapping attempts.

Example: glue for handling generator-based @contextmanagers:

@elaborate_context.register(contextlib._GeneratorContextManagerBase)
def elaborate_generatorbased_contextmanager(mgr: Any, context: Context) -> None:
    # Don't descend into @contextmanager frames if the context manager
    # is currently exiting, since we'll see them later in the traceback
    # anyway
    if not context.is_exiting:
        context.inner_stack = stackscope.extract(mgr.gen)
    context.description = f"{mgr.gen.__qualname__}(...)"

Utilities for use in customization hooks

stackscope.extract_child(stackitem: StackItem, *, for_task: bool) → Stack

Perform a recursive call equivalent to extract(stackitem), but reusing the options that were passed to the original extract(). You should only call this from within a customization hook such as elaborate_context().

If for_task is True, then this nested stackitem is considered to represent an async child task. Its stack will be fully extracted only if the outer extract() call specified recurse_child_tasks=True; otherwise you will get a stub Stack with a root but no frames.

stackscope.fill_context(context: Context) → None

Augment the given newly-constructed Context object using the context manager hooks (unwrap_context() and elaborate_context()), calling both hooks in a loop until a steady state is reached.

Customization hooks reference

stackscope.unwrap_stackitem(item: StackItem) → StackItem | Sequence[StackItem] | FrameIterator[StackItem] | None

Hook for turning something encountered during stack traversal into one or more objects that are easier to understand than it, eventually resulting in a Python frame. May return a single object, a sequence of objects, or None if we don’t know how to unwrap any further. When extracting a stack, unwrap_stackitem() will be applied repeatedly until a series of frames results.

The built-in unwrapping rules serve as good examples here:

• Unwrapping a coroutine object, generator iterator, or async generator iterator produces a tuple (frame, next) where frame is a Python frame object and next is the thing being awaited or yielded-from (potentially another coroutine object, generator iterator, etc).

• Unwrapping a threading.Thread produces the sequence of frames that form the thread’s stack, from outermost to innermost.

• Unwrapping an async generator asend() or athrow() awaitable produces the async generator it is operating on.

• Unwrapping a coroutine_wrapper object produces the coroutine it is wrapping. This allows stackscope to look inside most awaitable objects. (A “coroutine wrapper” is the object returned when you call coro.__await__() on a coroutine object, which acts like a coroutine object except that it also implements __next__.)

stackscope.yields_frames(fn: Callable[[P], Iterator[T]]) → Callable[[P], FrameIterator[T]]

Decorator for an unwrap_stackitem() implementation which allows the series of stack items to be yielded one-by-one instead of returned in a sequence.

The main benefit of this approach is that previously-yielded frames can be preserved even if an exception is raised. It is used by the built-in glue that handles StackSlice objects. A decorator is needed to distinguish an iterator of stack items (which should be unpacked and treated one-by-one) from a generator iterator with a different purpose (which is a common stack item that we definitely should not iterate over).

Put @yields_frames underneath @unwrap_stackitem.register.

stackscope.elaborate_frame(frame: Frame, next_inner: object) → StackItem | Sequence[StackItem] | None

Hook for providing additional information about a frame encountered during stack traversal. This hook uses @code_dispatch, so it can be customized based on which function the frame is executing.

next_inner is the thing that the frame is currently busy with: either the next Frame in the list of Stack.frames, or else the Stack.leaf (which may be None) if there is no next frame.

The elaborate_frame() hook may modify the attributes of frame, such as by setting Frame.hide. It may also redirect the remainder of the stack trace, by returning an object or sequence of objects that should be unwrapped to become the new next_inner. If the return value is a sequence and it ends with next_inner, then the items before next_inner are inserted before the remainder of the stack trace instead of replacing it. If you don’t want to affect the rest of the stack trace, then return None (equivalent to next_inner). If you want to remove the rest of the stack trace and not replace it with anything, then return PRUNE (which is equivalent to an empty tuple).

stackscope.PRUNE

Sentinel value which may be returned by elaborate_frame() to indicate that the remainder of the stack (containing the direct and indirect callees of the frame being elaborated) should not be included in the extracted stackscope.Stack.

stackscope.customize(target: Any = None, *inner_names: str, hide: bool = False, hide_line: bool = False, prune: bool = False, elaborate: Callable[[Frame, object], object] | None = None) → Any

Shorthand for common elaborate_frame() customizations which affect how stack extraction interacts with invocations of specific functions.

(target, *inner_names) identifies a code object; all frames executing that code object will receive the customizations specified by the keyword arguments to customize(). Typically you would a function as target, with no inner_names, to customize frames that are executing that function. You only need inner_names if you’re trying to name a nested function; see get_code() for details.

If you don’t specify a target, then customize() returns a partially bound invocation of itself so that you can use it as a decorator. The target in that case is the decorated function; the decorator returns that function unchanged. (Note the distinction: @customize decorates the function whose frames get the custom behavior, while @elaborate_frame.register decorates the function that implements the custom behavior.)

The customizations are specified by the keyword arguments you pass:

• If elaborate is specified, then it will be registered as an elaborate_frame hook for the matching frames.

• If hide is True, then the matching frames will have their Frame.hide attribute set to True, indicating that they should not be shown by default when printing the stack.

• If hide_line is True, then the matching frames will have their Frame.hide_line attribute set to True, indicating that the executing line should not be shown by default when printing the stack (but the function info and contexts still will be).

• If prune is True, then direct and indirect callees of the matching frames will not be included when extracting the stack. This option only has effect if elaborate either is unspecified or returns None.

stackscope.unwrap_context(manager: ContextManager[Any] | AsyncContextManager[Any], context: Context) → None | ContextManager[Any] | AsyncContextManager[Any] | tuple[()]

Hook for extracting an inner context manager from another context manager that wraps it. The stackscope context object is also provided in case it’s useful. Return None if there is no further unwrapping to do. Return PRUNE (equivalent to an empty tuple) to hide this context manager from the traceback. Unlike unwrap_stackitem(), it is not currently supported to let the result of unwrapping a context manager be a sequence of multiple context managers.

Note

If the original context manager is currently exiting, the frames implementing its __exit__ will appear on the stack regardless of any unwrapping you do here. You can customize elaborate_frame() for the appropriate __exit__ if you want to affect the display there as well.

stackscope.unwrap_context_generator(frame: Frame, context: Context) → None | ContextManager[Any] | AsyncContextManager[Any] | tuple[()]

Hook for extracting an inner context manager from the outermost frame of a generator-based context manager that wraps it. This hook uses @code_dispatch, so it can be customized based on the identity of the function that implements the context manager. Apart from that, its semantics are equivalent to unwrap_context().

Note

If the context manager you’re unwrapping uses yield from, it’s possible that you’ll need to access callees of frame to implement your logic. You can find these using Context.inner_stack, or if that’s None because the context is currently exiting, you can reconstruct it using stackscope.extract(context.obj.gen).

stackscope.elaborate_context(manager: ContextManager[Any] | AsyncContextManager[Any], context: Context) → None

Hook for providing additional information about a context manager encountered during stack traversal. It should modify the attributes of the provided context object (a Context) based on the provided manager (the actual context manager, i.e., the thing whose type has __enter__ and __exit__ attributes).

Low-level introspection tools and utilities

In order to implement its context manager analysis, stackscope includes some fairly arcane bits of Python introspection lore, including bytecode analysis and inspection of raw frame objects using ctypes. The stackscope.lowlevel module provides direct access to these lower-level bits, in case you want to use them for a different purpose. It also collects a few of the utilities used to track code objects for the stackscope.elaborate_frame() customization hook, in the hope that they might find some broader use.

These are supported public APIs just like the rest of the library; their membership in the lowlevel module is primarily because the problems they solve aren’t directly relevant to the typical end user.

Extracting context managers

stackscope.lowlevel.contexts_active_in_frame(frame: types.FrameType, origin: Any = None, next_inner: types.FrameType | None = None) → List[Context]

Inspects the given frame to try to determine which context managers are currently active; returns a list of stackscope.Context objects describing the active context managers from outermost to innermost.

This is the entry point to the frame-analysis functions in stackscope.lowlevel from the rest of the library. All the others are indirectly called from this one.

There are two implementations of this function with different tradeoffs. By default, the most capable one that appears to work in your environment will be chosen; you can override this choice using set_trickery_enabled(). See the documentation of that function for more information.

If frame is the frame of a generator or coroutine, then you are encouraged to pass that generator or coroutine as the origin parameter. This is required in order to get context manager information on CPython 3.11 and later when using the (fallback/safer) “referents” implementation.

next_inner should be the frame that frame is currently calling, if any. This is necessary to set the stackscope.Context.obj attribute correctly on a context manager that is currently exiting.

stackscope.lowlevel.set_trickery_enabled(enabled: bool | None) → None

Choose which of the two available implementations of contexts_active_in_frame() should be used. This is a global setting.

The “trickery” implementation (enabled = True) uses ctypes frame object introspection and bytecode analysis to determine all available information about context managers. It works on currently-executing frames and there are presently no known situations in which it can be fooled.

The “referents” implementation (enabled = False) uses gc.get_referents() to locate __exit__ and __aexit__ methods referenced by the frame. It doesn’t support executing frames on CPython (but does on PyPy, and supports suspended frames on CPython such as in a generator object/coroutine). It can’t tell which line a context manager started on or what name it was assigned to; you just get the context manager object and an is-async flag. It can be fooled by context managers whose __exit__ methods are not implemented by functions that know their name is __exit__, and by frames that keep direct references to methods called __exit__ for reasons unrelated to an active context manager in that frame. In exchange for these limitations, you get increased portability and robustness: it should be impossible by construction to crash the interpreter using this implementation, while with the “trickery” implementation you’re putting more trust in our level of testing and caution.

The default on CPython and (PyPy with the default “incminimark” garbage collector) is to attempt trickery-based analysis of a simple function the first time context managers need to be extracted, and to use trickery as long as that works. On other Python implementations the “referents” implementation is used. You may request a return to this dynamic default by passing enabled = None.

Frame analysis pieces

stackscope.lowlevel.analyze_with_blocks(code: types.CodeType) → Dict[int, Context]

Analyze the bytecode of the given code object, returning a partially filled-in Context object for each with or async with block.

Each key in the returned mapping uniquely identifies one with or async with block in the function, by specifying the bytecode offset of the WITH_CLEANUP_START (3.8 and earlier), WITH_EXCEPT_START (3.9 and 3.10), or PUSH_EXC_INFO (3.11+) instruction that begins its associated exception handler. The corresponding value is a Context object appropriate to that block, with the is_async, varname, and start_line fields filled in.

stackscope.lowlevel.inspect_frame(frame: types.FrameType) → FrameDetails

Return a FrameDetails object describing the exception handlers and evaluation stack for the currently executing or suspended frame frame.

There are three implementations of this function: one for CPython 3.8-3.10, one for CPython 3.11+, and one for PyPy when using the “incminimark” garbage collector. The appropriate one will be chosen automatically.

class stackscope.lowlevel.FrameDetails(blocks: List[FinallyBlock] = <factory>, stack: List[object] = <factory>)

A collection of internal interpreter details relating to a currently executing or suspended frame.

class FinallyBlock(handler: int, level: int)

Information about a currently active exception-catching context within the frame.

On CPython 3.11+, these are inferred from the “zero-cost exception handling” co_exceptiontable attribute of the code object. On earlier CPython and all PyPy, they are directly tracked at runtime by the frame object.

handler: int

The bytecode offset to which control will be transferred if an exception is raised.

level: int

The value stack depth at which the exception handler begins execution.

blocks: List[FinallyBlock]

Currently active exception-catching contexts in this frame (includes context managers too) in order from outermost to innermost

stack: List[object]

All values on this frame’s evaluation stack. This may be truncated at the position where an exception would unwind to, if the frame is currently executing and we don’t know its actual stack depth. Null pointers are rendered as None and local variables (including cellvars/freevars) are not included.

stackscope.lowlevel.currently_exiting_context(frame: types.FrameType) → ExitingContext | None

If frame is currently suspended waiting for one of its context managers’ __exit__ or __aexit__ methods to complete, then return an object indicating which context manager is exiting and whether it’s async or not. Otherwise return None.

This function uses some rather involved bytecode introspection, but only via public interfaces, and should always be safe to call even if something is incorrect in its output. It is used in the “referents” mode of contexts_active_in_frame() as well as “trickery” mode, because an exiting context manager is no longer referenced by its frame’s value stack.

For the curious, the implementation of this function contains extensive comments about the bytecode sequences to which context managers compile on different Python versions.

class stackscope.lowlevel.ExitingContext(is_async: bool, cleanup_offset: int)

Information about the sync or async context manager that’s currently being exited in a frame.

is_async: bool

True for an async context manager, False for a sync context manager.

cleanup_offset: int

The bytecode offset of the WITH_CLEANUP_START or WITH_EXCEPT_START instruction that begins the exception handler associated with this context manager.

stackscope.lowlevel.describe_assignment_target(insns: List[Instruction], start_idx: int) → str | None

Given that insns[start_idx] and beyond constitute a series of instructions that assign the top-of-stack value somewhere, this function returns a string description of where it’s getting assigned, or None if we can’t figure it out. Understands simple names, attributes, subscripting, unpacking, and positional-only function calls.

Code-object-based dispatch utilities

stackscope.lowlevel.get_code(thing: object, *nested_names: str) → types.CodeType

Return the code object that implements the behavior of thing or of a function nested inside it.

thing should be a code object or a callable: a function, method, or functools.partial object. If thing is a callable, it will be unwrapped if necessary to yield a function (looking inside methods, following decorator-produced wrappers to the original decorated function, and so forth), and then the function’s code object will be extracted. (If thing is already a code object, it is used as-is at this stage.)

If no nested_names are provided, then the code object obtained through the actions in the previous paragraph is returned directly. Otherwise, each of the nested_names is used to look up a function or class whose definition is nested inside the function we’re working with. The code object that results at the end of this traversal is returned from get_code().

A somewhat contrived example:

def make_calc(mult, add):
    class C:
        def __init__(self, addend):
            self.addend = addend
        def calculate(val):
            return val * mult + self.addend
    return C(add)

calc_code = get_code(make_calc, "C", "calculate")
calc_obj = make_calc(5, 2)
assert calc_obj.calculate.__func__.__code__ is calc_code
assert calc_obj.calculate(3) == (3 * 5) + 2 == 17

stackscope.lowlevel.code_dispatch(code_from_arg: Callable[[T], types.CodeType]) → Callable[[Callable[[Concatenate[T, P]], R]], _CodeDispatcher[T, P, R]]

Decorator for a function that should dispatch to different specializations depending on the code object associated with its first argument. Similar to functools.singledispatch, except that singledispatch dispatches on the type of its first argument rather than the implementation of something associated with its first argument.

“Associated with” is determined by the required code_from_arg argument, a callable which will be used to extract a code object from the first argument of the decorated function each time it is called. For example, a function that operates on Python frames, and wants to operate differently depending on what function those frames are executing, might pass lambda frame: frame.f_code as its code_from_arg.

Example use: creating a registry of “should this frame be hidden?” logic:

# The argument to code_dispatch() is used to obtain a code object
# from the first argument of each call to should_hide_frame()
@code_dispatch(lambda frame: frame.f_code)
def should_hide_frame(frame: types.FrameType) -> bool:
    return "__tracebackhide__" in frame.f_locals

# Hide any outcome.capture() or outcome.acapture() frames
@should_hide_frame.register(outcome.capture)
@should_hide_frame.register(outcome.acapture)
def hide_captures(frame: types.FrameType) -> bool:
    return True

As with singledispatch, the decorated function has some additional attributes in addition to being callable:

• Use @func.register(target, *names) as a decorator to register specializations. (target, *names) identifies a code object as documented under get_code(). register() also supports invocation as a non-decorator func.register(target, *names, impl).

• Use func.dispatch(first_arg) to return the function that will be invoked for calls to func(first_arg, ...).

• func.registry is a read-only mapping whose keys are code objects and whose values are the corresponding specializations of the @code_dispatch-decorated function.

class stackscope.lowlevel.IdentityDict(items: Iterable[Tuple[K, V]] = ())

Bases: MutableMapping[K, V]

A dict that hashes objects by their identity, not their contents.

We use this to track code objects, since they have an expensive-to-compute hash which is not cached. You can probably think of other uses too.

Single item lookup, assignment, deletion, and setdefault() are thread-safe because they are each implented in terms of a single call to a method of an underlying native dictionary object.

Release history

stackscope 0.2.2 (2024-02-27)

Bugfixes

• Fixed an inspection error that could occur on Python 3.11 when the subject expression of an async with block covers multiple source lines. (#14)

• Update greenback glue to support the internal reorganizations in version 1.2.0. (#15)

stackscope 0.2.1 (2024-02-02)

Bugfixes

• Fixed inspection of async context managers that contain a CLEANUP_THROW bytecode instruction in their __aenter__ sequence. This can appear on 3.12+ if you write an async context manager inside an except or finally block, and would previously produce an inspection warning. (#11)

• The first invocation of stackscope.extract() no longer leaves a partially-exhausted async generator object to be garbage collected, which previously could confuse async generator finalization hooks. (#12)

stackscope 0.2.0 (2023-12-22)

With this release, stackscope can print full Trio task trees out-of-the-box. Try print(stackscope.extract(trio.lowlevel.current_root_task(), recurse_child_tasks=True)).

Backwards-incompatible changes

• The unwrap_context() hook now accepts an additional Context argument. This saves on duplicated effort between elaborate_context() and unwrap_context(), avoiding exponential time complexity in some pathological cases.

• Removed support for Python 3.7.

User-facing improvements to core logic

• Added support for representing child tasks in structured concurrency libraries, by allowing Context.children to contain Stacks in addition to the existing support for child Contexts. By default, the child tasks will not have their frames filled out, but you can override this with the new recurse_child_tasks parameter to extract(). (#9)

• Added Frame.hide_line and Context.hide attributes for more precise control of output.

• Added a new attribute Stack.root which preserves the original “stack item” object that was passed to extract(). For stacks generated from async child tasks, this will be the Task object.

• Added support for Python 3.12.

Library support (“glue”) improvements

• stackscope can now trace seamlessly across Trio/thread boundaries when extracting a stack that includes calls to trio.to_thread.run_sync() and/or trio.from_thread.run(). The functions running in the cross-thread child will appear in the same way that they would if they had been called directly without a thread transition. (#8)

• Added glue to support pytest-trio. (#4)

• Updated Trio glue to support unwrapping trio.lowlevel.Tasks and filling in the child tasks of a trio.Nursery.

Improvements for glue developers

• A library can now ship its own stackscope customizations without requiring that all of its users install stackscope. Any module may define a function called _stackscope_install_glue_(), which stackscope will call when it is first used to extract a stack trace after the module has been imported. (#7)

• Added unwrap_context_generator() hook for more specific customization of generator-based context managers.

• Modified the elaborate_frame() hook to be able to return a sequence of stack items rather than just a single one. This permits more expressive augmentation rules, such as inserting elements into the stack trace without removing what would’ve been there if the hook were not present.

• Added a new function extract_child() for use in customization hooks. It is like extract() except that it reuses the options that were specified for the outer extract() call, and contains some additional logic to prune child task frames if the outer extract() didn’t ask for them.

• elaborate_frame() now runs after Frame.contexts is populated, so it has the chance to modify the detected context managers.

stackscope 0.1.0 (2023-04-12)

Initial release.

Indices and tables

• Index

• Module Index

• Search Page

• Glossary