The call stack
Exploring the call stack
We introduced the concept of a stack in Chapter 11: Loops, iteration, and iterables. Here, we’ll learn about the call stack and how calls to functions are handled.
The call stack
Python uses a stack to keep track of function calls. We’ve seen stacks before. They are LIFO (last-in, first-out) data structures.
Let’s consider a single function call. Say we have this function:
def h(z):
return z + 1When we call z(), say with z(1), Python creates a stack frame and pushes this frame onto the stack. This frame includes some crucial information:
- the frame’s address,
- the address of the code that called the function (technically, the address of the previous stack frame),
- the code object currently being executed,
- a dictionary of local variables used,
- a dictionary of the global namespace, and
- other information useful for tracing and debugging.
The code object includes:
- its address,
- the raw bytecode produced when executing the body of the function called,
- names and values of arguments, and
- other information useful for tracing and debugging.
The top level of a module has its own stack frame. This stack frame has no previous caller and so it does not include the address of the calling code (there isn’t any).
So if we have code like this:
def h(z):
return z + 1
if __name__ == '__main__':
result = h(1)
print(result) # prints 2 then we start with the top-level stack frame. Python pushes the frame for the function call to h() onto the call stack. Then h() does its work, and when it returns, the address of the calling code tells Python where to return to, and the frame for the call to h() is popped off the stack. (We’ll ignore the call to print().)
Now let’s consider another case, where we have two functions, one which calls the other.
def g(y):
return h(y) * 2
def h(z):
return z + 1
if __name__ == '__main__':
result = g(1)
print(result) # prints 4We start with the top-level stack frame, then Python pushes the frame for the function call to g() onto the call stack. The function g() calls h() and uses the result of that call in another calculation. When g() calls h(), Python pushes a frame for the call to h(x) onto the stack. This includes the address of the frame for g() so Python knows where to return to. Then h() does its work, and when it returns, Python pops the frame for the call to h() off the stack and returns to the point at which it was called. Then g() does its work, and returns, and the stack frame for g() is popped off the stack, and we’re back at the top level. (We’ll ignore the call to print().)
If we had yet another function f() that’s called at the top level, and f() calls g(), then when we get to the call to h() the depth of the stack would be four:
- frame for
h()at top, - frame for
g(), - frame for
f(), and - top-level frame.
As the function calls return, the frames are successively popped off the stack—pop, pop, pop—and we wind up back at the top level.
This is how Python keeps track of what’s going on when a function is called.
Python provides us with the inspect module, which allows us to get information about the call stack, stack frames, and code objects (among other things).
Here’s some code we can use to inspect and document nested function calls.
"""
Demonstration of call stack with nested function calls
"""
import inspect
import os
EXCLUSIONS = ('EXCLUSIONS', 'TAB', 'filter_vars',
'pretty_code', 'pretty_frame', 'depth',
'result')
TAB = ' '
def filter_vars(coll):
"""
Because we're being intrusive---adding code to functions
to demonstrate behavior of the stack---and because we
have additional variables to support this, we need to
filter variables reported by the inspect module. This
allows us to focus on what gets passed to and used by
f(), g(), and h(), and not have to face unnecessary
clutter. """
if isinstance(coll, dict):
# Filter out dunders
coll = {k: v for k, v in coll.items()
if not (k.startswith('__')
and k.endswith('__'))}
coll = {k: v for k, v in coll.items()
if '<module' not in repr(v)}
coll = {k: v for k, v in coll.items()
if k not in EXCLUSIONS}
return coll
elif isinstance(coll, tuple):
# Filter out dunders
coll = [name for name in coll
if not (name.startswith('__')
and name.endswith('__'))]
coll = [name for name in coll
if not name.startswith('<module')]
coll = [name for name in coll
if name not in EXCLUSIONS]
return coll
else:
raise TypeError(f'Expected dict or tuple, '
f'got {type(coll)}')
def pretty_code(code_obj):
"""
From the Python documentation:
https://docs.python.org/3/reference/datamodel.html
Selected read-only attributes
-------------------------------------------------------
co_name: The function name
co_argcount: The total number of positional parameters
(including positional-only parameters
and parameters with default values)
that the function has
co_varnames: A tuple containing the names of the
local variables in the function
(starting with the parameter names)
co_code: A string representing the sequence of
bytecode instructions in the function
co_firstlineno: The line number of the first line of
the function
"""
if code_obj is None:
raise TypeError("Expected a code object; got None.")
depth = len(inspect.stack())
print(f"{TAB * (depth - 1)}{'-' * 19} "
f"CODE OBJECT {'-' * 19}")
print(f"{TAB * (depth - 1)}Address: {hex(id(code_obj))}")
print(f"{TAB * (depth - 1)}Function name: "
f"{code_obj.co_name}")
print(f"{TAB * (depth - 1)}Number of arguments: "
f"{code_obj.co_argcount}")
varnames = filter_vars(code_obj.co_varnames)
# Don't use code_obj.co_nlocals; need to filter!
print(f"{TAB * (depth - 1)}Number of local variables "
f"used by the function: {len(varnames)}")
if varnames:
print(f"{TAB * (depth - 1)}Names of local variables: "
f"{', '.join(str(x) for x in varnames)}")
print(f"{TAB * (depth - 1)}Line number of the first "
f"line of the function: "
f"{code_obj.co_firstlineno}")
print(f"{TAB * (depth - 1)}{'-' * 17} "
f"END CODE OBJECT {'-' * 17}")
def pretty_frame(frm):
"""
From the Python documentation:
https://docs.python.org/3/reference/datamodel.html
Selected read-only attributes
-------------------------------------------------------
f_back: Points to the previous stack frame (towards
the caller), or None
f_code: The code object being executed in this frame.
f_locals: The mapping used by the frame to look up
local variables.
"""
if frm is None:
raise TypeError("Expected a frame object; got None.")
depth = len(inspect.stack())
print(f"{TAB * (depth - 1)}{'-' * 22} FRAME {'-' * 22}")
addr = hex(id(frm))
print(f"{TAB * (depth - 1)}Address of this stack frame: "
f"{addr}")
addr = hex(id(frm.f_back)) if frm.f_back else 'None'
print(f"{TAB * (depth - 1)}Address of calling frame: "
f"{addr}")
ls = filter_vars(frm.f_locals)
if ls:
print(f"{TAB * (depth - 1)}Dictionary of local "
f"variables:")
for k, v in ls.items():
print(f"{TAB * depth}{k}: {v}")
print(f"{TAB * (depth - 1)}Code object being executed "
f"in this frame...")
pretty_code(frm.f_code)
print(f"{TAB * (depth - 1)}{'-' * 20} "
f"END FRAME {'-' * 20}")
print()
def f(x):
depth = len(inspect.stack())
print(f"{TAB * (depth - 1)}Size of stack: {depth}")
pretty_frame(inspect.currentframe())
print(f"{TAB * (depth - 1)}Call g()...")
result = g(x) - 1
print(f"{TAB * (depth - 1)}Returning result from "
f"f({x}): {result}")
print(f"{TAB * (depth - 1)}Size of stack: "
f"{len(inspect.stack())}")
return result
def g(y):
depth = len(inspect.stack())
print(f"{TAB * (depth - 1)}Size of stack: "
f"{len(inspect.stack())}")
pretty_frame(inspect.currentframe())
print(f"{TAB * (depth - 1)}Call h()...")
result = h(y) * 2
print(f"{TAB * (depth - 1)}Returning result from "
f"g({y}): {result}")
print(f"{TAB * (depth - 1)}Size of stack: "
f"{len(inspect.stack())}")
return result
def h(z):
depth = len(inspect.stack())
print(f"{TAB * (depth - 1)}Size of stack: "
f"{len(inspect.stack())}")
pretty_frame(inspect.currentframe())
result = z + 1
print(f"{TAB * (depth - 1)}Returning result from "
f"h({z}): {result}")
print(f"{TAB * (depth - 1)}Size of stack: "
f"{len(inspect.stack())}")
return result
if __name__ == '__main__':
print("Demonstration of stack frames with nested "
"function calls...")
print("We'll make a call to f(), but f() calls g(), "
"and g() calls h().")
print(f"Size of stack: {len(inspect.stack())}")
pretty_frame(inspect.currentframe())
print(f"Call f()...")
print(f"result = {f(1)}")
print(f"Size of stack: {len(inspect.stack())}")
print("Done!")When we run this code here’s the result:
Demonstration of stack frames with nested function calls...
We'll make a call to f(), but f() calls g(), and g() calls h().
Size of stack: 1
---------------------- FRAME ----------------------
Address of this stack frame: 0x104169a40
Address of calling frame: None
Dictionary of local variables:
f: <function f at 0x117a5d900>
g: <function g at 0x117a5d990>
h: <function h at 0x117a5da20>
Code object being executed in this frame...
------------------- CODE OBJECT -------------------
Address: 0x117cd10b0
Function name: <module>
Number of arguments: 0
Number of local variables used by the function: 0
Line number of the first line of the function: 1
----------------- END CODE OBJECT -----------------
-------------------- END FRAME --------------------
Call f()...
Size of stack: 2
---------------------- FRAME ----------------------
Address of this stack frame: 0x117a78580
Address of calling frame: 0x104169a40
Dictionary of local variables:
x: 1
Code object being executed in this frame...
------------------- CODE OBJECT -------------------
Address: 0x117cd0df0
Function name: f
Number of arguments: 1
Number of local variables used by the function: 1
Names of local variables: x
Line number of the first line of the function: 145
----------------- END CODE OBJECT -----------------
-------------------- END FRAME --------------------
Call g()...
Size of stack: 3
---------------------- FRAME ----------------------
Address of this stack frame: 0x117a78e40
Address of calling frame: 0x117a78580
Dictionary of local variables:
y: 1
Code object being executed in this frame...
------------------- CODE OBJECT -------------------
Address: 0x117cd0ea0
Function name: g
Number of arguments: 1
Number of local variables used by the function: 1
Names of local variables: y
Line number of the first line of the function: 158
----------------- END CODE OBJECT -----------------
-------------------- END FRAME --------------------
Call h()...
Size of stack: 4
---------------------- FRAME ----------------------
Address of this stack frame: 0x117a79000
Address of calling frame: 0x117a78e40
Dictionary of local variables:
z: 1
Code object being executed in this frame...
------------------- CODE OBJECT -------------------
Address: 0x117cd1000
Function name: h
Number of arguments: 1
Number of local variables used by the function: 1
Names of local variables: z
Line number of the first line of the function: 172
----------------- END CODE OBJECT -----------------
-------------------- END FRAME --------------------
Returning result from h(1): 2
Size of stack: 4
Returning result from g(1): 4
Size of stack: 3
Returning result from f(1): 3
Size of stack: 2
result = 3
Size of stack: 1
Done!Notice that the top-level stack frame has the address 0x104169a40 (hexadecimal). When we call f(), the address of stack frame for f() is 0x117a78580, but the address of the calling frame is 0x104169a40—that’s precisely the address of the top-level stack, from which f() was called.
When f() calls g(), another frame is pushed onto the stack. The address of this frame is 0x117a78e40, and the address of the calling frame is 0x117a78580—that’s precisely the address of the stack frame for f(), from which g() was called.
When g() calls h(), another frame is pushed onto the stack. The address of this frame is 0x117a79000, and the address of the calling frame is 0x117a78e40—that’s precisely the address of the stack frame for g(), from which h() was called.
Notice too that the value of the argument passed to h() is one. That’s the original value that was passed to f() in the very first call. That value is passed all the way to this point, then h() does its work and returns the result to g(), g() does its work and returns the result to f(). Then f() does its work, and the result is returned to the top level. This process is referred to as stack unwinding.
So the stack serves to keep track of all the nested calls and provides context for each call. The “back” addresses included in each stack frame are like a trail of breadcrumbs, letting Python know where to go as the stack unwinds.
This mechanism is used in Python and other languages, so (I hope) you see how useful a stack can be!