Go Stack Allocated Variables Can Change Address

Due to Go compiler and runtime subtleties, some Go variables storage is allocated on goroutines’ call stacks.

Those call stacks can grow as needed. Larger stacks aren’t merely extending the stack allocation at the bottom, the larger stack is higher or lower in memory than the smaller stack. How do the larger stacks work? What happens to the stack allocated variables when a goroutine’s call stack grows (and moves to another address)?

As I see it, there’s two ways for the combination of Go compiler and runtime to do this.

  1. A goroutine’s call stack is not contiguous, but rather a linked list of “arenas” of stack frames. Stack allocated variables don’t entail any extraordinary work, and are comparable to C “local” variables and function formal arguments. The function prologues and epilogues that compilers insert into assembly language or internal representations have to do more work.
  2. The new, larger stacks are another block of memory altogether, but include a copy of the previous stack. A stack resize entails copying local, stack allocated variables’ values, and resetting any pointers. Function prologues end up doing extra work when a goroutine’s call stack needs more room, but the epilogue is easy.

I wrote another program to see what happens to the storage of stack allocated variables. Credit where credit is due, this program would be impossible without the extremely clever coding in the github.com/timandy/routine package.

  1. The program finds the address of the struct g representing the main goroutine in its processes Go runtime. It keeps the address in a global variable, to avoid every recursive incarnation of a function from re-finding that address.
  2. The program prints out values like:
    • The address of a local variable. In a C program, this would be stack allocated, but is not necessarily so in a Go program
    • Address of a slice’s backing store obtained via Go builtin make(). Previous experiments show that the backing store is allocated from the call stack.
    • Address of top of stack
    • Address of bottom of stack
    • The program also saves the address of the local variable in another local variable. This other local variable has Go type uintptr. Saving the address in another stack allocated variable lets me see how the stack enlarging code treats non-pointer stack allocated variables that hold values that look like addresses.
    • The program calls a function that has the (possibly previous) top-of-stack address as a formal argument. This function obtains the current top-of-stack address, and if it’s not numerically the same as the (possibly previous) top-of-stack address, calls itself with the current top-of-stack address. Otherwise, it returns. Flow of control ends up just after the recursive function was called. This should coerce the Go runtime to create a new, larger, call stack.
  3. After enlarging the stack, the program prints out
    • The address of the same local variable whose address it printed out before enlarging the stack.
    • The address of the same slice’s backing store it printed before enlarging the stack.
    • Address of top of stack
    • Address of bottom of stack

I should be able to see if the local variable and the slice’s backing store “move” with the stack by comparing them before and after the recursive function causes the process’ runtime to enlarge the stack.

After all that explanation,

 1	local pointer       000004e80985ee10
 2	local pointer val   000004e80985edf8
 3	local uintptr       000004e80985edf8
 4	stack top           000004e80985f000
 5	local variable      000004e80985edf8
 6	backing store       000004e80985ed78
 7	stack bottom        000004e80985e000
 8	stack size          0000000000001000
 9	
10	top was           000004e80985f000
11	stack top         000004e80986e000 has changed
12	formal argument   000004e80986d3a0
13	local variable    000004e80986d328
14	stack bottom      000004e80986c000
15	stack size        0000000000002000
16	
17	stack top 2         000004e80986e000
18	local variable 2    000004e80986ddf8
19	backing store 2     000004e80986dd78
20	stack bottom 2      000004e80986c000
21	stack size 2        0000000000002000
22	local pointer 2     000004e80986de10
23	local pointer val 2 000004e80986ddf8
24	local uintptr 2     000004e80985edf8
  • Lines 1-8, the printed values outlined in Step (2), above.
  • Lines 10-15, the recursive function printed this to show it caused the stack to enlarge.
  • Lines 17-24, the values outlined in Step (3).

Lines 1-8 and 17-24 all get printed out by the same function. They demonstrate that the variables change address if the Go runtime enlarges the stack.

Lines 4 and 7 compared to lines 17 and 20 show that the main goroutine’s call stack move and gets larger. Lines 5 and 18 are the result of printing &localVar, the address of a variable local to func startRecursion. Two identical lines print out that address. No manipulation of the variable occurs between the two identical lines of code. To the writer of the program, var localVar int stays the same. When the runtime enlarges the stack, the location in memory of that variable changes.

Similarly, lines 6 and 19 illustrate that a slice’s backing store is on the call stack, and that it changes address.

My option (2), the new, larger stacks are another block of memory altogether, but include a copy of the previous stack, is what really happens, at least right now with Go 1.26.2 Stack allocated values, including some slice’s backing store, gets copied to the new, larger stack. Pointers get updated.

Compare lines 3 and 24 of my program’s output.

 3	local uintptr       000004e80985edf8
24	local uintptr 2     000004e80985edf8

Numerically, 000004e80985edf8 is the address of a variable local to func startRecursion before the runtime enlarges the stack. The value 000004e80985edf8 is stored in a variable of Go type uintptr. That value, which looks like the contents of a pointer, is somehow not a pointer, and doesn’t get updated on stack enlargement.

Upon disassembling some Go executables, I see that local variables’ values get used by instructions like 0x1b8(%rsp), which means “the contents of the address obtained by adding 441 to contents of stack pointer register”. If the Go runtime copies the stack correctly, an enlarged stack would also update the value in %rsp. Those instructions would work correctly. The real magic involves any local variables that have pointer types. The Go runtime does not update everything that looks like a pointer, as witnessed by the values of the uintptr variable staying the same before and after stack enlargement.