Skipping library code in gdb with help from libClang
In my previous post I talked about one common frustration with heavily templated libraries: the verbose stack traces, and how that experience can be mitigated through gdb’s Python API.
Single-stepping through execution is often another trouble spot. Ideally we would stop only in our own code, but library functions - especially those that employ tag dispatching - can perform a seemingly interminable sequence of trivial function calls.
Library functions that never call user code can be skipped with the next
command, but for the rest, stepping can be quite tedious. A typical solution is to:
- Make an educated guess as to which functions and methods are likely to be called
- Set breakpoints on all of them
- Use
continue
to run forward until one is encountered
This can work but is error prone and time consuming. You might forget an important function without realizing it, and afterwards you have a bunch of stray breakpoints. What we need is a way to automatically locate downstream functions, insert the breakpoints, and clean up afterwards.
Implementation
Adding breakpoints is straightforward in Python:
bp1 = gdb.Breakpoint('main.cpp:25') # file and line
bp2 = gdb.Breakpoint('Foo::bar') # function name
But how can we get the necessary file and line information? gdb can tell us the current file and line number, but lacks the semantic information to know what functions may be subsequently called. Fortunately, with Python we have access to powerful help in the form of libClang’s Python bindings.
libClang
We can take gdb’s current frame information and use it to figure out where we are in the source:
frame = gdb.selected_frame()
line = frame.find_sal().line
fname = frame.find_sal().symtab.filename
Assuming your build flow creates a compilation database - a very useful thing for tools - you can recreate the command lines used to build this translation unit:
from os import path
from clang import cindex
compilation_database_path = path.dirname('./compile_commands.json')
compdb = cindex.CompilationDatabase.fromDirectory(compilation_database_path)
cmds = compdb.getCompileCommands(tu_fname)
It is often the case that the file the debugger has stopped in is not the one that was compiled, i.e., it was an included header. We can search through the current backtrace to find the translation unit:
tu_fname = None
while frame is not None:
fn = frame.find_sal().symtab.filename
cmds = compdb.getCompileCommands(fn) # None if not found
if cmds is not None:
tu_fname = frame.find_sal().symtab.filename
break
frame = frame.older()
# extract args from cmds...
Now we have Clang (via libClang) parse our source and create an AST, using the flags we collected:
index = cindex.Index.create()
tu = index.parse(tu_fname, args)
We can get a cursor representing the AST node at our current location:
f = tu.get_file(fname)
loc = cindex.SourceLocation.from_position(tu, f, line, 1)
cur = cindex.Cursor.from_location(tu, loc)
We can further interrogate libClang:
- are there any function or method calls in this node?
- for each such call:
- does the function name match our “library” regex?
- if not, are any arguments non-POD objects?
- what are the methods of each such object?
- where does each method begin?
- are there any lambda arguments?
- where is the first line of each lambda body?
And so on. We can generate new breakpoints from these locations via the gdb API:
breakpoints.add(gdb.Breakpoint('%s:%d'%(fn, line), internal=True))
Run continue
until we reach one:
gdb.execute("continue")
Then clean up the temporary breakpoints:
for bp in breakpoints:
bp.delete()
We have now implemented a variant of the step
command that skips uninteresting library code and lets us concentrate on our own stuff. You can access the full version on Github.
Sample Session
Here’s how the debugging experience looks now, in the example from my previous post:
$ PYTHONPATH=.. LD_LIBRARY_PATH=/usr/lib/llvm-5.0/lib gdb ./swl
GNU gdb (Ubuntu 8.0.1-0ubuntu1) 8.0.1
...
(gdb) python import gdb_util.stepping
(gdb) b main
Breakpoint 1 at 0xf51: file /.../gdb_python_api/stl_with_lambda.cpp, line 21.
(gdb) run
Starting program: /.../gdb_python_api/build/swl
Breakpoint 1, main () at /.../gdb_python_api/stl_with_lambda.cpp:21
21 int main() {
(gdb) n
27 {"Monoid", "Endofunctor", "Monad"}};
(gdb)
30 std::sort(data.begin(), data.end(),
(gdb) stepu
Breakpoint -24, <lambda(const Strings&, const Strings&)>::operator()(const Strings &, const Strings &) const (__closure=0x7fffffffd9d0, a=std::vector of length 3, capacity 3 = {...}, b=std::vector of length 3, capacity 3 = {...})
at /.../gdb_python_api/stl_with_lambda.cpp:32
32 if (a[0] < b[0]) {
Our custom stepu
command has dropped us directly into the sort predicate lambda, without having to manually add breakpoints or single-step through std::sort
.
Wrapping Up
The gdb Python API opens up a world of possibilities for tool development. Here we’ve taken just a single package (libClang) and integrated it to build some useful functionality.
The Python ecosystem is very rich; using tools available from it to inspect and modify running C++ code is tremendously exciting. Here are just a few ideas:
- Graphical display of C++ data structures (pretty-printing on steroids)
- Serving a REST endpoint for high-level debugging information
- Finding
shared_ptr
reference loops with graph algorithms
I believe organizations with large codebases can easily justify investing in this kind of custom tooling.