I have a FileSystemNode struct with "pointers" to all children and the parent
struct FileSystemNode{
current_directory_path: String,
current_directory_name: String,
//we need mutable and immutable access to a node
// (from here to child and from child's child to parent)
parent_directory: Option<Rc<RefCell<FileSystemNode>>>,
children: Vec<Rc<RefCell<FileSystemNode>>>,
//if size is 0, this is a directory
size: usize
}
Due to this behaviour, GDB will recursively print every parent and child, thus going into an endless recursion.
To prevent this, I tried to start a pretty printer, which will simply skip (or replace by a member) the parents and/or children. My first try was to print a hardcoded string, just to make sure I was on the right track:
class FileSystemNodePrinter:
def __init__(self, val):
self.val = val
def to_string(self):
return 'test'
def display_hint(self):
return 'FileSystemNode'
However, this does absolutely nothing. The output is the same as it would be without the pretty printer. The printer is listed in the info pretty-printer
output, but I cannot be sure it is using it.
The output is:
$1 = Rc(strong=1, weak=0) = {value = RefCell(borrow=0) = {
value = advent_of_code_2022::day7::FileSystemNode {
current_directory_path: "/",
current_directory_name: "",
parent_directory: core::option::Option<alloc::rc::Rc<core::cell::RefCell<advent_of_code_2022::day7::FileSystemNode>>>::none,
children: Vec(size=0),
size: 0},
borrow = 0},
strong = 1, weak = 0}
I assume however, the problem is FileSystemNode being within the Rc<RefCell>
Repro Steps
The rust code is:
use std::cell::RefCell;
use std::rc::Rc;
struct FileSystemNode{
current_directory_path: String,
current_directory_name: String,
//we need mutable and immutable access to a node
// (from here to child and from child's child to parent)
parent_directory: Option<Rc<RefCell<FileSystemNode>>>,
children: Vec<Rc<RefCell<FileSystemNode>>>,
//if size is 0, this is a directory
size: usize
}
impl FileSystemNode {
pub fn new() -> FileSystemNode {
return FileSystemNode {
current_directory_path: String::from(""),
current_directory_name: String::from(""),
parent_directory: None,
children: Vec::new(),
size: 0
};
}
}
pub fn main(){
let root = Rc::new(RefCell::new(FileSystemNode::new()));
root.borrow_mut().current_directory_path = String::from("/");
let node = Rc::new(RefCell::new(FileSystemNode::new()));
node.borrow_mut().current_directory_path = String::from("/test/");
node.borrow_mut().current_directory_name = String::from("test");
node.borrow_mut().parent_directory = Some(Rc::clone(&root));
root.borrow_mut().children.push(Rc::clone(&node));
println!("");
}
My .gdbinit file is:
import sys
sys.path.insert(0, "/path/to/FileSystemNodePrettyPrinterParent")
from FileSystemNodePrettyPrinter import register_printer
register_printer(None)
end
layout next
and the full FileSystemPrettyPrinter.py is
#tmp easy pretty printer to fix endless recursion
import gdb
class FileSystemNodePrinter:
def __init__(self, val):
self.val = val
def to_string(self):
return 'test'
def display_hint(self):
return 'FileSystemNode'
def register_printer(objfile):
if objfile == None:
objfile = gdb.current_objfile()
gdb.printing.register_pretty_printer(objfile, build_pretty_printer(), True)
print("Registered pretty printers for UE classes")
def build_pretty_printer():
pp = gdb.printing.RegexpCollectionPrettyPrinter("Custom")
pp.add_printer("FileSystemNode", '^FileSystemNode$', FileSystemNodePrinter)
return pp
register_printer(None)
(The registration process I have more or less copied from UE4).
Set the path in .gdbinit correctly, start the program and print root once it has been setup.
In order to match types you have to match the full path of the type with your RegEx.
Something like "^yourcrate::module::path::FileSystemNode$"
should work instead of the simple "^FileSystemNode$"
.
This is to make sure you can match only the types you actually want to and not types from other crates that happen to have the same name.