Search code examples
rdata.tablewarnings

Does data.table export a method for testing its .internal.selfref sentinel? If not, how to test this sentinel?


I'm asking because I'm trying to identify the root cause(s) for the warnings about "Invalid .internal.selfref detected and fixed" that I have triggered on five occasions while developing code using data.table. It seems pretty clear that, somewhere along the line, my code performs an operation on a data.table which causes it to be copied -- at least under some conditions. It's a long path to the warning, and my code-inspection hasn't revealed any likely suspects. Until I can localise the defect, I can't produce an MRE -- nor can I be confident of the integrity of the data.tables produced by my code.

A reasonably-efficient way to localise what I'm tentatively labelling an "inadvertent copy defect" would be to pepper-pot my code with stopifnot() invocations. But! I can't figure out how to write a valid.internal.selfref() method.

Section 5.13 of Writing R Extensions tells me that this method can't be written in R. I'm pretty sure there'll be a C-language internal method of data.table which guards the "Invalid .internal.selfref detected and fixed" warning.

Warning: 'Invalid .internal.selfref detected' when adding a column to a data.table returned from a function has a very nice explanation of .internal.selfref, but fails to reveal (at least in my reading) how a user of data.table could test this sentinel.


Solution

  • Looking at data.table source code, this warning seems to be triggered here.

    This means you could use the internal data.table:::selfrefok function:

    library(data.table)
    
    data.table:::selfrefok(data.table(x=1))
    #> [1] 1
    data.table:::selfrefok(data.frame(x=1))
    #> [1] 0