Search code examples
rdplyrdata-manipulationr-haven

Identifying a tbl_df column class


I am reading a SAS dataset into R. SAS stores missing character values as empty quotes, but thankfully zap_empty() converts those values to NA.

My data set contains almost 400 variables, and I'd rather not check each of those individually. I would like to make a loop that identifies whether or not the variable is a character and then apply zap_empty().

read_sas() imports data as tbl_df instead of data.frame. If I convert my data to a data.frame first, the following loop works.

x <- as.data.frame(mydf)
for (i in seq(ncol(x))) {
  if(is.character(x[,i])){
    x[,i] <- zap_empty(x[,i])
  }
}

I would like to understand how to identify a column class with a Boolean test using tbl_df. Below provides an example SAS dataset, read using read_sas() from haven.

> wolves <- read_sas('http://psych.colorado.edu/~carey/Courses/PSYC7291/DataSets/SAS/wolves.sas7bdat')
>
> # The first 3 variables are characters
> glimpse(wolves)
Observations: 25
Variables: 13
$ location (chr) "rm", "rm", "rm", "rm", "rm", "rm", "rm", "rm", "rm"...
$ wolf     (chr) "rmm1", "rmm2", "rmm3", "rmm4", "rmm5", "rmm6", "rm"...
$ sex      (chr) "m", "m", "m", "m", "m", "m", "f", "f", "f", "m", "m"...
$ x1       (dbl) 126, 128, 126, 125, 126, 128, 116, 120, 116, 117, 1...
$ x2       (dbl) 104, 111, 108, 109, 107, 110, 102, 103, 103, 99, 10...
$ x3       (dbl) 141, 151, 152, 141, 143, 143, 131, 130, 125, 134, 1...
$ x4       (dbl) 81.0, 80.4, 85.7, 83.1, 81.9, 80.6, 76.7, 75.1, 74....
$ x5       (dbl) 31.8, 33.8, 34.7, 34.0, 34.0, 33.0, 31.5, 30.2, 31....
$ x6       (dbl) 65.7, 69.8, 69.1, 68.0, 66.1, 65.0, 65.0, 63.8, 62....
$ x7       (dbl) 50.9, 52.7, 49.3, 48.2, 49.0, 46.4, 45.4, 44.4, 41....
$ x8       (dbl) 44.0, 43.2, 45.6, 43.8, 42.4, 40.2, 39.0, 41.1, 44....
$ x9       (dbl) 18.2, 18.5, 17.9, 18.4, 17.9, 18.2, 16.8, 16.9, 17....
$ subject  (dbl) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, ...
>
> # But my loop cannot identify that
> for (i in 1:ncol(wolves)){
+   if (is.character(wolves[,i])){
+     print('bar')
+   } else {print('foo')}
+ }
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"
[1] "foo"

When I access the column using $ it is identified as a character, but not when using indexing.

> class(wolves$sex)
[1] "character"
> class(wolves[,'sex'])
[1] "tbl_df"     "data.frame"

Using a loop, how can I identify which columns from a tbl_df object are character variables?


From @Sumedh I can now identify which columns are characters.
Does this error mean I cannot use zap_empty() in a loop?

> for (i in seq(which(sapply(wolves, class) == 'character'))){
+   wolves[,i] <- zap_empty(wolves[,i])
+ }
 Show Traceback

 Rerun with Debug
 Error: is.character(x) is not TRUE 

Solution

  • You'll want to make sure you are testing is.character(x) on a character vector and not a single-column data frame.

    Your is.character(x[,i]) is not checking correctly for characters because a single bracket subscript will always return an object of the same type. Since x is a data frame, x[,i] is also a data frame. To get a character vector, we use the [[1]] to select the first vector in your single column dataframe of x[,i].

    x <- as.data.frame(mydf)
    for (i in seq(ncol(x))) {
      if(is.character(x[,i][[1]])){
        x[,i] <- zap_empty(x[,i][[1]])
      }
    }
    

    This is explained better and more thoroughly in the R for Data Science book here: http://r4ds.had.co.nz/vectors.html#recursive-vectors-lists

    One way to do this without a loop:

    character_vars <- lapply(x, class) == "character"
    x[, character_vars] <- lapply(x[, character_vars], zap_empty)