Search code examples
rmerging-data

Trouble merging new data to each data frame element of a list


I am having difficulty using a for loop for appending new data to each data frame element of a list.

If I have a list of two data frames (filelist) and I wish to "dplyr::left_join" or "merge" each data frame in the list with other data from a single data frame, it does not seem to appear in the list afterward. However, if I use the same commands stepwise and separately for each data frame element of the list, I get the same warnings (due to missing factor levels), but the desired result. For example:

some data frames

df1 <- data.frame(x = 1:3, y=letters[1:3])
df2 <- data.frame(x = 1:5, y=letters[1:5])

# make list of dataframes
filelist <- list(df1,df2)

# new data frame to add to the data frames in the list by indexing "y"
df3 <- data.frame(animal = c(rep("snake", 7)), y=letters[1:7], geno = c("aa", "ab", "ac", "aa", "ac", "ab", "ae"))

# merge df3 into both data frames in the filelist
for (i in 1:length(filelist)) {dplyr::left_join(filelist[[i]], df3, by = "y")}

## Gives the following warning because some factor levels are missing between datasets
Warning message:
Column `y` joining factors with different levels, coercing to character vector 

returned result is the same as the original filelist

> filelist
[[1]]
  x y
1 1 a
2 2 b
3 3 c

[[2]]
  x y
1 1 a
2 2 b
3 3 c
4 4 d
5 5 e

The expected result (done by merging each element of the list separately, then making a new list)

new1 <- dplyr::left_join(filelist[[1]], df3, by = "y")
new2 <- dplyr::left_join(filelist[[2]], df3, by = "y")
newlist <-(new1,new2)
> newlist
[[1]]
  x y animal geno
1 1 a  snake   aa
2 2 b  snake   ab
3 3 c  snake   ac

[[2]]
  x y animal geno
1 1 a  snake   aa
2 2 b  snake   ab
3 3 c  snake   ac
4 4 d  snake   aa
5 5 e  snake   ac

What is the best way to do this without taking each data frame out of the original list, adding the new data, then creating a new list?


Solution

  • As says in the warning message, factors have different levels.

    You can convert factors to characters for each dataframe as follow with dplyr :

    df %>% mutate_if(is.factor, as.character) -> df
    

    Or homogenize factor levels of variable y :

    for (i in 1:length(filelist)) {
      x = factor(unique(c(levels(filelist[[i]]$y),levels(df3$y))))
      levels(filelist[[i]]$y) = x
      levels(df3$y) = x
      filelist[[i]] = dplyr::left_join(filelist[[i]], df3, by = "y")
    }