Search code examples
rreplacenas

Replacing NAs in R dataframe with mean based on group and apply the same to multiple columns


I have this dataframe.

library(tidyverse)
df <- tibble(
  "group" = c("A", "A", "B", "B"),
  "WC" = c(NA, 2.3, 3.5, 4),
  "Sixltr" = c(3.3, NA, NA, 2.7),
  "Dic" = c(NA, NA, NA, 2.4),
  "I" = c(3.1, 3, 2.7, 1.9),
  "We" = c(4.6, NA, 2.2, NA)
)

I have created the mean_NA_conditional_function function to replace the NAs with the mean (based on some conditions), and then I use lapply to do the same on all columns of the dataframe - this is, however, not important, I could also have simply used the regular mean.

mean_NA_conditional_function <- function(vector) {
  # when NA <= 1 in vector, return the mean of available data in vector
  if (sum(is.na(vector)) <= 1) {return(mean(vector, na.rm = TRUE))}
  # when NA >= 2 in vector, return the sum of available data in vector divided by vector length - 1
  if (sum(is.na(vector)) >= 2) {return((sum(vector, na.rm = TRUE)) / (length(vector) - 1))}
}

#Create the 'NAs_replace_function' function that replaces NAs applying the 'mean_NA_conditional_function'. 
NAs_replace_function <- function(vector) replace(vector, is.na(vector), mean_NA_conditional_function(vector))

#Apply the function 'NAs_replace_function' to selected columns and replace NAs with appropriate mean.
df_after_imputation <- replace(df, TRUE, lapply(df, NAs_replace_function))

So far, this works. But, what I want to do is to replace the NA based on the group that each value belongs to (i.e., 'A', 'B'). I tried to group_by(), but it didn't work. Not sure if I made something wrong. Any ideas on how to solve this?

# This doesn't work:
df_after_imputation <- df %>% group_by(group) %>% replace(., TRUE, lapply(df, NAs_replace_function))

Solution

  • You can use :

    library(dplyr)
    
    df %>%
      group_by(group) %>%
      mutate(across(WC:We, NAs_replace_function)) %>%
      ungroup -> df_after_imputation
    
    df_after_imputation
    
    #  group    WC Sixltr   Dic     I    We
    #  <chr> <dbl>  <dbl> <dbl> <dbl> <dbl>
    #1 A       2.3    3.3   0     3.1   4.6
    #2 A       2.3    3.3   0     3     4.6
    #3 B       3.5    2.7   2.4   2.7   2.2
    #4 B       4      2.7   2.4   1.9   2.2