I am using the following code in R :
df$max_col<- sapply(df$col, function(x) ifelse(x == "", 0, strsplit(as.character(x), "", perl = TRUE)[[1]] %>% as.numeric %>% max(na.rm = T)))
This code basically breaks a string like "123456" and turns it into numeric and returns the max value from it. Now, I have a column full of strings like these and this code runs fine till the data size is low. But when the data size is 25 million rows (which I currently am dealing with) this code becomes very slow. Is there any alternative for this code through which I can get the max value from the string stored in a new column?
An answer based on my comment above (but I've modified the code so that it actually works):
x <- c("123", "224", "221", "1912323", "445")
apply(sapply(1:9, function(p) grepl(p, x)), 1, function(k) max(which(k)))
# the above will work if 0 is never the largest number in any cell
A more generalized version:
doit <- function(x) apply(sapply(0:9, function(p) grepl(p, x)), 1, function(k) max(which(k)))-1
x <- c("123", "224", "221", "1912323", "445", "000")
doit(x)
# [1] 3 4 2 9 5 0
This is about 3 times faster than the original code using strsplit ... but I'm sure there is room for improvement. Umm ... actually, I'm gonna try with strsplit again:
doit3 <- function(.) sapply(strsplit(.,""), max)
doit3(x)
# [1] "3" "4" "2" "9" "5" "0"
This is about 5 times faster than my previous approach. So the problem was not in sapply or strsplit but in the other components. If you need to convert it to numeric, add as.numeric to the outer layer, this won't take much extra time:
doit4 <- function(.) as.numeric(sapply(strsplit(.,""), max))
> doit4(x)
# [1] 3 4 2 9 5 0