Search code examples
rtidyrdplyrspread

Transpose dplyr::tbl object


I am using src_postgres to connect and dplyr::tbl function to fetch data from redshift database. I have applied some filters and top function to it using the dplyr itself. Now my data looks as below:

   riid   day         hour 
   <dbl> <chr>       <chr>
 1 5542. "THURSDAY " 12   
 2 5862. "FRIDAY   " 15   
 3 5982. "TUESDAY  " 15   
 4 6022. WEDNESDAY   16 

My final output should be as below:
riid    MON   TUES  WED   THUR   FRI   SAT  SUN
5542                       12
5862                             15
5988           15
6022                 16

I have tried spread. It throws the below error because of the class type:

Error in UseMethod("spread_") : no applicable method for 'spread_' applied to an object of class "c('tbl_dbi', 'tbl_sql', 'tbl_lazy', 'tbl')"

Since this is a really big table, I do not want to use dataframe as it takes a longer time. I was able to use as below:

df_mon <- df2 %>% filter(day == 'MONDAY') %>% mutate(MONDAY = hour) %>% select(riid,MONDAY)
df_tue <- df2 %>% filter(day == 'TUESDAY') %>% mutate(TUESDAY = hour) %>% select(riid,TUESDAY)
df_wed <- df2 %>% filter(day == 'WEDNESDAY') %>% mutate(WEDNESDAY = hour) %>% select(riid,WEDNESDAY)
df_thu <- df2 %>% filter(day == 'THURSDAY') %>% mutate(THURSDAY = hour) %>% select(riid,THURSDAY)
df_fri <- df2 %>% filter(day == 'FRIDAY') %>% mutate(FRIDAY = hour) %>% select(riid,FRIDAY)

Is it possible to write all above in one statement?

Any help to transpose this in a faster manner is really appreciated.

EDIT Adding the dput of the tbl object:

structure(list(src = structure(list(con = <S4 object of class structure("PostgreSQLConnection", package = "RPostgreSQL")>, 
    disco = <environment>), .Names = c("con", "disco"), class = c("src_dbi", 
"src_sql", "src")), ops = structure(list(name = "select", x = structure(list(
    name = "filter", x = structure(list(name = "filter", x = structure(list(
        name = "group_by", x = structure(list(x = structure("SELECT riid,day,hour,sum(weightage) AS score FROM\n  (SELECT riid,day,hour,\n  POWER(2,(cast(datediff (seconds,convert_timezone('UTC','PKT',SYSDATE),TO_DATE(TO_CHAR(event_captured_dt,'mm/dd/yyyy hh24:mi:ss'),'mm/dd/yyyy hh24:mi:ss')) as decimal) / cast(7862400 as decimal))) AS weightage\n  FROM (\n  SELECT riid,convert_timezone('GMT','PKT',event_captured_dt) AS EVENT_CAPTURED_DT,\n  TO_CHAR(convert_timezone('GMT','PKT',event_captured_dt),'DAY') AS day,\n  TO_CHAR(convert_timezone('GMT','PKT',event_captured_dt),'HH24') AS hour\n  FROM Zameen_STO_DATA WHERE EVENT_CAPTURED_DT >= TO_DATE((sysdate -30),'yyyy-mm-dd') and LIST_ID = 4282\n  )) group by riid,day,hour", class = c("sql", 
        "character")), vars = c("riid", "day", "hour", "score"
        )), .Names = c("x", "vars"), class = c("op_base_remote", 
        "op_base", "op")), dots = structure(list(riid = riid, 
            day = day), .Names = c("riid", "day")), args = structure(list(
            add = FALSE), .Names = "add")), .Names = c("name", 
    "x", "dots", "args"), class = c("op_group_by", "op_single", 
    "op")), dots = structure(list(~min_rank(desc(~score)) <= 
        1), .Names = ""), args = list()), .Names = c("name", 
    "x", "dots", "args"), class = c("op_filter", "op_single", 
    "op")), dots = structure(list(~row_number() == 1), .Names = ""), 
    args = list()), .Names = c("name", "x", "dots", "args"), class = c("op_filter", 
"op_single", "op")), dots = structure(list(~riid, ~day, ~hour), class = "quosures", .Names = c("", 
"", "")), args = list()), .Names = c("name", "x", "dots", "args"
), class = c("op_select", "op_single", "op"))), .Names = c("src", 
"ops"), class = c("tbl_dbi", "tbl_sql", "tbl_lazy", "tbl"))

Solution

  • I think what you're looking for is the ability to run the tidyr::spread() function against a remote source, or database. I have a PR for dbplyr that attempts to implement that here: https://github.com/tidyverse/dbplyr/pull/72, you can try it out by using: devtools::install_github("tidyverse/dbplyr", ref = devtools::github_pull(72)).