Search code examples
rdata.tableshift

Is it currently possible to shift a variable number of rows in data.table?


First I'd like to thank the creators and maintainers of the R package data.table.

I have a simple question on the 'shift' feature in data.table. Is there a way not to enter an hardcoded offset but rather a variable offset stored e.g. in a column? I.e. on each row the offset could be different.

I didn't see anything in this direction in the current documentation.

Thank you very much in advance for your kind help.

The code below doesn't work (a vector is displayed instead):

DT[, Observation_Annual_Offset := shift(Observation, n = ifelse (Frequency == "A (Annual)", 1, ifelse (Frequency == "H (Half-yearly)", 2, ifelse (Frequency == "Q (Quarterly)", 4, ifelse(Frequency == "M (Monthly)", 12, 0)))), fill = NA, type = "lead")]

Solution

  • It would be better if you store the frequency in a vector to be referenced rather than using multiple ifelse:

    freq <- c("A (Annual)"=1, "H (Half-yearly)"=2, "Q (Quarterly)"=4, "M (Monthly)"=12)
    DT <- data.table(Observation=rep(1:25, length(freq)), Offset=rep(names(freq), each=25))
    DT[, Observation_Annual_Offset := shift(Observation, -freq[Offset[1L]]), Offset]
    

    data:

         Observation          Offset Observation_Annual_Offset
      1:           1      A (Annual)                         2
      2:           2      A (Annual)                         3
      3:           3      A (Annual)                         4
      4:           4      A (Annual)                         5
      5:           5      A (Annual)                         6
      6:           6      A (Annual)                         7
      7:           7      A (Annual)                         8
      8:           8      A (Annual)                         9
      9:           9      A (Annual)                        10
     10:          10      A (Annual)                        11
     11:          11      A (Annual)                        12
     12:          12      A (Annual)                        13
     13:          13      A (Annual)                        14
     14:          14      A (Annual)                        15
     15:          15      A (Annual)                        16
     16:          16      A (Annual)                        17
     17:          17      A (Annual)                        18
     18:          18      A (Annual)                        19
     19:          19      A (Annual)                        20
     20:          20      A (Annual)                        21
     21:          21      A (Annual)                        22
     22:          22      A (Annual)                        23
     23:          23      A (Annual)                        24
     24:          24      A (Annual)                        25
     25:          25      A (Annual)                        NA
     26:           1 H (Half-yearly)                         3
     27:           2 H (Half-yearly)                         4
     28:           3 H (Half-yearly)                         5
     29:           4 H (Half-yearly)                         6
     30:           5 H (Half-yearly)                         7
     31:           6 H (Half-yearly)                         8
     32:           7 H (Half-yearly)                         9
     33:           8 H (Half-yearly)                        10
     34:           9 H (Half-yearly)                        11
     35:          10 H (Half-yearly)                        12
     36:          11 H (Half-yearly)                        13
     37:          12 H (Half-yearly)                        14
     38:          13 H (Half-yearly)                        15
     39:          14 H (Half-yearly)                        16
     40:          15 H (Half-yearly)                        17
     41:          16 H (Half-yearly)                        18
     42:          17 H (Half-yearly)                        19
     43:          18 H (Half-yearly)                        20
     44:          19 H (Half-yearly)                        21
     45:          20 H (Half-yearly)                        22
     46:          21 H (Half-yearly)                        23
     47:          22 H (Half-yearly)                        24
     48:          23 H (Half-yearly)                        25
     49:          24 H (Half-yearly)                        NA
     50:          25 H (Half-yearly)                        NA
     51:           1   Q (Quarterly)                         5
     52:           2   Q (Quarterly)                         6
     53:           3   Q (Quarterly)                         7
     54:           4   Q (Quarterly)                         8
     55:           5   Q (Quarterly)                         9
     56:           6   Q (Quarterly)                        10
     57:           7   Q (Quarterly)                        11
     58:           8   Q (Quarterly)                        12
     59:           9   Q (Quarterly)                        13
     60:          10   Q (Quarterly)                        14
     61:          11   Q (Quarterly)                        15
     62:          12   Q (Quarterly)                        16
     63:          13   Q (Quarterly)                        17
     64:          14   Q (Quarterly)                        18
     65:          15   Q (Quarterly)                        19
     66:          16   Q (Quarterly)                        20
     67:          17   Q (Quarterly)                        21
     68:          18   Q (Quarterly)                        22
     69:          19   Q (Quarterly)                        23
     70:          20   Q (Quarterly)                        24
     71:          21   Q (Quarterly)                        25
     72:          22   Q (Quarterly)                        NA
     73:          23   Q (Quarterly)                        NA
     74:          24   Q (Quarterly)                        NA
     75:          25   Q (Quarterly)                        NA
     76:           1     M (Monthly)                        13
     77:           2     M (Monthly)                        14
     78:           3     M (Monthly)                        15
     79:           4     M (Monthly)                        16
     80:           5     M (Monthly)                        17
     81:           6     M (Monthly)                        18
     82:           7     M (Monthly)                        19
     83:           8     M (Monthly)                        20
     84:           9     M (Monthly)                        21
     85:          10     M (Monthly)                        22
     86:          11     M (Monthly)                        23
     87:          12     M (Monthly)                        24
     88:          13     M (Monthly)                        25
     89:          14     M (Monthly)                        NA
     90:          15     M (Monthly)                        NA
     91:          16     M (Monthly)                        NA
     92:          17     M (Monthly)                        NA
     93:          18     M (Monthly)                        NA
     94:          19     M (Monthly)                        NA
     95:          20     M (Monthly)                        NA
     96:          21     M (Monthly)                        NA
     97:          22     M (Monthly)                        NA
     98:          23     M (Monthly)                        NA
     99:          24     M (Monthly)                        NA
    100:          25     M (Monthly)                        NA
         Observation          Offset Observation_Annual_Offset