Search code examples
rlistdata.tablefwrite

data.table R ; how to preserve list element name by fwrite?


I have a following list of data.tables;

l[1:2]
$cluster1
           rn        p_val avg_log2FC pct.1 pct.2    p_val_adj
       <char>        <num>      <num> <num> <num>        <num>
  1: Adamts20 1.438101e-65   1.030705 0.001 0.118 4.314302e-62
  2:      Gal 3.233897e-54  16.358484 0.006 0.208 9.701690e-51
  3:     Hrnr 1.295286e-51  14.374645 0.007 0.182 3.885858e-48
  4:      Ngb 4.143060e-44   1.925631 0.021 0.240 1.242918e-40
  5:    Svep1 3.872262e-41   8.232291 0.283 0.582 1.161679e-37
 ---                                                          
366:   Cyfip2 1.468792e-05   2.417948 0.056 0.118 4.406375e-02
367:   Fcgr2b 1.505992e-05  -4.719973 0.231 0.148 4.517976e-02
368:      Ogn 1.538930e-05   9.708295 0.242 0.339 4.616791e-02
369:   Lrrc15 1.549036e-05  -2.966498 0.241 0.352 4.647109e-02
370: AY036118 1.624054e-05   2.602963 0.715 0.788 4.872162e-02

$cluster2
                rn         p_val avg_log2FC pct.1 pct.2     p_val_adj
            <char>         <num>      <num> <num> <num>         <num>
  1:        Samd13 1.841651e-127  8.7785032 0.001 0.144 5.524952e-124
  2:         Gata1  2.210414e-99 17.0522687 0.002 0.126  6.631242e-96
  3:        Col4a4  1.250827e-58  9.6145811 0.583 0.254  3.752481e-55
  4:           Bsx  4.028430e-54 18.2214242 0.202 0.568  1.208529e-50
  5:        Gimap3  4.650536e-53 -1.6106639 0.017 0.255  1.395161e-49
 ---                                                                 
217:          Cd3e  1.452524e-05 18.6983055 0.131 0.175  4.357573e-02
218:       Selenom  1.455810e-05  0.9648215 0.293 0.201  4.367429e-02
219: A330069E16Rik  1.469167e-05 -5.4387846 0.388 0.282  4.407501e-02
220:         Lypd1  1.560834e-05 -2.1249661 0.176 0.100  4.682501e-02
221:          Alad  1.633998e-05 -1.1929284 0.226 0.175  4.901995e-02

I'd like to export this list to a csv file by fwrite. I run this following code;

lapply(l[1:2], function(x) fwrite(as.data.table(x), "DGE.csv"))

A resulting csv file doesn't have list element names, $cluster1 and $cluster. Are there any ways to add list element names as the header of each of the element list and column headers as a follow desired output by using fwrite ?;

cluster1
   rn   p_val   avg_log2FC  pct.1   pct.2   p_val_adj
Samd13  1.84E-127   8.778503219 0.001   0.144   5.52E-124
Gata1   2.21E-99    17.05226868 0.002   0.126   6.63E-96
Col4a4  1.25E-58    9.614581115 0.583   0.254   3.75E-55
Bsx     4.03E-54    18.22142419 0.202   0.568   1.21E-50
Gimap3  4.65E-53    -1.61066387 0.017   0.255   1.40E-49
Gm26825 9.54E-51    12.96067221 0.341   0.596   2.86E-47
Erfe    3.72E-48    11.74946067 0.007   0.156   1.12E-44




cluster2
    rn  p_val   avg_log2FC  pct.1   pct.2   p_val_adj
Samd13  1.63E-12    15.05539372 0.038   0.141   4.90E-09
Frmd5   1.75E-12    -1.28277874 0.154   0.318   5.26E-09
Pla2g2f 2.06E-12    -1.51234633 0.142   0.056   6.17E-09
Aldh1a2 2.25E-12    5.127605606 0.274   0.426   6.75E-09
Fxyd2   2.51E-12    5.246386986 0.626   0.805   7.53E-09
Adam30  3.15E-12    1.53746735  0.203   0.362   9.44E-09
Gpr35   3.15E-12    -3.18282304 0.691   0.516   9.46E-09
Il10rb  3.65E-12    1.473277609 0.42    0.598   1.10E-08
Gk5     3.72E-12    -14.0221264 0.177   0.074   1.12E-08
    .......

Solution

  • An alternative to writing separate files: append the list-name as a column, then rbindlist them, write them, and when you read them then split on that column name.

    Sample data:

    library(data.table)
    ZZ <- lapply(list(mt1=mtcars[1:3,], mt2=mtcars[4:6,]), as.data.table)
    ZZ
    # $mt1
    #      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
    #    <num> <num> <num> <num> <num> <num> <num> <num> <num> <num> <num>
    # 1:  21.0     6   160   110  3.90 2.620 16.46     0     1     4     4
    # 2:  21.0     6   160   110  3.90 2.875 17.02     0     1     4     4
    # 3:  22.8     4   108    93  3.85 2.320 18.61     1     1     4     1
    # $mt2
    #      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
    #    <num> <num> <num> <num> <num> <num> <num> <num> <num> <num> <num>
    # 1:  21.4     6   258   110  3.08 3.215 19.44     1     0     3     1
    # 2:  18.7     8   360   175  3.15 3.440 17.02     0     0     3     2
    # 3:  18.1     6   225   105  2.76 3.460 20.22     1     0     3     1
    

    Append, combine, write:

    fwrite(rbindlist(ZZ, idcol = "name"), "~/Downloads/ZZ.csv")
    

    The resulting file, that can be read by any vanilla CSV reader:

    name,mpg,cyl,disp,hp,drat,wt,qsec,vs,am,gear,carb
    mt1,21,6,160,110,3.9,2.62,16.46,0,1,4,4
    mt1,21,6,160,110,3.9,2.875,17.02,0,1,4,4
    mt1,22.8,4,108,93,3.85,2.32,18.61,1,1,4,1
    mt2,21.4,6,258,110,3.08,3.215,19.44,1,0,3,1
    mt2,18.7,8,360,175,3.15,3.44,17.02,0,0,3,2
    mt2,18.1,6,225,105,2.76,3.46,20.22,1,0,3,1
    

    Reading it in and restoring the original object format:

    ZZ2 <- fread("ZZ.csv")
    ZZ2 <- lapply(split(ZZ2, ZZ2$name), function(x) x[, name := NULL])
    ZZ2
    # $mt1
    #      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
    #    <num> <int> <int> <int> <num> <num> <num> <int> <int> <int> <int>
    # 1:  21.0     6   160   110  3.90 2.620 16.46     0     1     4     4
    # 2:  21.0     6   160   110  3.90 2.875 17.02     0     1     4     4
    # 3:  22.8     4   108    93  3.85 2.320 18.61     1     1     4     1
    # $mt2
    #      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
    #    <num> <int> <int> <int> <num> <num> <num> <int> <int> <int> <int>
    # 1:  21.4     6   258   110  3.08 3.215 19.44     1     0     3     1
    # 2:  18.7     8   360   175  3.15 3.440 17.02     0     0     3     2
    # 3:  18.1     6   225   105  2.76 3.460 20.22     1     0     3     1