Search code examples
dataframecsvjuliaexport-to-csv

Exporting to CSV files in Julia

Warning: these are noob questions, since I'm really new to Julia.

In R, there is quite an "unified" function to export (almost) any kind of object with read.table(). It looks like things are slightly more complicated in Julia. If I understand well:

  1. Some standard types (e.g., arrays, dictionaries and tuples) can always be exported with writedlm, but not always with CSV.write. Conversely, DataFrames can always be exported with CSV.write, but never with writedlm. Is this correct? Thus, there is no "universal exporter", in a similar sense of R's write.table()?

  2. Along with writing a CSV file, CSV.write also seems to return the name of the exported file. Conversely, writedlm does not. This is a problem for me. Actually, I would need a way to export a DataFrame into a CSV file with a function which does not return a value, i.e. a function which only has side effects, like writedlm. Is there any way to achieve this in Julia?

    EDIT: To go into more details, my problem is that after a CSV.write, ans points towards the name of the exported file; and this is not the case with writedlm. An illustration here and here. Even with the idea given by @Przemyslaw Szufel, I can't get rid of this. (This is quite a subtle problem, but I'm actually trying to write a emacs lisp backend for Julia. This kind of inconsistencies, e.g. not knowing whether ans will be nothing or a filename after exporting an object, adds some more pain in this adventure... :-) Ideally, I just had wanted that CSV.write could be silent.)

Thanks!

Solution

  • I start with the second question because it is short. Just use semicolon ; at the end of line and no value will be returned!

    CSV.write(file, table);

    However if you want to be sure that ans has no value add nothing at the end:

    CSV.write(file, table);nothing;

    if you want you can pack it to a function:

    function my_write(file, table)
    CSV.write(file, table)
    nothing
end

    Or, if you want a one-liner wrap it into a lambda:

    julia> (() -> begin;CSV.write("file.csv", df);nothing;end)()

julia> ans == nothing
true

    In each case no side effect (returned value) will be observed.

    The first question is more tricky. The possible formats used for storing data in a file obviously need to depend on the format of the data. Basically the most common options for Julia include (I start from the most generic ones and finish on the most specific ones):

    • serialization via serialize command
    • binary JSON via the BSON.jl package
    • JSON via JSON.jl or slightly newer JSON3.jl package (both make a good choice as of today)
    • JSONTables.jl for tabular data stored as JSON
    • DelimitedFiles for storing Arrays
    • CSV.jl for storing DataFrames

    Now the choice of a particular package will depend on what is your objective. Serialization is the killer mechanism - fastest and most versatile. Any object can be stored this way in the shortest time possible. Nothing comes without the cost - when you update Julia version or your packages you might not be able to read your object back. So it is designed for short term storage.

    In the middle of the stake are JSON-based storage systems. Basically, everything can be stored as JSON and can be read later in Julia or other programming languages. Text based JSON can be also opened in an simple text editor.

    Finally, CSV.jl and Serialization can be used for tables and arrays respectively. However it is easy to convert a DataFrame to an Array or an Array to a DataFrame:

    julia> df = DataFrame(a=1:3, b=rand(3),c=["a","b","c"])
3×3 DataFrame
│ Row │ a     │ b        │ c      │
│     │ Int64 │ Float64  │ String │
├─────┼───────┼──────────┼────────┤
│ 1   │ 1     │ 0.440796 │ a      │
│ 2   │ 2     │ 0.44232  │ b      │
│ 3   │ 3     │ 0.282064 │ c      │

julia> Matrix(df)
3×3 Array{Any,2}:
 1  0.440796  "a"
 2  0.44232   "b"
 3  0.282064  "c"

    You can see that the only thing lost in the process is the type information which does not matter so much if you are exporting the data via DelimitedFiles.

    The conversion other way around is also straightforward:

    julia> rand(3,3) |> Tables.table |> DataFrame
3×3 DataFrame
│ Row │ Column1  │ Column2   │ Column3  │
│     │ Float64  │ Float64   │ Float64  │
├─────┼──────────┼───────────┼──────────┤
│ 1   │ 0.326649 │ 0.0278134 │ 0.111221 │
│ 2   │ 0.769378 │ 0.996156  │ 0.237821 │
│ 3   │ 0.802094 │ 0.726497  │ 0.619013 │

    In conclusion, as you can see, everything can be done in Julia.