Search code examples
jsonvariablessavejuliametaprogramming

How to save all global variables from a Julia code into a file?


I am trying to save all global variables from a Julia code into a file, such that it can be read in a separate Julia code and the same values will be assigned to the global variables of the same names. I am aware that data structures like dictionaries can be easily saved into a JSON/JLD2/.. file etc. but it will be very cumbersome and require a lot of manual works to save the variables into a dictionary and to read it in another file and assign the values again (or are there any quick way to do this?). It seems that metaprogramming in Julia may provide me a solution but I am unfamiliar with it. What would be the best solution to complete such a task?

Here it is assumed that all these variables are all "conventional" data structures like floating point numbers, arrays, dictionaries etc., and do not include interpolated functions, images, etc. And it is highly preferred that these parameters can be saved into popular types of files like txt/csv/json.

For instance, in a Julia file the following variables are defined:

x = 1
y = 2
z = [x,y,[x,y]]
str = "Hello World"
tup = (2,3,(4,5))
dict = Dict([("A", 1), ("B", 2)])

So how to save the information into a file such that the variables of the same names can be assigned the same values immediately in another Julia file?

It would be, for example, acceptable for me if the variables can be turned into a dictionary in the form like Dict("x"=>1, "y"=>2, "z"=>[1,2,[1,2]]) such that it can be saved in a JSON file, provided that it is possible to turn the keys of the dictionary into variables names and their values as the values of the variables in another Julia file.

Edit: I am using the following as a temporary solution:

# The definitions of variables are declared as a string
variables_definition = """
x = 1
y = 2
z = [x,y,[x,y]]
str = "Hello World"
tup = (2,3,(4,5))
dict = Dict([("A", 1), ("B", 2)])
"""
# Writing the string into another Julia file
open("variables.jl","w") do f
    print(f, variables_definition)
end

# Read the Julia file to import the variables
include("variables.jl")

But are there more elegant solutions that do not need explicit declarations of the definitions as strings?


Solution

  • This functionality is actually provided by JLD2 although it is not the recommended way of doing things. From the JLD2 docs:

    For interactive use you can save all variables in the current module's global scope using @save filename. More permanent code should prefer the explicit form to avoid saving unwanted variables.

    With JLD2 your example would become:

    using JLD2
    # Definitions of variables
    x = 1
    y = 2
    z = [x,y,[x,y]]
    str = "Hello World"
    tup = (2,3,(4,5))
    dict = Dict([("A", 1), ("B", 2)])
    
    @save "variables.jld2"
    

    in the file using the variables

    using JLD2
    
    @load "variables.jld2"
    

    The code that achieves this can be found here. It works by calling the names (docs) function on the current module and then filtering the result to find the variables that can/should be saved. When Julia is started the current module is Main so to try this out in the REPL you can run names(Main).

    Here is a proof of principle that uses names to create a function similar to the @save macro for JSON:

    using JSON
    
    function globals_to_json(m)
        json_compat = Dict{String, Any}()
        for name in names(m)
           try
               field = getfield(m, name)
               JSON.print(field)
               println()
               json_compat[string(name)] = field
           catch
               println("Skipping $name")
           end
        end
        JSON.json(json_compat)
    end
    

    In a real use case, you probably would want a better check for encodability than this try-catch, and like already mentioned just dumping all globals is not advisable in production code.

    Here the function is used in the REPL.

    julia> x = 1; y = 2; z = [x,y,[x,y]]; str = "Hello World"; tup = (2,3,(4,5)); dict = Dict([("A", 1), ("B", 2)]);
    
    julia> json_str = globals_to_json(Main);
    Skipping Base
    Skipping Core
    Skipping InteractiveUtils
    Skipping Main
    {"B":2,"A":1}
    {"B":2,"A":1}
    Skipping globals_to_json
    "Hello World"
    [2,3,[4,5]]
    1
    2
    [1,2,[1,2]]
    
    julia> println(json_str)
    {"dict":{"B":2,"A":1},"tup":[2,3,[4,5]],"str":"Hello World","x":1,"z":[1,2,[1,2]],"ans":{"B":2,"A":1},"y":2}