Erlang: Writing mnesia table to .csv one record per line

Question

I have got a Mnesia table which consists of the following format:

-record(state, {key, tuple, state, timestamp, fin_from}).

The entries look like follows (read with ets:tab2list(Tab)):

[{state,{80,43252,tcp,tcp_syn_received,{192,168,101,5},{192,168,101,89}},
        {80,43252,tcp,{192,168,101,5},{192,168,101,89}},
        tcp_syn_received,1463850419221,undefined},
 {state,{80,41570,tcp,tcp_syn_received,{192,168,101,5},{192,168,101,89}},
        {80,41570,tcp,{192,168,101,5},{192,168,101,89}},
        tcp_syn_received,1463850403214,undefined},
...]

I would like to write these data to a .csv file with one entry per line - preferred with the following format:

state,80,43252,tcp,tcp_syn_received,192.168.101.5,192.168.101.89,80,43252,tcp,192.168.101.5,192.168.101.89,tcp_syn_received,1463850419221,undefined
state,80,41570,tcp,tcp_syn_received,192.168.101.5,192.168.101.89,80,41570,tcp,192.168.101.5,192.168.101.89,tcp_syn_received,1463850419221,undefined

There should be a line break after undefined. I tried using the following code (while Content = ets:tab2list(states)):

do_logging_async(File, Format, Content, Append)->
F = fun() ->
    file:write_file(File, io_lib:fwrite(Format, [Content]), [Append])
end,
spawn(F).

However, I cannot get anything similar to my output. The data should afterwards be evaluated with R.

UPDATE: The key was to read the table line by line and parse it with ~w but not ~p. I ended up with the following solution (which produces a slightly different output, however, there is less redundant data):

do_state_logging(File, EtsAsList) ->
% write header (columnnames)
file:write_file(File, io_lib:fwrite("~w,~w,~w,~w,~w,~w,~w,~w~n", [record,dstPort,srcPort,proto,dstIP,srcIP,state,timestamp]),[append]),
case EtsAsList of
    [] ->
        ok;
    _ ->
        F = fun({Record,_Key, 
            [P1, P2, Proto, {D_Ip_1,D_Ip_2,D_Ip_3,D_Ip_4}, {S_Ip_1,S_Ip_2,S_Ip_3,S_Ip_4}],
            State, Timestamp, _}) -> 
            file:write_file(File, io_lib:fwrite("~w,", [Record]),[append]),
            file:write_file(File, io_lib:fwrite("~w,~w,~w,", [P1,P2,Proto]),[append]),
            file:write_file(File, io_lib:fwrite("~w.~w.~w.~w,", [D_Ip_1,D_Ip_2,D_Ip_3,D_Ip_4]), [append]),
            file:write_file(File, io_lib:fwrite("~w.~w.~w.~w,", [S_Ip_1,S_Ip_2,S_Ip_3,S_Ip_4]), [append]),
            file:write_file(File, io_lib:fwrite("~w,", [State]),[append]),
            file:write_file(File, io_lib:fwrite("~w", [Timestamp]),[append]),
            file:write_file(File, ["\n"],[append]) 
        end,
        lists:foreach(F, EtsAsList)
    end,
    io:format("Finished logging of statetable to file: ~p~n" , [File]).

Thanks to the answer who pushed me to this idea.

Answer 1

Assuming you change your ets records values to be in lists and not tuples you can use this code to write your ETS Table to a file.

do_logging_async(File, EtsAsList) ->
    F = fun({Key, Value}) -> 
        file:write_file(File, [atom_to_list(Key) ++ ","],[append]),
        write_value(File,lists:flatten(Value)),
        file:write_file(File, ["\n"],[append]) 
    end,
    lists:foreach(F,EtsAsList).


write_value(_File, []) -> ok;
write_value(File, [H|T]) ->
    case is_integer(H) of
        true -> file:write_file(File, [integer_to_list(H)],[append]);
        false -> file:write_file(File, [atom_to_list(H)],[append])
    end,
    case T=:=[] of
        true -> ok;
        false -> file:write_file(File, [","],[append])
    end,
    write_value(File,T).

do_logging_async/2 takes every {Key, Value} pair. First, it writes the Key to the file and then it runs write_value/2 on the Value, at the end of each pair it writes \n.

write_value/2 takes the flatten value list (assuming it is a flatten list that contains only integers and atoms) and writes it to the file.