Search code examples
elixirets

How to retrieve a list of ets keys without scanning entire table?


I'm using ets via elixir as a simple in-memory persistence layer to store and retrieve keys and also for the occasional foldl which involves reducing many duplicate keys that have different values. I am using the bag option.

Is there a simple, perhaps O(1) way to retrieve a list of just the current keys without having to do a more involved table traversal or match or foldl?

Erlang or Elixir syntax responses welcome.

:ets.new(:cache, [:bag, :named_table, :protected])

I have a static map of atom keys indexed by integer I am using to assist with insertions. But not all the keys are used..

chunk_key_map = %{2 => :chunk_key_2, ..... 28 => :chunk_key_28}

If there's no quick way, I am aware I can do an ets:lookup trying each of my static atom key values and testing for != [] and generating my own list, but wanted to see if ets supports such a feature.

Thanks


Solution

  • So I didn't find the ets technique, but rather implemented the key list retrieve code in constant time in elixir as my key map is static.

        list = Enum.reduce(2..28, [], fn head, acc -> 
                case :ets.lookup(:cache, Map.get(chunk_key_map, head)) do
                    [] -> acc
                    _ -> [acc, head]
                end
            end)
    
        List.flatten(list)
    

    UPDATE: Based on the replies, I took Hamidreza's ets traversal logic and wrapped it into an Elixir Stream using Stream.resource/3.

    defp get_ets_keys_lazy(table_name) when is_atom(table_name) do
        eot = :"$end_of_table"
    
        Stream.resource(
            fn -> [] end,
    
            fn acc ->
                case acc do
                    [] -> 
                        case :ets.first(table_name) do
                            ^eot -> {:halt, acc}
                            first_key -> {[first_key], first_key}                       
                        end
    
                    acc -> 
                        case :ets.next(table_name, acc) do  
                            ^eot -> {:halt, acc}
                            next_key -> {[next_key], next_key}
                        end
                end
            end,
    
            fn _acc -> :ok end
        )
    end
    

    Then I ran the stream through a pipeline

    get_ets_keys_lazy(table_name) 
        |> Stream.map(lambda1) 
        |> Stream.each(lambda2)
        |> Stream.run