Search code examples
rubykeyhashtableuniqueprefix

Efficient Ruby code to find the shortest prefix for each word that is unique among the words

I have similar hash tables (but much bigger ones):

h={
  "Wilhelm_Conrad_Röntgen": "http://www.example.com/w2xtt4w/001_1901.jpg",
  "Hendrik_Lorentz": "http://www.example.com/apbfksz/004_1902.jpg",
  "Pieter_Zeeman": "http://www.example.com/d2cpwj3/007_1902.jpg",
  "Antoine_Henri_Becquerel": "http://www.example.com/g8sueyg/010_1903.jpg",
  "Maria_Skłodowska-Curie": "http://www.example.com/gfcgur8/013_1903.jpg",
  "Lord_Rayleigh": "http://www.example.com/dcjiwq8/016_1904.jpg",
  "Joseph_John_Thomson": "http://www.example.com/4a66bc9/019_1906.jpg",
  "Gabriel_Lippmann": "http://www.example.com/xjefgoa/022_1908.jpg",
  "Guglielmo_Marconi": "http://www.example.com/x359w62/025_1909.jpg",
  "Karl_Ferdinand_Braun": "http://www.example.com/1edm469/028_1909.jpg",
  "Johannes_Diderik_van_der_Waals": "http://www.example.com/31hpue7/031_1910.jpg",
  "Wilhelm_Wien": "http://www.example.com/yel9iff/034_1911.jpg",
  "Nils_Gustaf_Dalén": "http://www.example.com/iezss59/037_1912.jpg",
  "Heike_Kamerlingh-Onnes": "http://www.example.com/8zl4uj2/040_1913.jpg",
  "Max_von_Laue": "http://www.example.com/ta3w6rn/043_1914.jpg",
  "William_Henry_Bragg": "http://www.example.com/u43qn5h/046_1915.jpg",
  "William_Lawrence_Bragg": "http://www.example.com/n7gkk6e/049_1915.jpg",
  "Charles_Glover_Barkla": "http://www.example.com/2kxxroc/052_1917.jpg",
  "Max_Planck": "http://www.example.com/eyw7tm6/055_1918.jpg",
  "Johannes_Stark": "http://www.example.com/gcjcv2p/058_1919.jpg",
  "Charles_Édouard_Guillaume": "http://www.example.com/nox3s6o/061_1920.jpg",
  "Niels_Bohr": "http://www.example.com/mo9ga29/064_1922.jpg",
  "Robert_Andrews_Millikan": "http://www.example.com/kxq71if/067_1923.jpg",
  "Manne_Siegbahn": "http://www.example.com/2uwhw9y/070_1924.jpg",
  "James_Franck": "http://www.example.com/iwdavip/073_1925.jpg",
  "Gustav_Hertz": "http://www.example.com/73mbso2/076_1925.jpg",
  "Jean_Baptiste_Perrin": "http://www.example.com/rgugmas/079_1926.jpg",
  "Arthur_Holly_Compton": "http://www.example.com/vy7is1v/082_1927.jpg",
  "Owen_Willans_Richardson": "http://www.example.com/3jz5ve8/085_1928.jpg",
  "Louis_Victor_Pierre_Raymond": "http://www.example.com/srvj8d5/088_1929.jpg",
  "John_M_Kosterlitz": "http://www.example.com/7op2wb1/091_1929.jpg",
  "Chandrasekhara_Venkata_Raman": "http://www.example.com/1vqqwua/094_1930.jpg"
}

I would like to find the shortest prefix in an efficient way for each key which prefix is unique among the given keys. In other words: having the shorter prefixes each row is still recognizable in the hash table. For this hash table the shortest prefixes which still makes the lines recognizable:

h={
  "Wilhelm_C": "http://www.example.com/w2xtt4w/001_1901.jpg",
  "Hen": "http://www.example.com/apbfksz/004_1902.jpg",
  "P": "http://www.example.com/d2cpwj3/007_1902.jpg",
  "An": "http://www.example.com/g8sueyg/010_1903.jpg",
  "Mar": "http://www.example.com/gfcgur8/013_1903.jpg",
  "Lor": "http://www.example.com/dcjiwq8/016_1904.jpg",
  "Jos": "http://www.example.com/4a66bc9/019_1906.jpg",
  "Ga": "http://www.example.com/xjefgoa/022_1908.jpg",
  "Gug": "http://www.example.com/x359w62/025_1909.jpg",
  "K": "http://www.example.com/1edm469/028_1909.jpg",
  "Johannes_D": "http://www.example.com/31hpue7/031_1910.jpg",
  "Wilhelm_W": "http://www.example.com/yel9iff/034_1911.jpg",
  "Nil": "http://www.example.com/iezss59/037_1912.jpg",
  "Hei": "http://www.example.com/8zl4uj2/040_1913.jpg",
  "Max_v": "http://www.example.com/ta3w6rn/043_1914.jpg",
  "William_H": "http://www.example.com/u43qn5h/046_1915.jpg",
  "William_L": "http://www.example.com/n7gkk6e/049_1915.jpg",
  "Charles_G": "http://www.example.com/2kxxroc/052_1917.jpg",
  "Max_P": "http://www.example.com/eyw7tm6/055_1918.jpg",
  "Johannes_S": "http://www.example.com/gcjcv2p/058_1919.jpg",
  "Charles_É": "http://www.example.com/nox3s6o/061_1920.jpg",
  "Nie": "http://www.example.com/mo9ga29/064_1922.jpg",
  "R": "http://www.example.com/kxq71if/067_1923.jpg",
  "Man": "http://www.example.com/2uwhw9y/070_1924.jpg",
  "Ja": "http://www.example.com/iwdavip/073_1925.jpg",
  "Gus": "http://www.example.com/73mbso2/076_1925.jpg",
  "Je": "http://www.example.com/rgugmas/079_1926.jpg",
  "Ar": "http://www.example.com/vy7is1v/082_1927.jpg",
  "O": "http://www.example.com/3jz5ve8/085_1928.jpg",
  "Lou": "http://www.example.com/srvj8d5/088_1929.jpg",
  "John": "http://www.example.com/7op2wb1/091_1929.jpg",
  "Chan": "http://www.example.com/1vqqwua/094_1930.jpg"
}

My first attempt to find the shortest prefixes for each key or word:

ret=h.keys.map{|k| 
   l=1; 
   while h.keys.select{|z| z=~/^#{Regexp.quote(k[0..l-1])}/}.size != 1 do
    l+=1
   end
   puts "#{k} -> #{k[0..l-1]}"
   [k[0..l-1],h[k]]
};

Algorithm and code is far from optimal, in fact it is really slow even on a fast machine, and when hash table is much bigger then it is unusable.

Example hash can be fetched:

require "json"
h=JSON.load(%x{curl http://pastebin.com/raw/Cs0dTJmA})

A quick benchmark of two different method:

generating all prefices, mark them as good / bad - ambigous a faster method which generates only short prefices

X-axis is for size of input hash / array, Y-axis is for running time in seconds.

Final code, now the fastest and not so memory hungry:

def optimize_keys(ih)

    h=ih.dup #for not messing with the original
    result={}
    bh={}
    l=1
    ks=h.keys.size

    while result.size < ks do

         h.keys.each{|k| 
            prefix=k[0..l-1]
            bh[prefix]==nil ? bh[prefix]=[1,k] : bh[prefix][0]+=1
         }

         ones=bh.select{|k,v| v[0]==1}

         ones.each{|k,v| 
              result[k]=h[v[1]]
              h.delete(v[1])
         }

         bh={}
         l+=1
    end

    return result
end
Solution

  • Here is an algorithm

    words = %w{ruby rules}

hash1 = {}
words.each do |word|
  abbr = word
  until abbr.empty?
    hash1[abbr] = hash1[abbr] ? :ambigous : word
    break if hash1[abbr] == :ambigous 
    abbr = abbr.chop
  end
end
hash2 = {}
hash1.each { |abbr, word| hash2[word] = abbr }
hash2.delete(:ambigous)

p hash2

    How does this work?

    • Generate all the possible prefices, mapping them to either the whole word or :ambigous
    • Reverse the hash and since prefices are in descending order of length the reversed hash will map to the shortest prefix
    • Remove the "word" :ambigous from the reversed hash