Search code examples
pythonxhtmltesseracthocr

Extract data from tesseract hocr xhtml file


I'm trying to use Python to extract data from Tesseract's hocr output file. We're limited to tesseact version 3.04, so no image_to_data function or tsv output is available. I have been able to do it with beautifulsoup and in R, but that's neither are available in the environment in which it needs to be deployed. I am just trying to extract the word and confidence "x_wconf." An example output file is below, for which I'd be happy to just return lists of [90, 87, 89, 89] and ['the', '(quick)', '[brown]', '{fox}', 'jumps!'].

lxml is the only available xml parser outside of the elementtree in the environment so I'm a bit at a loss for how to proceed.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <title></title>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
  <meta name='ocr-system' content='tesseract 3.05.00dev' />
  <meta name='ocr-capabilities' content='ocr_page ocr_carea ocr_par ocr_line ocrx_word'/>
</head>
<body>
  <div class='ocr_page' id='page_1' title='image "./testing/eurotext.png"; bbox 0 0 1024 800; ppageno 0'>
   <div class='ocr_carea' id='block_1_1' title="bbox 98 66 918 661">
    <p class='ocr_par' id='par_1_1' lang='eng' title="bbox 98 66 918 661">
     <span class='ocr_line' id='line_1_1' title="bbox 105 66 823 113; baseline 0.015 -18; x_size 39; x_descenders 7; x_ascenders 9"><span class='ocrx_word' id='word_1_1' title='bbox 105 66 178 97; x_wconf 90'>The</span> <span class='ocrx_word' id='word_1_2' title='bbox 205 67 347 106; x_wconf 87'><strong>(quick)</strong></span> <span class='ocrx_word' id='word_1_3' title='bbox 376 69 528 109; x_wconf 89'>[brown]</span> <span class='ocrx_word' id='word_1_4' title='bbox 559 71 663 110; x_wconf 89'>{fox}</span> <span class='ocrx_word' id='word_1_5' title='bbox 687 73 823 113; x_wconf 89'>jumps!</span> 
     </span>
    </p>
   </div>
  </div>
 </body>
</html>

Solution

  • Figured out a (gross) way to do it using xpath.

    def hocr_to_dataframe(fp):
    
        from lxml import etree
        import pandas as pd
        import os
    
        doc = etree.parse('fp')
        words = []
        wordConf = []
    
        for path in doc.xpath('//*'):
            if 'ocrx_word' in path.values():
                conf = [x for x in path.values() if 'x_wconf' in x][0]
                wordConf.append(int(conf.split('x_wconf ')[1]))
                words.append(path.text)
    
        dfReturn = pd.DataFrame({'word' : words,
                                 'confidence' : wordConf})
    
        return(dfReturn)