Search code examples
macosapplescriptosascriptlivetext

macOS: How to access the Live Text OCR functionality from AppleScript/JXA?

As of macOS Monterey it is possible to select text in images in Preview.

Is this OCR functionality available from AppleScript and/or JXA (JavaScript for Automation)?

enter image description here

In Script Editor.app File > Open dictionary... I selected the Preview.app and looked at the API for Standard Suite and Text Suite but there doesn't seem to be anything related to OCR. (The Text Suite apparently has to do with drawing text on pictures and not with text extraction.)

I have also searched for text recognition actions in Automator.app but didn't see anything suitable.

Solution

  • You can use AppleScriptObjC and the Vision framework to extract text from images. The overall process is to obtain image data, set up an image request of the desired type (in this case, VNRecognizeTextRequest), create a corresponding image handler, perform the request, and return the resulting text strings.

    use framework "Vision"

on getImageText(imagePath)
    -- Get image content
    set theImage to current application's NSImage's alloc()'s initWithContentsOfFile:imagePath

     -- Set up request handler using image's raw data
    set requestHandler to current application's VNImageRequestHandler's alloc()'s initWithData:(theImage's TIFFRepresentation()) options:(current application's NSDictionary's alloc()'s init())
    
    -- Initialize text request
    set theRequest to current application's VNRecognizeTextRequest's alloc()'s init()
  
     -- Perform the request and get the results
    requestHandler's performRequests:(current application's NSArray's arrayWithObject:(theRequest)) |error|:(missing value)
    set theResults to theRequest's results()

    -- Obtain and return the string values of the results
    set theText to {}
    repeat with observation in theResults
        copy ((first item in (observation's topCandidates:1))'s |string|() as text) to end of theText
    end repeat
    return theText
end getImageText

on run (argv)
    if (count of argv) is 0 then error "Must provide an image path"
    getImageText(item 1 of argv)
end run

    You can run this in Script Editor -- just make sure to update the file path.