Erlport/Python STDOUT capture to Elixir

Question

I'm trying to pipe STDOUT from Python/Erlport back to Elixir. I've got :python calls working fine, I just want to send the STDOUT stuff from Python back to Elixir for logging but I can't wrap my head around how to achieve that. I know it's possible even though I'm using Python 2.7.

I've got a Genserver wrapper around the :python module so that my call works like so:

pid = Python.start()
Python.call(pid, :bridge, :register_handler, [self()]) 

Python.call just looks like this:

def call(pid, module, function, args \\ []) do
  :python.call(pid, module, function, args)
end

Anything from :bridge (i.e., bridge.py) is lost to STDOUT unless I explicitly return something (obviously halting the function). What can I do to capture STDOUT?

My idea was to call something like Python.call(pid, :builtins, :print, [self()]) but that results in a bunch of errors and I really don't know if that's the right direction at all.

I actually want to pipe it into a Phoenix channel, but that's the easy part (I hope). Any advice? Thanks.

Answer 1

For anyone else getting stuck with this kinda thing: since I've got a GenServer around the :python instance, I've just leveraged handle_info:

def handle_info({:python, message}, session) do
  message |> String.split("\n", trim: true)
  SomeWeb.Endpoint.broadcast("log", "update", %{body: message})

  {:stop, :normal,  session}
end

Detail

To more fully outline my solution as @7stud advised, I'll include the wider approach based on erlport and this great post. Accordingly, I've got a Python module that looks like this:

defmodule App.Python do
   @doc """
      Python instance pointing to priv/python.
    """
   def start() do
      path = [
         :code.priv_dir(:prefect),
         "python"
      ]|> Path.join()

      {:ok, pid} = :python.start([
         {:python_path, to_charlist(path)}
      ])
      pid
   end

   def call(pid, module, function, args \\ []) do
      :python.call(pid, module, function, args)
   end

   def cast(pid, message) do
      :python.cast(pid, message)
   end

   def stop(pid) do
      :python.stop(pid)
   end
end

It's called from a GenServer that handles its spawning and termination:

defmodule App.PythonServer do
   @doc """
      Receives async. messages from Python instance.
    """
   use GenServer

   alias App.Python

   def start_link() do
      GenServer.start_link(__MODULE__, [])
   end

   def init(_args) do
      pid = Python.start()
      Python.call(pid, :bridge, :register_handler, [self()])
      App.Application.broadcast_change

      {:ok, pid}
   end

   def cast_draw(id) do
      {:ok, pid} = start_link()

      GenServer.cast(pid, {:id, id})
   end

   def call_draw(id) do
      {:ok, pid} = start_link()

      GenServer.call(pid, {:id, id}, 10_000)
   end

   def handle_call({:id, id}, _from, session) do
      result = Python.call(session, :bridge, :draw, [id])

      {:reply, result, session}
   end

   def handle_cast({:id, id}, session) do
      Python.cast(session, id)

      {:noreply, session}
   end

   def handle_info({:python, message}, session) do
      msg = message |> format_response
      {:ok, time} = Timex.now |> Timex.format("{h12}:{m}{am} {D}/{M}/{YYYY}")
      AppWeb.Endpoint.broadcast("log", "update", %{time: time, body: msg, process: message})

      {:stop, :normal,  session}
   end

   def terminate(_reason, session) do
      Python.stop(session)
      App.Application.broadcast_change

      :ok
   end

   defp format_response(message) do
      if String.contains? message, "[result] Sent" do
         message |> String.split("\n", trim: true) |> Enum.at(-2)
      else
         message |> String.split("\n", trim: true) |> Enum.take(-12) |> Enum.join("\n")
      end
   end
end

You can see at the end if STDOUT doesn't return a certain string from bridge.py (or any other Python module) it'll return a stacktrace. Speaking of, bridge.py looks like this:

import os
import sys
import subprocess

from erlport.erlang import set_message_handler, cast
from erlport.erlterms import Atom

message_handler = None # reference to the elixir process to send

cmd = "xvfb-run -a python"
py = os.path.join("/home/ubuntu/app/priv/python/export.py")

def cast_message(pid, message):
  cast(pid, message)

def register_handler(pid):
  global message_handler
  message_handler = pid

def handle_message(id):
    try:
      result = draw(id)
      print result
      if message_handler:
        cast_message(message_handler, (Atom('python'), result))
    except Exception, error:
      print error
      if message_handler:
        cast_message(message_handler, (Atom('python'), error))
      pass

def draw(id):
  proc = subprocess.check_output(
    "{0} {1} {2}".format(cmd, py, id), stderr = subprocess.STDOUT, shell = True
  )
  return proc

set_message_handler(handle_message)