I'm using HTTPoison to get the elixir guide website and then parsing it with Floki to build a HTML 2 Jupyter Notebook transformer (with Markdown for the description). I have to put in the ` backtick. \u0060 for code highlighting, which works so far. I have some places where I use string Interpolation "#{Floki.text(childs_nodes)}" and in other Places Enum.join "" to process and transform from HTML to Markdown.
The transformed result is stored in a map according to the jupyter notebookformat. When I call Poison.encode notebook I get an error, as the codepoints are gone. I tried different things, but don't know yet, where the issue is.
Any hints what I'm doing wrong when processing the text? This is the exception:
** (Poison.EncodeError) unable to encode value: {:source, ["Elixir also provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types."]}
lib/poison/encoder.ex:377: Poison.Encoder.Any.encode/2
lib/poison/encoder.ex:255: anonymous fn/3 in Poison.Encoder.List.encode/3
lib/poison/encoder.ex:256: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
lib/poison/encoder.ex:256: Poison.Encoder.List.encode/3
lib/poison.ex:41: Poison.encode!/2
(guide2nb) lib/cli.ex:27: CLI.process/1
(elixir) lib/kernel/cli.ex:76: anonymous fn/3 in Kernel.CLI.exec_fun/2
The problem here is that you are trying to encode a Tuple, while Poison only works with Maps and Lists. If the value you were trying to encode was a Map instead of a tuple, it would work perfectly. Unicode has nothing to do with this.
iex(1)> value = %{source: ["Elixir also provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types."]}
iex(2)> Poison.encode(value)
{:ok,
"{\"source\":[\"Elixir also provides `Port`, `Reference` and `PID` as data types (usually used in process communication), and we will take a quick look at them when talking about processes. For now, let’s take a look at some of the basic operators that go with our basic types.\"]}"}