I'm trying to build a streamlit app. Below is the working example I'm trying
from pm4py.objects.conversion.log import converter as log_converter
import pandas as pd
import pm4py
df = pm4py.format_dataframe(pd.read_csv('https://raw.githubusercontent.com/pm4py/pm4py-core/release/notebooks/data/running_example.csv', sep=';'), case_id='case_id',activity_key='activity',
timestamp_key='timestamp')
log = log_converter.apply(df)
precedence_dict = pm4py.discover_dfg(log)[0]
precedence_dict is a dict of (antecedent,consequent) and count
precedence_dict = {('check ticket', 'decide'): 6,
('check ticket', 'examine casually'): 2,
('check ticket', 'examine thoroughly'): 1,
('decide', 'pay compensation'): 3,
('decide', 'reinitiate request'): 3,
('decide', 'reject request'): 3,
('examine casually', 'check ticket'): 4,
('examine casually', 'decide'): 2,
('examine thoroughly', 'check ticket'): 2,
('examine thoroughly', 'decide'): 1,
('register request', 'check ticket'): 2,
('register request', 'examine casually'): 3,
('register request', 'examine thoroughly'): 1,
('reinitiate request', 'check ticket'): 1,
('reinitiate request', 'examine casually'): 1,
('reinitiate request', 'examine thoroughly'): 1
}
Converting the above dict to pandas dataframe
precedence_df = pd.DataFrame.from_dict(precedence_dict, orient='index').reset_index()
rename_map = {"index" : "Antecedent,Consequent", 0 : "Count"}
precedence_df = precedence_df.rename(columns=rename_map)
precedence_df['Antecedent'], precedence_df['Consequent'] = zip(*precedence_df["Antecedent,Consequent"])
# precedence_df.assign(**dict(zip(['Antecedent', 'Consequent'], precedence_df["Antecedent,Consequent"].str)))
# precedence_df['Antecedent'], precedence_df['Consequent'] = precedence_df["Antecedent,Consequent"].str
precedence_mat = precedence_df[['Antecedent', 'Consequent', 'Count']]
st.dataframe(precedence_df)
I got the ArrowInvalid Error while running the app at this line
Complete error traceback
ArrowInvalid: ("Could not convert (x, y) with type tuple: did not recognize Python value type when inferring an Arrow data type", 'Conversion failed for column Antecedent, Consequent with type object')
Traceback:
File "C:\Users\zz\Documents\Streamlit\preced\app.py", line 1353, in <module>
st.dataframe(precedence_df)
File "c:\users\zz\anaconda3\lib\site-packages\streamlit\elements\dataframe_selector.py", line 85, in dataframe
return self.dg._arrow_dataframe(data, width, height)
File "c:\users\zz\anaconda3\lib\site-packages\streamlit\elements\arrow.py", line 82, in _arrow_dataframe
marshall(proto, data, default_uuid)
File "c:\users\zz\anaconda3\lib\site-packages\streamlit\elements\arrow.py", line 160, in marshall
proto.data = type_util.data_frame_to_bytes(df)
File "c:\users\zz\anaconda3\lib\site-packages\streamlit\type_util.py", line 371, in data_frame_to_bytes
table = pa.Table.from_pandas(df)
File "pyarrow\table.pxi", line 1561, in pyarrow.lib.Table.from_pandas
File "c:\users\zz\anaconda3\lib\site-packages\pyarrow\pandas_compat.py", line 595, in dataframe_to_arrays
for c, f in zip(columns_to_convert, convert_fields)]
File "c:\users\zz\anaconda3\lib\site-packages\pyarrow\pandas_compat.py", line 595, in <listcomp>
for c, f in zip(columns_to_convert, convert_fields)]
File "c:\users\zz\anaconda3\lib\site-packages\pyarrow\pandas_compat.py", line 581, in convert_column
raise e
File "c:\users\zz\anaconda3\lib\site-packages\pyarrow\pandas_compat.py", line 575, in convert_column
result = pa.array(col, type=type_, from_pandas=True, safe=safe)
File "pyarrow\array.pxi", line 302, in pyarrow.lib.array
File "pyarrow\array.pxi", line 83, in pyarrow.lib._ndarray_to_array
File "pyarrow\error.pxi", line 99, in pyarrow.lib.check_status
Current pyarrow version 5.0.0.
I DON'T HAVE ANY ISSUES/ERRORS when I tried to run the same code in colab(expect st.dataframe), I don't have any issues/errors. What does ArrowInavlid Error mean, how to fix this error?
I'm not too familiar with streamlit and st.dataframe but it looks like it's trying to convert precedence_df to a pyarrow.Table.
While in pandas you can have arbitrary objects as the data type of your column, in pyarrow it's not possible. So the column Antecedent,Consequent is causing issues because it's a tuple.
| | Antecedent,Consequent | Count |
|---:|:---------------------------------------------|--------:|
| 0 | ('check ticket', 'decide') | 6 |
| 1 | ('check ticket', 'examine casually') | 2 |
| 2 | ('check ticket', 'examine thoroughly') | 1 |
| 3 | ('decide', 'pay compensation') | 3 |
| 4 | ('decide', 'reinitiate request') | 3 |
| 5 | ('decide', 'reject request') | 3 |
It's easier and more idiomatic to work on a dataframe like precedence_mat, because it uses flat strings columns (instead of tuples).
| | Antecedent | Consequent | Count |
|---:|:-------------------|:-------------------|--------:|
| 0 | check ticket | decide | 6 |
| 1 | check ticket | examine casually | 2 |
| 2 | check ticket | examine thoroughly | 1 |
| 3 | decide | pay compensation | 3 |
| 4 | decide | reinitiate request | 3 |
Having that said you have two options if you really need to pass tuples to pyarrow/streamlit:
It's a bit tricky, you need to provide a schema for your tuples, explaining what they are:
import pyarrow as pa
schema = pa.schema(
[
pa.field(
"Antecedent,Consequent",
pa.struct(
[
pa.field("antecedent", pa.string()),
pa.field("consequent", pa.string()),
])
),
pa.field("Count", pa.int32())
]
)
table = pa.Table.from_pandas(precedence_df, schema=schema)
st.dataframe(table)
copy_df = precedence_df.copy()
copy_df["Antecedent,Consequent"] = precedence_df["Antecedent,Consequent"].apply(list)
table = pa.Table.from_pandas(copy_df).to_pandas()
st.dataframe(table)
Note that in this case the "Antecedent,Consequent" data is converted from tuple of strings to list of strings.