Snowflake’s modin.pandas.dataframe df has 7 columns Snowflake Table “Db.S.table” has 10 columns - 1 auto increment and most that are nullable
I’m not able to do session.write_pandas(df,database=“Db”,schema=“schema”,table_name=“table”,overwrite=False)
Since it’s giving a column mismatch error. Why is this not an issue with native pandas
Interesting issue. I am able to reproduce it, and it seems it's related how data is ingested by modin-pandas and (regular) pandas.
According to my tests, write_pandas uses COPY INTO command, and specify the column names. Therefore you don't get any error even if your dataframe doesn't have all columns in the table.
The modin-pandas uses INSERT INTO command, and it doesn't specify the column names, therefore your dataframe should have all the columns in the table.
Please submit a Support case, explain the business impact and the error you get. So Support can engage Engineering to fix this issue.