If one only wants to insert new rows, not update existing ones: Is Apache Spark SQL generally faster with DROP TABLE and INSERT, or a MERGE & INSERT? Or some third option.
There's quite a bit written on traditional SQL, but I don't see much for Spark SQL.
If it depends...depends on what?
(I am working within Azure Synapse, writing to a Delta table, if that makes any difference0.
DROP TABLE and INSERT would be relatively faster as MERGE and INSERT of Delta Lake involves JOIN operations (Inner Join and outer join), which makes the MERGE slower and costly