Search code examples
sqlapache-sparkapache-spark-sqldatabricksazure-databricks

commenting in spark sql

How can I comment multiple lines in spark-sql

%sql

/*
    select * from database.tablename 
    where condition
*/

gives me the following error

Error in SQL statement: 

    ParseException: 
    mismatched input '/' expecting {'(', 'CONVERT', 'COPY', 'OPTIMIZE', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)
Solution

  • For single line comment we should use -- and for multiline /* comments */.

    Actually comment is working in your case, problem is - spark ignores those comments after that it looks for sql commands but you didn't specify any.

    screenshot from databricks

    screenshot from databricks

    Below code will throw error.

      spark.sql(
    """
      | /* comment..1
      | comment..2 */
      |""")
    .show()

    but this works perfectly, since spark got the actual sql command to process.

      spark.sql(
    """
      | /* comment..1
      | comment..2 */
      | select current_date
      |""")
    .show()