Search code examples
pysparksnowflake-cloud-data-platform

ValueError: substring not found when refactoring PySpark to work with snowpark


I am trying to build on the help I've received from SO members when refactoring PySpark code to snowflake.

Can someone take a look at the following snowpark python code and let me know why I'm getting invalid syntax error:

enter image description here

import snowflake.snowpark as snowpark
from snowflake.snowpark.functions import col

def registerDelta(session: snowpark.Session,regName, stage, filePath):
      
  pos = regName.index(".")
  if pos > 0:
    dbName = stage + regName[0:pos]
  else:
    dbName = stage
  
  try:
    crtdb = f"CREATE DATABASE IF NOT EXISTS {dbName}"
    session.sql(crtdb)
    print(f"{stage} - Database created")
  except:
    print("Create Database Failed")
  
  try:
    SQL = f"CREATE TABLE IF NOT EXISTS {stage}{regName} USING DELTA LOCATION '{filePath}'"
    session.sql(SQL)
    print(f"{stage}{regName} - Table created")
  except:
    print("Create Table Failed")

def main(session: snowpark.Session):
    return registerDelta(session, 'test1', 'test2', 'test3')

I've got feeling this will highlight my lack of Python coding knowledge.

Anyway, any thoughts?


Solution

  • This issue is purely based on code and it is not specific to Snowpark.

    pos = regName.index(".")  # when string is not found it raises an error
    if pos > 0:
      dbName = stage + regName[0:pos]
    else:
      dbName = stage
    

    I'm getting invalid syntax error:

    ValueError: substring not found

    This error is not a syntax error but a runtime error due to provided input and it is correct behavior.


    Python str.index()

    str.index(sub[, start[, end]])
    

    Like find(), but raise ValueError when the substring is not found.

    Python str.find()

    str.find(sub[, start[, end]])
    

    Return the lowest index in the string where substring sub is found within the slice s[start:end]. Optional arguments start and end are interpreted as in slice notation. Return -1 if sub is not found.