how to read hdfs file with wildcard character used by pyspark

Question

there are some parquet file paths are:

/a/b/c='str1'/d='str'

/a/b/c='str2'/d='str'

/a/b/c='str3'/d='str'

I want to read the parquet files like this:

df = spark.read.parquet('/a/b/c='*'/d='str')

but it doesn't work by using "*" wildcard character.How can I do that? thank you for helping

Answer 1

You need to escape single quotes:

df = spark.read.parquet('/a/b/c=\'*\'/d=\'str\'')

... or just use double quotes:

df = spark.read.parquet("/a/b/c='*'/d='str'")