Unable to select _ts from Cosmos db in databricks

Question

unable to select _ts from cosmos db from python script written in databricks.

read_config = {
    "spark.cosmos.accountEndpoint": url,
    "spark.cosmos.accountKey": key,
    "spark.cosmos.database": database,
    "spark.cosmos.container": container,
    "spark.cosmos.read.customQuery" : "select * from c "
}

rsltdf = spark.read.format("cosmos.oltp").options(**read_config).load()
display(rsltdf)

This select is selecting fields apart from auto generated fields like _ts, _etag. _rid..

Databricks runtime version - 7.3 LTS (includes Apache Spark 3.0.1, Scala 2.12)

libraries - com.azure.cosmos.spark:azure-cosmos-spark_3-3_2-12:4.18.1, azure-cosmos, pyDocumentDB

Answer 1

As @David mentioned you need to query _ts explicitly.

Make sure to give alias for it.

select c.name ,c._ts as ts, c._etag as etag from c

Output: