Search code examples
azure-databricks

Reading Row values in DataBricks Notebook

I have a Databricks python notebook reading ADL configurations from a JSON file on DBFS. I am able to read the file fine and the DF generated shows the values with the below schema.

testJsonData:pyspark.sql.dataframe.DataFrame

clientId:string
oauth2Credential:string
oauth2RefreshUrl:string
providerType:string

I am having challenges in reading a specific row value. I am getting a row object but doesn't give me the value of the row. 

testJsonData.select('clientId').collect()
Out[65]: [Row(clientId='6xxxx08vvvvvvvvv7f')]

Appreciate any pointers helping with this.

Solution

  • Hope this helps someone having a similar issue, I read the config values like this

       configlist = testJsonData.select("*").collect()
   dataProviderType = configlist[0].providerType
   dataClientId = configlist[0].clientId
   dataOAuthCredential = configlist[0].oauth2Credential
   dataOAuthUrl = configlist[0].outh2RefreshUrl