I am trying to implement a Spark program in a Databricks Cluster and I am following the documentation whose link is as follows:
Now, after this line of code:
def mapKeyToVal(mapping):
def mapKeyToVal_(col):
return mapping.get(col)
return udf(mapKeyToVal_, StringType())
I am using this:
gameInfDf = gameInfDf.withColumn("country_code", mapKeyToVal(countryCodeMap)("country"))
And I am getting the Error: name 'countryCodeMap' is not defined
It will be great if anyone can help me with the same.
https://databricks.com/blog/2018/07/09/analyze-games-from-european-soccer-leagues-with-apache-spark-and-databricks.html is the formal guide for databricks.
See picture below. You need to click on the link and IMPORT the .dbc
You will then see various setupo things. E.g. the Maps needed. Good stuff.
You can see the maps, some of them:
situationMap = {1:'Open play', 2:'Set piece', 3:'Corner', 4:'Free kick', 99:'NA'}
countryCodeMap = {'germany':'DEU', 'france':'FRA', 'england':'GBR', 'spain':'ESP', 'italy':'ITA'}