Search code examples
scalafunctionselectoverloading

error: overloaded method value select with alternatives:

I am reading a CSV file in dataframe1 and then filter some columns in dataframe2, during selecting columns for dataframe2 from dataframe1 I want to apply my function on the column value. Like

import utilities._
val Logs = sqlContext.read
  .format("csv")
  .option("header", "true")
  .load("dbfs:/mnt/records/Logs/2016.07.17/2016.07.17.{*}.csv")
val Log = Logs.select(
  "key1",
  utility.stringToGuid("username"),
  "key2",
  "key3",
  "startdatetime",
  "enddatetime")
display(Log)

so here I am calling utility.stringToGuid("username"). And it is giving me error:

notebook:5: error: overloaded method value select with alternatives:
  (col: String,cols: String*)org.apache.spark.sql.DataFrame <and>
  (cols: org.apache.spark.sql.Column*)org.apache.spark.sql.DataFrame
Solution

  • So actually I found the answer to my question. Actually I was passing the string "username" to the utility function instead of passing the column value of "username". So in argument it should be like utility.stringToGuid($"username"). In scala $"" is used to send the column value and in python col() is used.