The following program is suppose to calculate an expression with the possibility of having two different data types , Float and RDD. I already created an RPN from the infix expression and now I am trying to perform calculations on them.
Note: I have also overloaded :+,-,/,* for doing calculations on RDD and float.
def calcRPN(s: String): RDD[(Int,Array[Float])] =
(s.split(' ').toList.foldLeft(Nil: List[Either[Float, RDD[(Int,Array[Float])]]) {foldingFunction}).head
def foldingFunction(list: List[Either[Float, RDD[(Int,Array[Float])]]], next: String): List[Either[Float,RDD[(Int,Array[Float])]]] = (list, next) match {
//apply * on inputs
case (Right(x) :: Right(y) :: ys, "*") =>{(sv.*(x,y)) :: ys} //both RDD sv is another class containing overloads
case (Left(x) :: Right(y) :: ys, "*") =>{sv.*(x,y) :: ys} //x being float
case (Right(x) :: Left(y) :: ys, "*") =>{sv.*(x,y) :: ys} //y being float}
case (Left(x) :: Left(y) :: ys, "*") => (x * y) :: ys //both float
//apply + on inputs
case (Right(x) :: Right(y) :: ys, "+") => {(sv.+(x,y)) :: ys} //both RDD
case (Left(x) :: Right(y) :: ys, "+") =>{(sv.+(x,y)):: ys} //x being float
case (Right(x) :: Left(y) :: ys, "+") =>{(sv.+(x,y)):: ys} //y being float
case (Left(x) :: Left(y) :: ys, "+") => (y + x) :: ys //both float
//apply - on the inputs
case (Right(x) :: Right(y) :: ys, "-") => {(sv.-(x,y)):: ys} //both RDD
case (Left(x) :: Right(y) :: ys, "-") =>{(sv.-(x,y)) :: ys} //x being float
case (Right(x) :: Left(y) :: ys, "-") =>{(sv.-(x,y)):: ys} //y being float
case (Left(x) :: Left(y) :: ys, "-") => (y - x) :: ys //both float
//apply / on the inputs
case (Right(x) :: Right(y) :: ys, "/") => {(sv./(x,y)) :: ys} //both RDD
case (Left(x) :: Right(y) :: ys, "/") =>{(sv./(x,y)) :: ys} //x being float
case (Right(x) :: Left(y) :: ys, "/") =>{(sv./(x,y)):: ys} //y being float
case (Left(x) :: Left(y) :: ys, "/") => {(y / x) :: ys} //both float
case (xs, numString) => numString.toInt :: xs //**
case (xs, pathxml) => sv.getArrayRDD() :: xs //***
}
I know this code is ugly sorry about that. I can make it shorter but right now I need to make it work then brush it up!
So in the ** part it is working for two numbers but I added *** to make it accept RDD as well. Don't know if it works for both Float and RDD! plus I have faced the following error because of using Either and apparently Left and Right are not helping me here!
[error] type mismatch;
[error] found : Either[Float,org.apache.spark.rdd.RDD[(Int, Array[Float])]]
[error] required: org.apache.spark.rdd.RDD[(Int, Array[Float])]
[error] (s.split(' ').toList.foldLeft(Nil: List[Either[Float, RDD[(Int,Array[Float])]]]) {foldingFunction}).head
[error] ^
I also tried Scalaz but it made it more complex.
Ok first things first, lets split up things for a better understanding:
val creepyListOfoperatorsAndStuff: List[String] = s.split(' ').toList
val eitherList: List[Either[Float, RDD[(Int,Array[Float])]]] =
creepyListOfoperatorsAndStuff.foldLeft(
List.empty[Either[Float, RDD[(Int,Array[Float])]]
) (foldingFunction)
val headEither:Either[Float, RDD[(Int,Array[Float])]] = eitherList.head
The head of that List is an Either. Thus neither a Float nor a RDD. That means we have to decide whether it is a Float or a RDD[(Int,Array[Float])]. If you are REALLY sure head contains an RDD, you can just do:
headEither.right.get
A better way to do this might be to deal with both cases:
headEither.fold[RDD[(Int,Array[Float])]](
// function to convert a Left() result to the RDD you want
fa = someFloat => <code to get to the RDD you want>,
// this is a function to transform the Right() result to what is desired
// as RDD is what you want you can just return the input
fb = anRDD => anRDD
)
Now onwards to the cases ** and ***:
in
case (xs, numString) => numString.toInt :: xs //**
case (xs, pathxml) => sv.getArrayRDD() :: xs //***
The second case seems unreachable, because both cases match the same input. You would probably be better off using regex to match the strings you expect there. I am not exactly an expert on regular expression matching but something like the following might point in the right direction.
val Numeric = """(\d+)""".r
// don't forget to put the matched string into a group
val XmlPath = """<some regular expression that matches your expected xml path""".r
...
case (xs, NumericString(numString)) => numString.toInt :: xs //**
case (xs, XmlPath(pathxml)) => sv.getArrayRDD() :: xs //***
However, there are more essential problems in theses two cases:
case (xs, numString) => numString.toInt :: xs //**
xs would be a List[Either[Float, RDD[(Int,Array[Float])]]. Thus I have to wonder, does this compile?
numString.toInt :: xs
If so then numString.toInt is probably converted to Float and then to Left[Float]. But I'm just guessing.
case (xs, pathxml) => sv.getArrayRDD() :: xs //***
While I donn't see what sv might possibley be and where it comes form, it might be ok, with the regex matcher.
I would only be able to help with that with more information form you.