No ClassTag available for Vec[Double]

Question

I am kind of new to scala and spark. Now I would like to generate a vector on each of the worker. When I use this line, I got two errors:

val b = sc.parallelize(1 to n, n).map( i => DenseVector[Double]](10.0,20.0,30.0,40.0))

1. No ClassTag available for Vec[Double]
2. not enough arguments for method map: (implicit evidence$3: scala.reflect.ClassTag[Vec[Double]])org.apache.spark.rdd.RDD[Vec[Double]]. Unspecified value parameter evidence$3.

Could anybody help me on this?

Answer 1

Following works for me

scala> import breeze.linalg._
scala> val n = 2
n: Int = 2
scala> val b = sc.parallelize(1 to n, n).map( i => DenseVector[Double](10.0,20.0,30.0,40.0))
b: org.apache.spark.rdd.RDD[breeze.linalg.DenseVector[Double]] = MapPartitionsRDD[1] at map at <console>:24

Here is some data in the rdd:

   scala> b.take(1)
res1: Array[breeze.linalg.DenseVector[Double]] = Array(DenseVector(10.0, 20.0, 30.0, 40.0))