Search code examples
scalaapache-sparkspark-graphx

How to check if an edge exist in a Spark Graphx graph

I have a Spark Graphx graph, and I want to check wether an edge exists between two vertices or not. What is the preferred method for doing this in Spark Graphx?

More specifically I would like to count all the edges between all vertices in one list to all vertices in another list.

I tried this:

graph.edges.filter { case Edge(src, dst, prop) => ids1.contains(src)&&ids2.contains(dst)}.count

where ids1 and ids2 are two arrays containing vertex ids. But this doesn't work and I get the error:

org.apache.spark.SparkException: Task not serializable

I'm not very familiar with Graphx so any help is highly appreciated.

Solution

  • This works for me :

    graph.edges.filter(edge=>( ids.contains(edge.srcId) && ids.contains(edge.dstId))