Reading guarantees for full table scan while updating the table?

Question

Given schema:

CREATE TABLE keyspace.table (
    key text,
    ckey text,
    value text
    PRIMARY KEY (key, ckey)
) 

...and Spark pseudocode:

val sc: SparkContext = ...
val connector: CassandraConnector = ...
sc.cassandraTable("keyspace", "table")
  .mapPartitions { partition =>
    connector.withSessionDo { session =>
      partition.foreach { row =>
        val key = row.getString("key")
        val ckey = Random.nextString(42)
        val value = row.getString("value")
        session.execute(s"INSERT INTO keyspace.table (key, ckey, value)" + 
          " VALUES ($key, $ckey, $value)")
      }
    }
  }

Is it possible for a code like this to read an inserted value within a single application (Spark job) run? More generalized version of my question would be whether a token range scan CQL query can read newly inserted values while iterating over rows.

Answer 1

Yes, it is possible exactly as Alex wrote but I don't think it's possible with above code

So per data model the table is ordered by ckey in ascending order

The funny part however is the page size and how many pages are prefetched and since this is by default 1000 (spark.cassandra.input.fetch.sizeInRows), then the only problem could occur, if you wouldn't use 42, but something bigger and/or the executor didn't page yet

Also I think you use unnecessary nesting, so the code to achieve what you want might be simplified (after all cassandraTable will give you a data frame).

(I hope I understand that you want to read per partition (note a partition in your case is all rows under one primary key - "key") and for every row (distinguished by ckey) in this partition generate new one (with new ckey that will just duplicate value with new ckey) - use case for such code is a mystery for me, but I hope it has some sense:-))