I wanted to know if the DCP based streaming is stable in Spark Couchbase Official Connector? The documents state that they aren't ready to be used in production as yet. Do we have any examples for Java?
Also, if someone could shed some light on the use case that I am trying to solve and whether a spark connector makes sense here:
We get daily updates about some badges from a couchbase lite db to sync gateway. Now, over a window of 10 days, if someone earns 5 or more badges, then we need to push a congratulatory badge. This needs to be done after every 10 days.
So, what I was planning was that using the DCP, I check for changes on a daily basis, run a job and then push it to sync gateway. If within the 10 day interval, 5 or more badges have been earned then I push it to the sync gateway..
Is this the right way to go about this or are there better ways to do this?
One more thing that could be used is:
So which approach would be better of here?
The DCP streaming from the Spark Connector is supported as that product is GA and that feature is not marked experimental. Couchbase does introduce features early and marks them experimental through annotations if the feature is not supported.
That said, I know there are some known issues with the JVM Core DCP implementation as of this writing which are expected to be fixed in the next couple weeks, if not sooner. The one I know of for sure is tracked here: https://issues.couchbase.com/browse/JVMCBC-312
New test scenarios have been added and Couchbase is working on making sure it passes these tests during different failure/rebalance modes before the next release.