I need to convert records in a Phoenix table into a JSON file using StreamSets. For inital POC purposes, I am trying to do a simple fetch from Phoenix into a file. The origin is a JDBC Query Consumer that point to the Phoenix and, for now, it is directly connected to Local FS destination. In actual implementation, of course there will be some more components in between like Jython Evaluator.
It validates just fine. But when I try to run it, I get the following error:
Pipeline Status: RUNNING_ERROR: com.streamsets.pipeline.api.StageException: JDBC_77 - ColumnNotFoundException attempting to execute query 'SELECT <columns I had selected> FROM <schema>.<table>'. Giving up after 1 errors as per stage configuration. First error: SQLState: 42703 Error Code: 504 Message: ERROR 504 (42703): Undefined column. columnName=0
I don't have basically any experience with StreamSets or Phoenix (Yea, I am trying to wing it here a bit). Is this error coming up because I am connecting the JDBC object to Local FS directly? In Informatica, we would have to create a template of columns for Source object. I don't see any option for that in the JDBC object. I had left almost everything at default except for below fields:
It's getting so frustrating. Please help!
I think the problem is that you have 0 as the Offset Column - this should be the name of the offset column. In fact, I don't think you need the offset stuff at all if you're not doing an incremental query, so you might want to just delete those settings.