Search code examples
apache-sparkwso2apache-spark-sqlwso2-das

WSO2 DAS Spark script fails to execute


Following is the spark script I'm trying to execute. It's run succesefuly on DAS(3.0.1) Batch analytics console. But fails to execute when saved as a script in batch analytics.

insert overwrite table CLASS_COUNT select  ((timestamp / 120000) * 120000) as time , vin , username , classType,        
sum(acceleCount) as acceleCount , sum(decceleCount) as decceleCount
from ACCELE_COUNTS
group by ((timestamp / 120000) * 120000) ,classType, vin, username;

Error:

ERROR: [1.199] failure: ``limit'' expected but identifier ACCELE_COUNTSgroup found insert overwrite table X1234_CLASS_COUNT select ((timestamp / 120000) * 120000) as time , vin , username , classType, sum(acceleCount) as acceleCount , sum(decceleCount) as decceleCountfrom ACCELE_COUNTSgroup by ((timestamp / 120000) * 120000) ,classType, vin, username ^

Prior to this I'm executing following without any issue.

CREATE TEMPORARY TABLE ACCELE_COUNTS 
USING CarbonAnalytics 
OPTIONS (tableName "KAMPANA_RECKLESS_COUNT_STREAM", 
     schema "timestamp LONG , vin STRING, username STRING, classType STRING, acceleCount INT,decceleCount INT");

CREATE TEMPORARY TABLE CLASS_COUNT 
USING org.wso2.carbon.analytics.spark.event.EventStreamProvider 
OPTIONS (receiverURL "tcp://localhost:7611",
     username "admin",
     password "admin",
     streamName "DAS_RECKELSS_COUNT_STREAM", 
     version "1.0.0",
     description "Events are published  when product quantity goes beyond a certain level",
     nickName "product alerts",
     payload "time LONG,vin STRING,username STRING, classType STRING, acceleCount INT, decceleCount INT"
);

Solution

  • This happens as you don't have spaces between

    1) decceleCount and from

    2) ACCELE_COUNTS AND group by

    So, make sure to have spaces between words even when the 2nd word is in a new line.