Search code examples
javascriptsqldatabasestored-proceduressnowflake-cloud-data-platform

UDF JavaScript implementation into Snowflake


I found usefull JS function that I want to implement into snowflake's UDF in order to move complex computation to the dwh.

http://trentrichardson.com/2010/04/06/compute-linear-regressions-in-javascript/

Attempt to rework above as SQL procedure:

CREATE OR REPLACE PROCEDURE JS_LNR_REG(a array, b array)
RETURNS ARRAY
LANGUAGE javascript
AS
$$
function linearRegression(y,x){
var lr = {};
var n = y.length;
var sum_x = 0;
var sum_y = 0;
var sum_xy = 0;
var sum_xx = 0;
var sum_yy = 0;

for (var i = 0; i < y.length; i++) {

    sum_x += x[i];
    sum_y += y[i];
    sum_xy += (x[i]*y[i]);
    sum_xx += (x[i]*x[i]);
    sum_yy += (y[i]*y[i]);
} 

lr['slope'] = (n * sum_xy - sum_x * sum_y) / (n*sum_xx - sum_x * sum_x);
lr['intercept'] = (sum_y - lr.slope * sum_x)/n;

return lr;}      ; $$;

Execution fails with error: SQL compilation error: Invalid identifier which is odd since procedure has been created.

SELECT JS_LNR_REG('[2,4,5,3,1]', '[5.2, 5.7, 5.0, 4.2]');

Solution

  • You're almost there. To pass an array, try using ARRAY_CONSTRUCT or PARSE_JSON, like so:

    SELECT JS_LNR_REG(array_construct(1,2,3,4), array_construct(5,4,3,2));
    SELECT JS_LNR_REG(parse_json('[1,2,3,4]'), parse_json('[5,4,3,2]'));
    

    To get the code working, make it a FUNCTION that returns OBJECT, and be sure to actually invoke linearRegression() from within your UDF:

    CREATE OR REPLACE FUNCTION JS_LNR_REG(A array, B array)
      RETURNS OBJECT
      LANGUAGE JAVASCRIPT
      STRICT
    AS
    $$
    function linearRegression(y,x){
      var lr = {};
      var n = y.length;
      var sum_x = 0;
      var sum_y = 0;
      var sum_xy = 0;
      var sum_xx = 0;
      var sum_yy = 0;
    
      for (var i = 0; i < y.length; i++) {
    
          sum_x += x[i];
          sum_y += y[i];
          sum_xy += (x[i]*y[i]);
          sum_xx += (x[i]*x[i]);
          sum_yy += (y[i]*y[i]);
      } 
    
      lr['slope'] = (n * sum_xy - sum_x * sum_y) / (n*sum_xx - sum_x * sum_x);
      lr['intercept'] = (sum_y - lr.slope * sum_x)/n;
    
      return lr;
      }      
    
      return linearRegression(A,B)
    ; $$;
    

    Hope that's helpful.