Search code examples
gramex

Is there a way to write single prepare function for all the KPIs in Gramex


I have a common function which needs to be evaluated before executing all the KPIs. So I wrote a prepare function but prepare function is calling for all the KPIs separately. Instead I want a prepare function to be executed once for the each endpoint. My endpoint is like this

project-test:
    pattern: /$YAMLURL/test
    handler: FormHandler
    kwargs:
      auth: &AUTH
        login_url: /$YAMLURL/login
      kpi1:
        prepare: validations.validate_request(args, handler)
        url: $CONNECTION_STRING
        queryfunction: queries.query1(args) 
      kpi2:
        prepare: validations.validate_request(args, handler)
        url: $CONNECTION_STRING
        queryfunction: queries.query2(args)
      modify: project.refactor(data, handler)

I tried to giving prepare function in the kwargs but getting

AttributeError: 'str' object has no attribute 'get'


Solution

  • The approach you shared is correct. The error may be in one of the custom functions, e.g. validations.validate_request, queries.query* or project.refactor

    To test this, I created the following data1.csv

    a,b
    1,2
    3,4
    

    ... and data2.csv

    a,b
    4,5
    6,7
    

    This is my gramex.yaml

    url:
      project-test:
          pattern: /$YAMLURL/test
          handler: FormHandler
          kwargs:
            kpi1:
              prepare: validations.validate_request(args, handler)
              url: data1.csv
            kpi2:
              prepare: validations.validate_request(args, handler)
              url: data2.csv
    

    ... and this is my validations.py:

    def validate_request(args, handler):
        print(args, handler)
        args['_c'] = ['a']    # Filter column 'a' alone
    

    When I visit /test, the output only shows column 'a':

    {"kpi1":[{"a":1},{"a":3}],"kpi2":[{"a":4},{"a":6}]}
    

    ... and the command prompt shows that the prepare function is called once for each dataset:

    {'_limit': [10000]} <gramex.services.FormHandler object at 0x000001B1C14A43C8>
    {'_limit': [10000]} <gramex.services.FormHandler object at 0x000001B1C14A43C8>
    

    Perhaps you could share the full error message about where the AttributeError appeared? That might help.