Search code examples
pythonflask-restfulpyparsingmarshmallowwebargs

Parse delimited and nested field names from URL parameter for partial response

In a Flask-RESTful based API, I want to allow clients to retrieve a JSON response partially, via the ?fields=... parameter. It lists field names (keys of the JSON object) that will be used to construct a partial representation of the larger original.

This may be, in its simplest form, a comma-separated list:

GET /v1/foobar?fields=name,id,date

That can be done with webargs' DelimitedList schema field easily, and is no trouble for me.

But, to allow nested objects' keys to be represented, the delimited field list may include arbitrarily nested keys enclosed in matching parentheses:

GET /v1/foobar?fields=name,id,another(name,id),date
{
  "name": "",
  "id": "",
  "another": {
    "name": "",
    "id": ""
  },
  "date": ""
}

GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
{
  "id": "",
  "one": {
    "id: "",
    "two": {
      "id": "",
      "three": {
        "id": ""
      },
      "date": ""
    }
  },
  "date": ""
}

GET /v1/foobar?fields=just(me)
{
  "just": {
    "me: ""
  }
}

My question is two-fold:

  1. Is there a way to do this (validate & deserialize) with webargs and marshmallow natively?

  2. If not, how would I do this with a parsing framework like pyparsing? Any hint on what the BNF grammar is supposed to look like is highly appreciated.

Solution

  • Pyparsing has a couple of helpful built-ins, delimitedList and nestedExpr. Here is an annotated snippet that builds up a parser for your values. (I also included an example where your list elements might be more than just simple alphabetic words):

    import pyparsing as pp

# placeholder element that will be used recursively
item = pp.Forward()

# your basic item type - expand as needed to include other characters or types
word = pp.Word(pp.alphas + '_')
list_element = word

# for instance, add support for numeric values
list_element = word | pp.pyparsing_common.number

# retain x(y, z, etc.) groupings using Group
grouped_item = pp.Group(word + pp.nestedExpr(content=pp.delimitedList(item)))

# define content for placeholder; must use '<<=' operator here, not '='
item <<= grouped_item | list_element

# create parser
parser = pp.Suppress("GET /v1/foobar?fields=") + pp.delimitedList(item)

    You can test any pyparsing expression using runTests:

    parser.runTests("""
    GET /v1/foobar?fields=name,id,date
    GET /v1/foobar?fields=name,id,another(name,id),date
    GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
    GET /v1/foobar?fields=just(me)
    GET /v1/foobar?fields=numbers(1,2,3.7,-26e10)    
    """,  fullDump=False)

    Gives:

    GET /v1/foobar?fields=name,id,date
['name', 'id', 'date']
GET /v1/foobar?fields=name,id,another(name,id),date
['name', 'id', ['another', ['name', 'id']], 'date']
GET /v1/foobar?fields=id,one(id,two(id,three(id),date)),date
['id', ['one', ['id', ['two', ['id', ['three', ['id']], 'date']]]], 'date']
GET /v1/foobar?fields=just(me)
[['just', ['me']]]
GET /v1/foobar?fields=numbers(1,2,3.7,-26e10)
[['numbers', [1, 2, 3.7, -260000000000.0]]]