How do you validate application logic using Pyramid's Colander?

Question

So far I am using colander to validate the data in my aiohttp application.

The problem I face is that I don't know how to do "deep" validation.

Given the following schema:

import colander

class User(colander.MappingSchema):
    username = colander.SchemaNode(colander.String())
    password = colander.SchemaNode(colander.String())
    confirmation = colander.SchemaNode(colander.String())

I do both validate that the input datastructure has all the required fields are there (constraints are minimal for the sake of clarity) but I also need to check that:

• username is not already taken by another user
• password and confirmation are the same

So in my controllers, the code looks like the following pseudo code:

def create_user(request):
    user = await request.json()
    schema = User()
    # do schema validation
    try:
        user = schema.deserialize(user)
    except colander.Invalid, exc:
        response = dict(
            status='error',
            errors=errors.asdict()
        )
        return json_response(response)
    else:
        # check password and confirmation are the same
        if user['password'] != user['confirmation']:
            response = dict(
                status='error'
                errors=dict(confirmation="doesn't match password")
            )
            return json_response(response)
        # check the user is not already used by another user
        # we want usernames to be unique
        if user_exists(user['user']):
            response = dict(
                status='error',
                errors=dict(username='Choose another username')
            )
            return json_response(response)

        return json_response(dict(status='ok'))

Basically there is two kinds of validation. Is it possible to have both logic in single colander schema? Is it a good pattern?

Answer 1

Obviously it's a matter of taste but IMHO it's better to keep data validation separate from application logic.

You'll also run into a few problems trying to confirm that the username is unique:

1. Colander would need to have knowledge about your application eg. get access to the database connection to check with the database that that username doesn't exist.
2. Colander (AFAIK) isn't setup for asyncio programming so it'll have problems coping with an async method which checks the user exists.
3. You really want to user creation to be ACID, so simultaneous calls to create_user with the same username cannot possibly create two users with the same username.

Checking passwords match is another story, that doesn't require any knowledge about the rest of the world and should fairly trivial with colander. I'm not expert on colander, but it looks like you can use a deferred validator to check the two passwords match.

A few other notes on your code:

1. create_user should be an async method
2. I don't know anything about your db, but to get any advantage from async programming user_exists should be async too
3. The user existence check should be wrapped into the ACID user creation. Eg. you should use postgres's on conflict or equivalent to catch a duplicate user as you create them rather than checking they exist first
4. To be properly restful and make testing easier your view should return the correct http response code on an error (currently you return 200 for all states). You should use 201 for created, 400 for invalid date and 409 or a username conflict.