In Pydantic, how do I set env_ignore_empty to True

Question

I'm currently trying to use Pydantic to check if JSON received is in the right format, and it's one of those where we have 30, 40 fields coming in the JSON, and they have decided that if an amount (for example) is empty, they will provide a null string. Of course, when I check for a float, even if it is Optional, the system finds a string.

So if I have

from typing import Optional
from pydantic import BaseModel


class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None


data1 = Balance(**{"outstanding": 1000.00})
data2 = Balance(**{"outstanding": 1000.00, "paidtodate": 500.00})
data3 = Balance(**{"outstanding": 1000.00, "paidtodate": ""})

Problem is that data3 fails, because (rigthly so) paidtodate is a string, but I want to be able to say, if it's empty string, just accept it as none.

There seems to be an environment variable called env_ignore_empty, but I can't for the life of me figure out how to set it.

Of course, if there is some other way, would love that as well.

Using pydantic_core-2.16.3-cp39-cp39-macosx_11_0_arm64.whl

Answer 1

All you need is to convert empty strings to None and let the pydantic do the rest, i.e. validate it against Optional[float]:

from typing import Optional
from pydantic import BaseModel, field_validator

class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None

    # add more field names after 'paidtodate' or use '*' for all fields
    @field_validator('paidtodate',  mode='before')
    @classmethod
    def _emtpy_to_none(cls, field_value):
        if field_value == "":
            return None
        return field_value   # unchanged

Docs: https://docs.pydantic.dev/latest/concepts/validators/#field-validators

---

The updated version below should enable the preprocessing for all fields defined as Optional. Experimental code warning, I'm not a pydantic guru.

from typing import Optional, get_origin, get_args, Union
from pydantic import BaseModel, field_validator


class Balance(BaseModel):
    outstanding: float
    paidtodate: Optional[float] = None

    @field_validator('*', mode='before')
    @classmethod
    def _emtpy_to_none(cls, field_value, vinfo):
        if field_value == "":
            # check for "Optional"
            fa = cls.model_fields[vinfo.field_name].annotation
            # credit: https://stackoverflow.com/a/58841311/5378816
            if get_origin(fa) is Union and type(None) in get_args(fa)
                return None
        return field_value