Deserializing json to Pydantic model

Question

I have the following Pydantic model:

class OptimizationResponse(BaseModel):
     routes: List[Optional[Route]]
     skippedShipments: Optional[List[SkippedShipment]] = []
     metrics: AggregatedGlobalMetrics

With Route modeled as:

class Route(BaseModel):
    vehicleIndex: Optional[int] = 0
    vehicleStartTime: datetime
    vehicleEndTime: datetime
    visits: List[Visit]
    transitions: List[Transition]
    metrics: AggregatedRouteMetrics
    endLoads: List[EndLoad]
    travelSteps: List[TravelStep]
    vehicleDetour: str

The attribute routes can contain en empty Route, so the json may look like:

{
"routes": [
{},
{non empty route}]...rest of stuff}

And I'm trying to deserialize like:

# Convert the response to JSON format and write it to the file
    json_response = MessageToJson(fleet_routing_response._pb)
    async with aiofiles.open(self.config.get('api', 'response_file'), 'w') as response_file:
        await response_file.write(json_response)
# Deserializing the json
    optimization_response = parse_obj_as(OptimizationResponse, json.loads(json_response))
    return optimization_response

The issue arises when the program is dealing with an empty Route element since is trying to deserialize it even it's tagged as optional:

routes.0.vehicleStartTime Field required [type=missing, input_value={}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.5/v/missing

Otherwise, the deserialization happens without any issue.

And so for the rest of the attributes.

Answer 1

If you want to just ignore empty routes, use field validator (Pydantic v2) and remove empty dicts from the list of routes:

class OptimizationResponse(BaseModel):
    routes: List[Route]
    ...

    @field_validator("routes", mode="before")
    @classmethod
    def verify_routes(cls, v):
        v = [route for route in v if len(route.keys())]
        return v

In pydantic v1 use @validator("routes", pre=True) instead of @field_validator ...

Example:

from typing import Optional, List

from pydantic import BaseModel, field_validator, validator

class Route(BaseModel):
        vehicleStartTime: str

class OptimizationResponse(BaseModel):
    routes: List[Route]
    metrics: str

    @field_validator("routes", mode="before")
    # @validator("routes", pre=True)
    @classmethod
    def verify_routes(cls, v):
        v = [route for route in v if len(route.keys())]
        return v

r = OptimizationResponse.model_validate(
     {
          "metrics": "dfdfd",
          "routes": [
               {},
               {"vehicleStartTime": "123"}
          ]
     }
)

print(r.model_dump())

Output:

{'routes': [{'vehicleStartTime': '123'}], 'metrics': 'dfdfd'}