Datamodel Inflator

This repository allows you to take a dictionary and construct an instance of a (predefined) dataclass. This works by inspecting the dataclasses' field types, and currently works for the following field types:

• Optional and List
• dict, str, int, float, and bool
• Enum
• Other dataclasses

Example

The code presented here is in the example directory. First, let's define a few simple dataclasses in model.py below. As you can see, there is a hierarchy: a Person has an Address, which in turn has a Street and a State. In practice, these class definitions would probably have been generated from an OpenAPI specification using datamodel-code-generator (see next section).

from dataclasses import dataclass
from enum import Enum
from typing import Optional


class AusState(Enum):
    WA = "WA"
    SA = "SA"
    NT = "NT"
    TAS = "TAS"
    QLD = "QLD"
    VIC = "VIC"
    NSW = "NSW"
    ACT = "ACT"


@dataclass
class Street:
    apartment_no: Optional[int]
    number: int
    name: str


@dataclass
class Address:
    street: Street
    state: AusState


@dataclass
class Person:
    name: str
    aliases: list[str]
    delivery_address: Address
    billing_address: Optional[Address]
    gender: Optional[str]

Now, suppose we have obtained an instance of these classes in dictionary form, as in person_dict below; in practice, you would have obtained this data from a web API call. This library allows you to convert the dictionary data into a bonafide Person object.

from model import Person
from parser_generator import make_parser

person_dict = {
    "name": "Tom",
    "aliases": ["T-bone", "t3h pwn3r3r"],
    "delivery_address": {
        "street": {"apartment_no": None, "number": 13, "name": "Fake Street"},
        "state": "WA",
    },
    "billing_address": None,
    "gender": None,
}
person_parser = make_parser(Person)
person = person_parser(person_dict)
print(person)  # Person(name='Tom', aliases=['T-bone', 't3h pwn3r3r'], delivery_address=Address(street=Street(apartment_no=None, number=13, name='Fake Street'), state=<AusState.WA: 'WA'>), billing_address=None, gender=None)

If dynamically generating a parser makes you uncomfortable, you can also generate the parser functions ahead of time. Running generate_parser_code([Person]) gives the code below. In the example above, you could have used person_parser(person_dict).

from parser_generator import *
from model import *

street_parser = obj_parser(
    Street,
    apartment_no=field(
        name="apartment_no", f=lambda x: None if x is None else (identity)(x)
    ),
    number=field(name="number", f=identity),
    name=field(name="name", f=identity),
)
address_parser = obj_parser(
    Address,
    street=field(name="street", f=street_parser),
    state=field(name="state", f=AusState),
)
person_parser = obj_parser(
    Person,
    name=field(name="name", f=identity),
    aliases=field(name="aliases", f=lambda xs: [identity(x) for x in xs]),  # type: ignore
    delivery_address=field(name="delivery_address", f=address_parser),
    billing_address=field(
        name="billing_address",
        f=lambda x: None if x is None else (address_parser)(x),  # type: ignore
    ),
    gender=field(name="gender", f=lambda x: None if x is None else (identity)(x)),
)

If you're generating a parser file, you probably want to generate parsers for all dataclasses in a given file (e.g. "model.py"). You can either enumerate these yourself, or run the following code:

import inspect
import importlib
import dataclasses
from parser_generator import generate_parser_code
all_dcs = [
    dc
    for _, dc in inspect.getmembers(
        importlib.import_module("model"), dataclasses.is_dataclass
    )
]
generate_parser_code(all_dcs)

Working with OpenAPI

1. Install datamodel-code-generator.
2. Download your OpenAPI specification of interest into the file "up.json".
3. Generate "model.py" containing dataclass definitions via the following command: datamodel-codegen --input "up.json" --input-file-type openapi --target-python-version 3.11 --output-model-type dataclasses.dataclass --use-standard-collections --reuse-model --use-schema-description --capitalise-enum-members --use-double-quotes --strict-nullable --output "model.py".
4. Call the web API and obtain the response.json() dictionary.
5. Use the parser for the expected dataclass as generated by this repository (see previous section) on the dictionary. The expected dataclass depends on your API call, but could have Response on the end (e.g. GetPersonResponse).
6. You have now data from a web API in a typed, easy-to-use format. Enjoy!

A general template for doing this:

import model  # Generated using datamodel-code-generator
import requests
from parser_generator import make_parser

# Set up session
token = "<enter your token here>"
session = requests.Session()
header = {"Authorization": f"Bearer {token}"}
session.headers.update(header)

# Make API call
response = session.get(
    "https://api.you.want/api/v1/a_function_which_returns_mydataclass",
)
result = response.json()

# Construct the parser and call it.
# Alternatively, you could have generated the parser code ahead of time using generate_parser_code.
parser = make_parser(model.MyDataClass)
my_obj = parser(result)  # Done!

Known issues

• Dataclasses with non-default arguments that inherit from dataclasses with default arguments will produce /"TypeError: non-default argument 'b' follows default argument"/. Currently the only fix is to change the dataclass decorator to @dataclass(kw_only=True); see koxudaxi/datamodel-code-generator#1559 for a discussion.