load_all()/select_related() pydantic validation errors for ormar.JSON() field type

[dingxuanyao]

Describe the bug
When running load_all() or select_related() on model with foreign model with ormar.JSON() field, we see a pydantic validation error.

To Reproduce

import asyncio
import databases

import ormar
import sqlalchemy

DATABASE_URL = "sqlite:///db.sqlite"
base_ormar_config = ormar.OrmarConfig(
    database=databases.Database(DATABASE_URL),
    metadata=sqlalchemy.MetaData(),
    engine=sqlalchemy.create_engine(DATABASE_URL),
)


class Author(ormar.Model):
    ormar_config = base_ormar_config.copy(tablename="authors")
    id: int = ormar.Integer(primary_key=True)


class Book(ormar.Model):
    ormar_config = base_ormar_config.copy(tablename="books")
    id: int = ormar.Integer(primary_key=True)
    author: Author = ormar.ForeignKey(Author, name="author_id")
    my_data: dict = ormar.JSON(nullable=True)


async def main():
    base_ormar_config.metadata.drop_all(base_ormar_config.engine)
    base_ormar_config.metadata.create_all(base_ormar_config.engine)

    author = await Author.objects.create()
    book = await Book.objects.create(
        author=author,
        my_data={}
    )
    # this errors out
    authors =  await Author.objects.select_related(Author.books).all()
    # this also errors out
    authors = await Author.objects.select_all().all()

asyncio.run(main())

Error traceback

Traceback (most recent call last):
  File "/Users/tonyyao/workspace/ormar-bug/main.py", line 50, in <module>
    asyncio.run(main())
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/tonyyao/workspace/ormar-bug/main.py", line 47, in main
    authors = await Author.objects.select_all().all()
  File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/queryset/queryset.py", line 1084, in all
    result_rows = await self._process_query_result_rows(rows)
  File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/queryset/queryset.py", line 189, in _process_query_result_rows
    self.model.from_row(
  File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/models/model_row.py", line 104, in from_row
    instance = cast("Model", cls(**item))
  File "/Users/tonyyao/.local/pipx/.cache/f0aff6fe1a77366/lib/python3.9/site-packages/ormar/models/newbasemodel.py", line 138, in __init__
    self.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 12 validation errors for Author
books.int
  Input should be a valid integer [type=int_type, input_value=Book({'id': 1, 'author': ...040>]}), 'my_data': {}}), input_type=Book]
    For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.Book.my_data
  JSON input should be string, bytes or bytearray [type=json_type, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/json_type
books.PkOnlyBookrybjdp
  Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=Book({'id': 1, 'author': ...040>]}), 'my_data': {}}), input_type=Book]
    For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.0.int
  Input should be a valid integer [type=int_type, input_value=('id', 1), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.0.Book
  Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('id', 1), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.0.PkOnlyBookrybjdp
  Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('id', 1), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.1.int
  Input should be a valid integer [type=int_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.1.Book
  Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.1.PkOnlyBookrybjdp
  Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('author', Author({'id': ...Book at 0x102391040>]})), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_type
books.`list[nullable[union[int,Book,...]]]`.2.int
  Input should be a valid integer [type=int_type, input_value=('my_data', {}), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/int_type
books.`list[nullable[union[int,Book,...]]]`.2.Book
  Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value=('my_data', {}), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_attributes_type
books.`list[nullable[union[int,Book,...]]]`.2.PkOnlyBookrybjdp
  Input should be a valid dictionary or instance of PkOnlyBookrybjdp [type=model_type, input_value=('my_data', {}), input_type=tuple]
    For further information visit https://errors.pydantic.dev/2.5/v/model_type

Expected behavior
Unless I'm using these functions the wrong way, I expect either of these functions to not hit any pydantic errors.

Versions (please complete the following information):

# /// script
# dependencies = [
#   "databases[aiosqlite]==0.7.0",
#   "pydantic==2.5.3",
#   "ormar==0.20.1",
#   "sqlalchemy==1.4.52"
# ]
# ///

Additional context

• does NOT hit validation errors if json field is optional and not populated
• does NOT hit validation errors for other field types
• does NOT hit validation errors if using prefetch_related instead of selected_related
• one mitigation to not hit validation errors is the following author = await author.load(); books = await author.books.all()

Thanks in advance!

[tsalpekar21]

Having this same issue. Glad you were able to figure out it was related to a JSON field!