Output Models to Different Files

[d-w-d]

Feature Request: would be nice to have the option to output all of the model classes to separate files; saves having to do a lot of cut/pasting when the single output file is huge.

[agronholm]

How often do you need to do this, really?

[canine-afficionado]

Actually this would be really handy for large projects with lots of tables. Thanks

[agronholm]

Would you like to elaborate? One file per class / table? And where would association tables go?

It just seemed like a lot of effort for a tiny bit of convenience. This tool is meant to be run just once when you start writing your models against an existing database.

[canine-afficionado]

Thanks for asking. If the tool is meant to be run just once then it's probably not worth the effort but that's not how I use this sort of tool. I like to do my design with database modelling tools and then regenerate the models from the databases when they change (aka database-first).

The tools I've used in the past for code generation (in other object-oriented languages) generate a seperate file for each class probably because 1) They prefer one file per class and 2) It's more manageable when you have a lot of tables. Normally an association table would also have it's own file as it's a separate class.

I realise that in Python it's pretty common to have more than one class in a file and modern programming tools make it easy to move between those classes so it's not essential IMHO.

[agronholm]

I realise that in Python it's pretty common to have more than one class in a file and modern programming tools make it easy to move between those classes so it's not essential IMHO.

Exactly. I find that the people who want to separate models to their own modules usually come from other languages like Java (like I did) where you literally have no choice about it.

[nixjdm]

I am a Python developer, but I'd still like this option. Over several years I've a few times been given the task taking ownership of a large, existing database, and to begin managing it with sqla and alembic. Sometimes there's dozens, or hundreds of tables. I can work with that all in one file, but tbh I'd rather have a hundred files in models/ than one models.py that's possibly thousands of lines long. After the fact, I might choose to reorganize them a bit further, but having the option to start with things broken out would be nice.

[ab5y]

I used ChatGPT to create a script that does this quite easily.

[devspacenine]

I used ChatGPT to create a script that does this quite easily.

Doing flying spaghetti monster's work. Thank you sir/ma'am

[GuilhermeLimaSP]

It would be useful to have this natively

I made a homemade solution to deal with imports too, maybe it will help someone in the future: sqlacodegen-model-split

[Genaker]

I made file it separates models:

import os
import re

# Path to your single large models file:
input_file_path = "Models.py"
# Where you want to output the separate model files:
output_dir = "models"

# Make sure the output directory exists
os.makedirs(output_dir, exist_ok=True)

# Common imports that will appear at the top of each generated file.
# This includes the Base = declarative_base() if you want each file
# to define it. If you prefer a single shared Base, remove it here and
# define it only once in some other file.
common_imports = [
    "from sqlalchemy import BigInteger, Boolean, CHAR, Column, Date, Float, ForeignKey, Index, Integer, JSON, LargeBinary, Numeric, SmallInteger, String, Table, Text, UniqueConstraint, text\n",
    "from sqlalchemy.ext.declarative import declarative_base\n",
    "from sqlalchemy.orm import relationship, configure_mappers\n",
    "from sqlalchemy.dialects.postgresql import JSONB, TIMESTAMP, TIME, UUID\n",
    "from . import Base, metadata\n",
    "\n"
]

def comment_out_relationships(lines):
    """
    Given the lines of a single class, comment out any multi-line relationship(...) block.
    """
    updated = []
    inside_relationship = False
    paren_count = 0

    for line in lines:
        if not inside_relationship:
            # Check if this line starts a relationship(...) definition
            if "relationship(" in line:
                inside_relationship = True

                # Count parentheses in this line
                open_parens = line.count("(")
                close_parens = line.count(")")
                paren_count = open_parens - close_parens

                # Comment out this line
                #updated.append("# " + line)
                updated.append("" + line)

                # If paren_count returns to 0, the relationship ended on the same line
                if paren_count <= 0:
                    inside_relationship = False
                    paren_count = 0
            else:
                updated.append(line)
        else:
            # Already inside a relationship(...) block
            #updated.append("# " + line)
            updated.append("" + line)

            # Count parentheses on this line
            open_parens = line.count("(")
            close_parens = line.count(")")
            paren_count += (open_parens - close_parens)

            # If we've balanced out parentheses, exit relationship mode
            if paren_count <= 0:
                inside_relationship = False
                paren_count = 0

    return updated


# 1) Read Models.py into memory
with open(input_file_path, "r") as file:
    lines = file.readlines()

class_definitions = {}
current_class = None
current_lines = []

# 2) Split out each class definition
for line in lines:
    if line.startswith("class "):
        if current_class:
            class_definitions[current_class] = current_lines
        current_class = line.split()[1].split("(")[0]
        current_lines = [line]
    else:
        current_lines.append(line)

# Save the last one
if current_class:
    class_definitions[current_class] = current_lines

# 3) Generate one .py file per class, commenting out multi-line relationships
for cls, cls_lines in class_definitions.items():
    # Start with your shared imports
    file_contents = common_imports[:]
    file_contents.append("\n")

    # Comment out entire multi-line relationship(...) blocks
    updated_lines = comment_out_relationships(cls_lines)

    file_contents += updated_lines

    # Write them to models/<ClassName>.py
    out_path = os.path.join(output_dir, f"{cls}.py")
    with open(out_path, "w") as f:
        f.writelines(file_contents)

# 4) Generate __init__.py without referencing Base
init_lines = [
    "# models/__init__.py\n\n",
    "# We are NOT importing Base here.\n\n",
    "\n",
    "from sqlalchemy.ext.declarative import declarative_base\n",
    "Base = declarative_base()\n",
    "metadata = Base.metadata\n",
    "\n"
]

for cls in class_definitions.keys():
    init_lines.append(f"from .{cls} import {cls}\n")

init_lines.append("\n__all__ = [\n")
for cls in class_definitions.keys():
    init_lines.append(f'    "{cls}",\n')
init_lines.append("]\n")

with open(os.path.join(output_dir, "__init__.py"), "w") as init_file:
    init_file.writelines(init_lines)

print(f"Split {len(class_definitions)} classes into separate model files in '{output_dir}'.")
print("Commented out entire multi-line 'relationship(...)' blocks.")
print("Generated __init__.py ")

But here is an issue. You must import all models in init.py; the first run will be slow. The main problem is that you will have circular dependencies in your relationship.