Weird Error depending on import order

[SWeav02]

I've been working on updating some of my code to being more class oriented. I recently committed some changes and noticed that something I had done was causing an error with running the simmate command. After quite a bit of trial and error, I've determined that the error started after I ran isort . and black .. After even more investigation I discovered that changing the order of imports in two files resulted in the error. If I have this:

# from simmate.apps.warrenapp.workflows.badelf.prebadelf_dft import (
#     StaticEnergy__Warren__PrebadelfHse,
# )
from simmate.apps.warrenapp.workflows.nested_dft.relaxation_static_base import (
    RelaxationRelaxationStaticBase,
)

from simmate.apps.warrenapp.workflows.relaxation.hse import (
    Relaxation__Warren__Hse
)
from simmate.apps.warrenapp.workflows.nested_dft.prepop_relaxation_static_pbesol_hse import (
    Relaxation__Warren__PbeWithWavecar,
)
from simmate.apps.warrenapp.workflows.badelf.prebadelf_dft import (
    StaticEnergy__Warren__PrebadelfHse,
)

everything runs fine. But if I have this:

from simmate.apps.warrenapp.workflows.badelf.prebadelf_dft import (
    StaticEnergy__Warren__PrebadelfHse,
)
from simmate.apps.warrenapp.workflows.nested_dft.relaxation_static_base import (
    RelaxationRelaxationStaticBase,
)

from simmate.apps.warrenapp.workflows.relaxation.hse import (
    Relaxation__Warren__Hse
)
from simmate.apps.warrenapp.workflows.nested_dft.prepop_relaxation_static_pbesol_hse import (
    Relaxation__Warren__PbeWithWavecar,
)
# from simmate.apps.warrenapp.workflows.badelf.prebadelf_dft import (
#     StaticEnergy__Warren__PrebadelfHse,
# )

I get this error when I run simmate:

(simmate_dev) sweav@digital-storm:~/Documents/github/simmate$ simmate
Traceback (most recent call last):
  File "/home/sweav/anaconda3/envs/simmate_dev/bin/simmate", line 5, in <module>
    from simmate.command_line.base_command import simmate_app
  File "/home/sweav/Documents/github/simmate/src/simmate/command_line/base_command.py", line 12, in <module>
    from simmate.apps.warrenapp.command_line.badelf import badelf_app
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/command_line/badelf.py", line 17, in <module>
    from simmate.apps.warrenapp.workflows.badelf.badelf import (
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/workflows/__init__.py", line 3, in <module>
    from .badelf import (
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/workflows/badelf/__init__.py", line 3, in <module>
    from .badelf import BadElfAnalysis__Warren__Badelf
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/workflows/badelf/badelf.py", line 4, in <module>
    from simmate.apps.warrenapp.models import BadElfAnalysis
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/models/__init__.py", line 9, in <module>
    from .badelf import BadElfAnalysis
  File "/home/sweav/Documents/github/simmate/src/simmate/apps/warrenapp/models/badelf.py", line 13, in <module>
    from simmate.database.base_data_types import Calculation, Structure, table_column
  File "/home/sweav/Documents/github/simmate/src/simmate/database/base_data_types/__init__.py", line 7, in <module>
    from .base import DatabaseTable, table_column, SearchResults
  File "/home/sweav/Documents/github/simmate/src/simmate/database/base_data_types/base.py", line 244, in <module>
    class DatabaseTable(models.Model):
  File "/home/sweav/anaconda3/envs/simmate_dev/lib/python3.11/site-packages/django/db/models/base.py", line 127, in __new__
    app_config = apps.get_containing_app_config(module)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sweav/anaconda3/envs/simmate_dev/lib/python3.11/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
    self.check_apps_ready()
  File "/home/sweav/anaconda3/envs/simmate_dev/lib/python3.11/site-packages/django/apps/registry.py", line 137, in check_apps_ready
    settings.INSTALLED_APPS
  File "/home/sweav/anaconda3/envs/simmate_dev/lib/python3.11/site-packages/django/conf/__init__.py", line 92, in __getattr__
    self._setup(name)
  File "/home/sweav/anaconda3/envs/simmate_dev/lib/python3.11/site-packages/django/conf/__init__.py", line 72, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

@jacksund ever seen anything like this? It feels super strange to me. For now I'm just running isort/black and copying back the original import order on the two files that are effected, but I'm hoping I'm missing something really basic so I don't need to do that every time.

[jacksund]

when something goes wrong after running isort, one of two things could be happening:

1. you're importing a django table before django apps & settings are actually registered (your error)
2. you have a circular import somewhere

Your error is with django config, which I normally see with the first case.

This error is the reason I have from simmate.database import connect before I ever import a table:

from simmate.database import connect   # sets up django to avoid the error you're seeing
from simmate.database.third_parties import MatprojStructure

Behind the scenes, the import connect is really just calling the django-setup method (see here and here)

I'm surprised you see this error because importing the simmate.engine module (which holds the base Workflow class) fixes this for you here. Basically, this error should never happen if all of your workflows are inheriting from the base workflow class.

Do you have a module somewhere that imports a table before either import connect or import Workflow (or a subclass of that workflow) is called? This could be tricky to track down, so if you had a PR in your repo, I could test locally and try to debug for you. If you can, share the working version before isort is ran

[SWeav02]

I currently have a working branch before I ran isort under my badelf_class1 branch here. I found at least one instance where I didn't call import connect/workflow before calling a model is called, so I'm checking if that was the problem now. I'll keep you updated.

[jacksund]

okay! let me know how testing goes. I'll wait before trying to reproduce your dev env myself

[SWeav02]

Ok I think found the source (or maybe just a solution)! I did find at least one workflow script I had made where I didn't call import connect/workflow before calling a model. I fixed that one but still had the issue. Instead of searching through each file, I added from simmate.database import connect to the init.py file in my models folder. Now wherever its being imported out of order its going to run import connect first anyways. Let me know if you think that's a reasonable solution or if its against best practice in some way.

[jacksund]

I wouldn't add it to your models module because django will probably throw a fit. It can't run that line before reading/registering all models, so by adding it to the init, you create a circular import.

But you can try adding from simmate.database import connect to the __init__.py of your workflows module. If you add that, you might have to set the __all__ parameter too (see here in the docs).

But yeah, this is kind of a hacky fix. Let me know if it works, but I still might go through and find where your imports are broken

[SWeav02]

Yep, it's working with that fix. I may also go through and look for the import error at some point, but I may not have time for a bit. If you happen to and find the error please let me know!