Repository reorganisation

[mlange05]

As we are maturing the outward-facing components of Loki, we have been contemplating a slight repository reorganisation to better encapsulate the different layers of the API and organise the re-use within these. I'm filing this as an issue here to draw up a first draft of the overall shape and contours of the sub-packages that we would long-term envision. This is primarily meant for discussion and is open to input from all.

Moreover, I think @reuterbal had also suggested in offline discussions to potentially split up the test base and move the specific tests into the respective sub-packages, allowing us to tighten up the interfaces a little.

Here a first draft organisation that we should probably evolve over time:

-loki
    - analyse
    - backend         | fgen, cgen, pygen, etc
    - batch              | scheduler and associated items, configs etc
        - tests
        - scheduler
        - items
        - config
        - filter
    -cli
    - expression     | The expression/symbol layer alongside it's pymbolic interfaces and visitors
        - tests
        - symbols                    | Only variables / literals?
        - operations                | I'd like to separate operations (inlinecall, subscript, sum, ...) from the symbols/literals
        - symbolic
        - expr_visitor
    - frontend
    - ir                    | I'd like to merge the middle-layer (IR plus IR-visitors); better name needed!
        - tests
        - nodes
        - visitor
        - transformer
        - pragma_utils
        - scope
        - program_unit
        - subroutine/module
        - sourcefile
    - transformations
        - tests
        - general
        - inline_outline
        - single_column
        - transpile
        - temporaries
        - offload

Just a few general ideas, open for comments... @reuterbal @awnawab @MichaelSt98

[reuterbal]

Just a few ideas off the top of my head:

• We should also add a cli module that could provide common CLI functionality that is currently replicated between the scripts, similar to what has been done in ifsbench. This could also provide a common "loki" CLI command and retain the "loki-transform.py" and "loki-lint.py" as additional aliases. That way, the scripts directory could be eliminated
• Naming proposals:

- expression
  - tests
  - symbols                    | Only variables / literals?
  - operations                | I'd like to separate operations (inlinecall, subscript, sum, ...) from the symbols/literals
  - symbolic
  - expr_visitor
- ir
  - tests
  - nodes
  - visitor
  - transformer
  - pragma_utils

• While we reshuffle the tests, we should also retire the habit of writing stuff to the current directory. Instead, we should always use a test-local temporary working directory (via a fixture with auto-cleanup), and also supply this as xmods directory for OMNI. That avoids spurious failures/false sharing between tests simply due to naming conflicts, and removes the cluttering with files.

[mlange05]

Just a quick update, after offline discussion: Due to the inherent circular dependencies between subroutine/module, the IR nodes/visitors and the scope/type classes, we will probably move scope/type and the program_units (subroutine/module) all into loki.ir and potentially separate them further inside loki.ir at a later stage.

[reuterbal]

Just noticed: We need to update references in the Sphinx docs accordingly now.