Skip to content

refactor: Implement Split-Directory Workspace with PEP 420 Namespace Packaging#166

Merged
jhutar merged 19 commits intomainfrom
jhutar3
Mar 18, 2026
Merged

refactor: Implement Split-Directory Workspace with PEP 420 Namespace Packaging#166
jhutar merged 19 commits intomainfrom
jhutar3

Conversation

@jhutar
Copy link
Contributor

@jhutar jhutar commented Mar 16, 2026

Summary

This PR reorganizes the repository structure to eliminate code duplication between the core library and the full installation.

Implementation detail: We have transitioned to a modern workspace/monorepo layout using PEP 420 Implicit Namespace Packages, allowing the opl namespace to be shared across physically separate directories.

Architectural Changes

  • Directory Split: Migrated all source code into core/opl/ (minimal dependencies) and extras/opl/ (heavy dependencies).
  • Deduplication: Removed the duplicated source tree in core/, establishing a single source of truth for all modules.
  • Meta-Packaging: Converted the root setup.py into a "meta-package" that orchestrates the installation of both sub-packages.

AI Software Architect Framework

This repository now utilizes the AI Software Architect framework for tracking architectural decisions and conducting specialist reviews.

It's core concept (as far as I understand it) is to allow you breaking changes into manageable steps and employing a automatic review by different AI personas to achieve better outcome.

Key Commands for AI Assistants

If you are using an AI assistant in this repository, you can now issue commands such as:

  • "What is our architecture status?"
  • "Create ADR for [Topic]"
  • "Start architecture review for [Version/Feature]"
  • "Ask [Specialist Role] to review [File/PR]"

Detailed architectural reasoning for this refactor can be found in (check is as an example of what can be done):

  • ADR-012: Reorganize Repository Structure using Split Directories - see file .architecture/decisions/adrs/ADR-012-reorganize-repository-structure-for-core-and-full-installations.md
  • Review: Architecture Review: ADR-012 split-directories - see file .architecture/reviews/ADR-012-split-directories.md

Validation

To test this, I successfully ran this test:

set -x
set -ue

rm -rf venv-*

python -m venv venv-extras-e
python -m venv venv-extras
python -m venv venv-core-e
python -m venv venv-core
python -m venv venv-all-e
python -m venv venv-all
python -m venv venv-dev-e

source venv-all/bin/activate
python -m pip install git+https://github.com/redhat-performance/opl.git@jhutar3
status_data.py -h
script-skip-to-end.py -h
deactivate

source venv-all/bin/activate
python -m pip install -e "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team"
status_data.py -h
script-skip-to-end.py -h
deactivate

source venv-core/bin/activate
python3 -m pip install "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-core&subdirectory=core"
status_data.py -h
script-skip-to-end.py -h && false
deactivate

source venv-core-e/bin/activate
python3 -m pip install -e "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-core&subdirectory=core"
status_data.py -h
script-skip-to-end.py -h && false
deactivate

source venv-extras/bin/activate
python3 -m pip install "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-core&subdirectory=core"
python3 -m pip install "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-extras&subdirectory=extras"
status_data.py -h
script-skip-to-end.py -h
deactivate

source venv-extras-e/bin/activate
python3 -m pip install -e "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-core&subdirectory=core"
python3 -m pip install -e "git+https://github.com/redhat-performance/opl.git@jhutar3#egg=opl-rhcloud-perf-team-extras&subdirectory=extras"
status_data.py -h
script-skip-to-end.py -h
deactivate

source venv-extras-e/bin/activate
python -m pip install -e .[dev]
status_data.py -h
script-skip-to-end.py -h
deactivate

echo "ALL GOOD"

jhutar added 19 commits March 15, 2026 17:27
- Set up .architecture directory with specialized reviewers and Pythonic principles
- Conducted initial system analysis and comprehensive repository review
- Added GEMINI.md for framework usage instructions

Prompts:
- Initialize architecture framework
- Conduct comprehensive review of this repository

Generated-by: Gemini
Generated-by: Gemini
- Marked Phase 1 as completed in ADR-012
- Removed __init__.py files from opl and core/opl to enable implicit namespace packaging
- Created extras/opl directory

Generated-by: Gemini
- Marked Phase 2 as completed in ADR-012
- Moved heavy-dependency tools from opl/ to extras/opl/
- Removed root opl/ directory as core/opl/ already contains the lightweight core tools

Generated-by: Gemini
- Marked Phase 3 as completed in ADR-012
- Updated core/setup.py to find_namespace_packages
- Created extras/setup.py for heavy dependencies
- Transformed root setup.py into a meta-package depending on core and extras

Generated-by: Gemini
- Marked Phase 4 as completed in ADR-012
- Added CI step to ensure __init__.py files are not created in namespace roots

Generated-by: Gemini
- Documented new core/ and extras/ split architecture
- Updated file paths in examples to point to their new locations
- Refined installation instructions for both core and full modes

Generated-by: Gemini
- Reverted the direct file URI dependency in extras/setup.py
- Added explicit documentation in README.md about manual core installation requirement for extras-only installs

Generated-by: Gemini
…gparse conflict

- Add 'responses' to dev dependencies in setup.py to fix ModuleNotFoundError.
- Update opl.skelet.test_setup to allow passing explicit arguments to parser.parse_args(), preventing interference from pytest command-line arguments.
- Update tests/test_skelet.py to use the new args parameter.

Prompt: A trest is failing: ... ImportError while importing test module '/home/jhutar/Checkouts/opl/tests/test_cluster_read.py'. ... ModuleNotFoundError: No module named 'responses' ... Please fix it
Generated-by: Gemini
- Add pytest-cov to development dependencies in setup.py.
- Create pytest.ini to configure coverage for core/opl and extras/opl.
- Configure terminal and HTML coverage reports.

Prompt: Implement test coverage measurements during the tests please.
Generated-by: Gemini
- Removed framework's own development files (.github, CHANGELOG, etc.)
- Removed framework's own ADRs (ADR-001 through ADR-011)
- Removed framework's own architecture reviews
- Retained only project-specific configuration, principles, ADR-012, and initial system analysis

Generated-by: Gemini
- Renamed ADR-012 to ADR-001 since it is the first ADR of the project
- Replaced references to ADR-012 with ADR-001 in architecture documents
- Trimmed .architecture/deferrals.md to a clean template to remove framework-specific deferrals

Generated-by: Gemini
- Add try/finally blocks in TestGenericGenerator to ensure temporary template files are properly cleaned up.
- Update README.md with information on how to find the generated test coverage reports.

Prompt: Before git adding and commiting, also add a anote about where to find codecov report to readme.
Generated-by: Gemini
- Added missing whitespace around arithmetic operators in hbi_utils.py and horreum_api.py.

Prompt: Please fix issues reported by `find core/ extras/ -name '*.py' -exec flake8 '{}' +`
Generated-by: Gemini
- Fix 'no-self-argument' in core/opl/shovel.py.
- Convert relative imports to absolute imports to support PEP 420 namespace packages across split directories.
- Fix missing 'org_id' argument in extras/opl/generators/qpc_tarball.py.
- Replace deprecated 'assertEquals' with 'assertEqual' in extras/opl/rbac_utils.py.
- Resolve psycopg2.errors and locust import issues with explicit imports and targeted pylint silences.

Prompt: Please fix `find setup.py core/ extras/ -name '*.py' -exec pylint-3 --errors-only '{}' +` in our code
Generated-by: Gemini
@jhutar
Copy link
Contributor Author

jhutar commented Mar 18, 2026

Checked current linter errors, and these are false positives I think.

No feedback, merging.

@jhutar jhutar merged commit be2c557 into main Mar 18, 2026
2 of 11 checks passed
@jhutar
Copy link
Contributor Author

jhutar commented Mar 18, 2026

Oops, had to delete bunch of DELME/* files :)

@jhutar jhutar deleted the jhutar3 branch March 18, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant