Add shared infrastructure for independently reviewed Python guides by n0tl3ss · Pull Request #1914 · micronaut-projects/micronaut-guides

n0tl3ss · 2026-10-06T14:29:14Z

Shared prerequisite for the individually reviewed Python guide PRs. Merge this before those guide contributions.

Scope:

Python project, configuration/resource, source-macro, ZIP/index and reusable test-script support.
Metadata/schema support for Python options while retaining JVM feature isolation.
Rendering order, filtered regeneration, docs-only semantics, and focused regressions.
Official released Starter dependency and master-targeted generation CI; no fork checkout or platform SNAPSHOT downgrade.

No published guide content is included. Original source contributors are credited in the extraction commit.

Validation:

Full buildSrc:test: 149 executed tests in 46 suites, zero failures/errors/skips.
All 16 currently declared master Python guides generate projects and reusable test scripts.
Representative first-application Python documentation generation passes.
Whitespace and exact extraction-scope checks pass.

No native image was built. This validates generator support, not every guide runtime, Windows execution or cloud deployment. Azure-specific support requiring the pending Starter API is deliberately excluded.

Follow-up: reusable guide-base templates, then one master-targeted PR per guide with explicit prerequisite links.

CI corrections (normal new commits):

Fixes Python test-script generator silently omits application commands in CI #1916: keep complete Python test commands in CI-generated scripts. Existing regression RED with CI=true; GREEN across all 149 infrastructure tests, with all 16 generated Python scripts retaining actual commands.
Fixes General guide CI needs explicit JVM-only runtime selection for Python-enabled samples #1922: the general Test Guides workflow explicitly uses JVM-only runtime selection; 12 matrix-selector controls are GREEN. Original JVM/native tasks and Python generation/docs/ZIP/script tasks remain. Default task selection and dedicated Python runtime tasks are unchanged; this hosted workflow does not claim Python native execution.

First five master-targeted child PRs (all draft; each depends only on this prerequisite): #1919 Basic Auth, #1917 HTTP Client, #1918 GraphQL, #1920 Oracle ETag, #1921 Data MongoDB synchronous. Their full tested source is unchanged, fresh Python/Java rendering passes, and their prior source assessment remains 4/5 rather than an unverified 5/5.

Preserve committed source and existing tests; separate shared prerequisites from guide changes. Co-authored-by: Graeme Rocher <graeme.rocher@oracle.com> Co-authored-by: Nemanja Mikic <n0t_l3ss@outlook.com> Co-authored-by: Nemanja Mikic <nemanja.mikic@oracle.com> Co-authored-by: Sergio del Amo <sergio.delamo@softamo.com>

Keep the released platform coordinates, remove the obsolete Starter fork checkout, and validate master, python and python-guide-support without native image compilation.

Remove the obsolete environment-only skip; dedicated Python selection already separates Python and JVM scripts. The existing DocsOnlyGuideTest reproduces the bug with CI=true and the complete 149-test suite passes after the fix. Fixes #1916

n0tl3ss · 2026-10-06T14:55:37Z

First per-guide master review batch (each depends on this PR only):

Guide	PR	Retained complete native suite
Basic authentication	#1919	3/3
HTTP client	#1917	2/2
GraphQL	#1918	3/3
Oracle ETag	#1920	5/5
Synchronous Data MongoDB	#1921	8/8

All five preserve the previously tested source and original assertions; fresh Python/Java document generation and rendered HTML checks pass. They remain drafts pending this prerequisite, applicable CI and final Pythonic acceptance. None depends on #1915's family bases.

Hosted CI exposed #1916: the Groovy generator omitted Python commands whenever CI was set. The unchanged five-test regression reproduced RED (one failure), and removing the obsolete three-line generation guard gives GREEN for all 149 infrastructure tests with CI=true. All 16 current Python scripts now retain their actual test commands. The correction is a normal new commit, also merged normally into the child branches; no force push or native-image build.

Keep default guide task selection unchanged. The general workflow excludes only existing dedicated Python runtime tasks, while retaining JVM/native tests and Python generation, docs, ZIPs and scripts. A separately configured Python runtime workflow can use the original task selection. Matrix regression: 12 tests RED before the option, 12 GREEN after it; default selection, implicit/explicit apps, docs-only apps, JVM guides, transitive bases, deduplication and CLI controls preserved. Fixes #1922

Run on pushes to master and pull requests targeting master only. Keep the existing path filters and validation steps unchanged.

Create missing output parents in the shared generator and preserve file-conflict errors. Cover fresh nested output and repeated generation. Remove the inert python schema shorthand, retain canonical language parsing, and assert the actual Python language icon. Refs #1910; addresses review feedback on #1915.

n0tl3ss · 2026-10-06T20:18:38Z

Added normal commit d477857 for the review feedback inherited by #1915 and its clean-checkout CI failure. The shared generator now creates missing output parents; schema no longer advertises the inert python shorthand; index tests assert the actual Python PNG asset. Regression coverage retains the canonical explicit language path, repeated generation and safe file-conflict rejection.

Full buildSrc suite: 152 passing tests, zero skips/failures/errors. All 16 declared Python projects/test scripts generate. The fix is also normally merged into #1915, where all four docs-only cloud database Build tasks pass from a fresh output tree. Master-only CI branch filters remain in place. No force push or local native image build.

Handle inherited applications in the same sorted pass as GuideUtils and treat omitted applications as empty. Verify parser language defaults and all four language-filter renderings. Trigger Python validation when the local-composite build configuration changes.

n0tl3ss · 2026-10-06T21:34:48Z

Follow-up for the three summary-only findings in #1915: new commit 1ffbfa7 fixes the shared selector and workflow here in the prerequisite. The selector now mirrors GuideUtils’ sorted in-place application merge: it accounts for inherited applications when the child explicitly enables Python, and omitted applications are empty. The real parser control confirms that omitted languages instead default to Java/Groovy/Kotlin; there is no Python language inheritance to add.

The expanded selector suite had 17 executed / three failures before the production fix and now passes 17/17. The real-parser regression plus all-four-language marker coverage pass in the complete 153-test infrastructure suite (zero failures/errors/skips). Existing Python generation/script tasks for all 16 master samples also complete successfully. Both master-only Python workflow path lists now include gradle/local-composite-builds.gradle, verified structurally.

This was normally merged into #1915; no history rewrite or local native-image build. Fresh hosted checks are running on 1ffbfa7. Issue #1922 records the reproducer and clarifies the parser semantics.

Fingerprint the merged Guide record value, so base-only application changes invalidate generated scripts without requiring serializer service loading during Gradle input evaluation. Normalize mixed-case identifiers with underscores in both existing filename helpers. Add parser-backed input and filesystem-independent exclusion regressions. Fixes #1923 Fixes #1925

Copilot

Copilot review overview

🟡 Changes recommended

Python setup failures can report success, and skipped Python tests still require local runtime staging.

Review effort: Balanced
Findings: 4

Open (4)

Test path normalization mishandles underscored names · New
Skipped tests still require a local Pyronaut checkout · New
POM dependencies are rendered as ordinary dependencies · New
Setup failures are ignored in generated test scripts · New

What changed in this PR

Adds shared Python support to the guide infrastructure as a prerequisite for separately reviewed guide contributions.

Changes:

Extends Python metadata, project generation, resources, macros, and packaging.
Adds reusable test scripts and regressions for documentation-only guides and regeneration.
Updates Starter, local composite builds, and CI runtime selection.

File	Description
src/docs/common/snippets/common-enable-distributed-configuration.adoc	Adds Python configuration guidance.
settings.gradle	Loads local composite configuration.
README.md	Documents local checkout options.
gradle/local-composite-builds.gradle	Configures optional local builds.
buildSrc/src/test/resources/file-transfer/python-application-config/src/test/resources/bootstrap-test.properties	Adds inherited test configuration fixture.
buildSrc/src/test/resources/file-transfer/python-application-config/src/main/resources/bootstrap.properties	Adds inherited bootstrap fixture.
buildSrc/src/test/resources/file-transfer/python-application-config/python/tests-config/application-test.toml	Adds Python test configuration fixture.
buildSrc/src/test/resources/file-transfer/python-application-config/python/config/application.toml	Adds Python application configuration fixture.
buildSrc/src/test/resources/file-transfer/python-application-config/metadata.json	Defines resource-transfer fixture metadata.
buildSrc/src/test/resources/expected_test_script.sh	Removes documentation-only directory commands.
buildSrc/src/test/resources/expected_test_script_native.sh	Updates native-script expectations.
buildSrc/src/test/java/io/micronaut/guides/tasks/SampleProjectGenerationTaskTest.java	Tests filtered regeneration and cleanup.
buildSrc/src/test/java/io/micronaut/guides/tasks/PythonTestScriptTaskTest.java	Checks helper-resource task input.
buildSrc/src/test/java/io/micronaut/guides/IndexGeneratorTest.java	Tests Python links and table layout.
buildSrc/src/test/java/io/micronaut/guides/GuidesPluginTest.java	Tests documentation rendering order.
buildSrc/src/test/java/io/micronaut/guides/GuideProjectGeneratorTest.java	Tests resource isolation and symlink safety.
buildSrc/src/test/java/io/micronaut/guides/GuideAsciidocGeneratorTest.java	Tests language-filter boundaries.
buildSrc/src/test/java/io/micronaut/guides/DocsOnlyGuideTest.java	Tests documentation-only generation semantics.
buildSrc/src/test/java/io/micronaut/guides/core/TestScriptGeneratorTest.java	Tests dedicated Python scripts.
buildSrc/src/test/java/io/micronaut/guides/core/TestMacroSubstitutionTest.java	Tests Python test includes.
buildSrc/src/test/java/io/micronaut/guides/core/SourceMacroSubstitutionTest.java	Tests Python source includes.
buildSrc/src/test/java/io/micronaut/guides/core/MacroUtilsTest.java	Tests Python name normalization.
buildSrc/src/test/java/io/micronaut/guides/core/GuideUtilsTest.java	Tests inherited language-specific features.
buildSrc/src/test/java/io/micronaut/guides/core/GuideTest.java	Updates validation and schema expectations.
buildSrc/src/test/java/io/micronaut/guides/core/GuideProjectZipperTest.java	Makes ZIP assertions order-independent.
buildSrc/src/test/java/io/micronaut/guides/core/GuideProjectGeneratorTest.java	Tests Python H2 runtime generation.
buildSrc/src/test/java/io/micronaut/guides/core/GuideParserTest.java	Tests Python metadata defaults.
buildSrc/src/test/java/io/micronaut/guides/core/GuideGenerationUtilsTest.java	Tests Python test-framework selection.
buildSrc/src/test/java/io/micronaut/guides/core/FilesTransferUtilityTest.java	Tests bootstrap configuration isolation.
buildSrc/src/test/java/io/micronaut/guides/core/DependencyMacroSubstitutionTest.java	Tests TOML dependency rendering.
buildSrc/src/test/java/io/micronaut/guides/core/AppTest.java	Updates schema URL expectations.
buildSrc/src/main/resources/pyronaut-test-functions.sh	Adds reusable Python test helpers.
buildSrc/src/main/resources/pom.xml	Adds H2 dependency coordinates.
buildSrc/src/main/resources/guide-metadata.schema.json	Documents Python options and empty apps.
buildSrc/src/main/java/io/micronaut/guides/feature/SpringSecurityCrypto.java	Adds fallback security feature support.
buildSrc/src/main/java/io/micronaut/guides/feature/H2Runtime.java	Adds runtime-only H2 feature.
buildSrc/src/main/java/io/micronaut/guides/core/ZipIncludeMacroSubstitution.java	Adapts source-title signature.
buildSrc/src/main/java/io/micronaut/guides/core/TestScriptGenerator.java	Exposes Python script generation.
buildSrc/src/main/java/io/micronaut/guides/core/SourceBlockMacroSubstitution.java	Supports Python paths and includes.
buildSrc/src/main/java/io/micronaut/guides/core/MacroUtils.java	Adds Python module-name conversion.
buildSrc/src/main/java/io/micronaut/guides/core/GuideUtils.java	Merges Python options and filters bootstrap files.
buildSrc/src/main/java/io/micronaut/guides/core/GuideGenerationUtils.java	Resolves Python source/test paths.
buildSrc/src/main/java/io/micronaut/guides/core/Guide.java	Allows empty apps and Python test skipping.
buildSrc/src/main/java/io/micronaut/guides/core/DependencyLines.java	Renders Pyronaut dependency snippets.
buildSrc/src/main/java/io/micronaut/guides/core/DefaultTestScriptGenerator.java	Separates Python and JVM scripts.
buildSrc/src/main/java/io/micronaut/guides/core/DefaultGuideProjectGenerator.java	Removes Python package markers.
buildSrc/src/main/java/io/micronaut/guides/core/DefaultGuideParser.java	Normalizes Python metadata options.
buildSrc/src/main/java/io/micronaut/guides/core/DefaultFilesTransferUtility.java	Handles Python resources and exclusions.
buildSrc/src/main/java/io/micronaut/guides/core/App.java	Adds Python-specific features.
buildSrc/src/main/groovy/io/micronaut/guides/TestScriptGenerator.groovy	Generates complete dedicated Python scripts.
buildSrc/src/main/groovy/io/micronaut/guides/tasks/TestScriptTask.groovy	Honors configured script filenames.
buildSrc/src/main/groovy/io/micronaut/guides/tasks/TestScriptRunnerWorkAction.groovy	Executes the selected script.
buildSrc/src/main/groovy/io/micronaut/guides/tasks/SampleProjectGenerationTask.groovy	Limits filtered regeneration cleanup.
buildSrc/src/main/groovy/io/micronaut/guides/tasks/PythonTestScriptTask.groovy	Adds cacheable Python script generation.
buildSrc/src/main/groovy/io/micronaut/guides/IndexGenerator.groovy	Adjusts Python build-language tables.
buildSrc/src/main/groovy/io/micronaut/guides/GuidesPlugin.groovy	Wires Python tasks and documentation ordering.
buildSrc/src/main/groovy/io/micronaut/guides/GuideProjectGenerator.groovy	Adds Python assembly and resource handling.
buildSrc/settings.gradle	Enables local composites for buildSrc.
buildSrc/build.gradle	Updates Starter and schema base URL.
.github/workflows/python-guides.yml	Adds generation-only Python validation.
.github/workflows/gradle.yml	Selects JVM-only runtime execution.
.github/scripts/test_guide_ci_tasks.py	Tests runtime-selection controls.
.github/scripts/guide_ci_tasks.py	Adds Python runner exclusions.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        if (target.contains('_') || target == target.toLowerCase(Locale.ENGLISH)) {
+            return target


+            it.environment.set(metadata.env())
+            it.environment.put("PYRONAUT_LOCAL_REPOSITORY", localPyronautRepository)
+            it.environment.put("PYRONAUT_LOCAL_CORE_VERSION", localPyronautCoreVersion)
+            it.environment.put("PYRONAUT_LOCAL_PLATFORM_VERSION", localPyronautPlatformVersion)
+            it.environment.put("PYRONAUT_INSTALL_EXECUTABLE", localPyronautInstallExecutable)
+            it.environment.put("PYRONAUT_VALIDATE_CONFIG_EXECUTABLE", localPyronautValidateConfigExecutable)
+            it.environment.put("PYRONAUT_PROCESS_EXECUTABLE", localPyronautProcessExecutable)
+            it.environment.put("PYRONAUT_PROCESSOR_EXECUTABLE", localPyronautProcessExecutable)
+            it.environment.put("PYRONAUT_TEST_EXECUTABLE", localPyronautTestExecutable)
+            it.environment.put("PYRONAUT_TEST_RESOURCES_SERVER_EXECUTABLE", localPyronautTestResourcesServerExecutable)
+            it.environment.put("PYRONAUT_CLI_PYTHONPATH", localPyronautCliPythonPath)
+            it.testScript.set(pythonTestScriptTask.flatMap { t -> t.scriptFile })
+            it.guideSourceDirectory.set(project.layout.projectDirectory.dir("guides/${metadata.slug()}"))
+            it.outputFile.set(codeDirectory.map(d -> d.file("python-output.log")))
+            it.dependsOn(stageLocalPyronautArtifactsTask)


+            String coordinate = groupId + ':' + artifactId + (version != null ? ':' + version : "");
+            String callout = extractCallout(attributes);
+            String tomlCallout = callout.isEmpty() ? "" : " #" + callout.substring(" //".length());
+            dependencies.get(toPyronautScope(attributes)).add("\"" + coordinate + "\"," + tomlCallout);


+    set_pyronaut_local_versions
+    set_pyronaut_test_resources
+    set_pyronaut_native_toolchain
+    set_pyronaut_project_venv
+    run_pyronaut_install && run_pyronaut_validate_config && run_pyronaut_process && run_pyronaut_test
+    return $?
+  fi
+
+  local tmp_dir
+  tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/pyronaut-guide.XXXXXX")"
+  tar --exclude='./.micronaut' --exclude='./__pyronaut__' --exclude='./.pytest_cache' --exclude='./build' --exclude='./dist' --exclude='./*.egg-info' -cf - . | (cd "$tmp_dir" && tar -xf -)
+  (
+    cd "$tmp_dir"
+    set_pyronaut_local_repositories
+    set_pyronaut_local_versions
+    set_pyronaut_test_resources
+    set_pyronaut_native_toolchain
+    set_pyronaut_project_venv
+    run_pyronaut_install && run_pyronaut_validate_config && run_pyronaut_process && run_pyronaut_test


n0tl3ss and others added 2 commits October 6, 2026 16:23

Use released official Starter for Python guide validation on master

e0add73

Keep the released platform coordinates, remove the obsolete Starter fork checkout, and validate master, python and python-guide-support without native image compilation.

This was referenced Oct 6, 2026

Add reusable Python guide bases and family snippets #1915

Draft

Python test-script generator silently omits application commands in CI #1916

Open

n0tl3ss mentioned this pull request Oct 6, 2026

General guide CI needs explicit JVM-only runtime selection for Python-enabled samples #1922

Open

n0tl3ss requested a review from sdelamo October 6, 2026 15:05

Restrict Python guide CI triggers to master

8b5253b

Run on pushes to master and pull requests targeting master only. Keep the existing path filters and validation steps unchanged.

n0tl3ss mentioned this pull request Oct 6, 2026

Support documentation-only guides without unrelated sample applications #1910

Open

n0tl3ss requested a balanced review from Copilot October 6, 2026 21:56

n0tl3ss mentioned this pull request Oct 6, 2026

Python test-script task inputs omit inherited guide metadata #1923

Open

Copilot started reviewing on behalf of n0tl3ss October 6, 2026 21:57 View session

n0tl3ss mentioned this pull request Oct 6, 2026

Mixed-case underscore Python exclusions retain incorrectly cased paths #1925

Open

Copilot AI reviewed Oct 6, 2026

View reviewed changes

Navigation Menu

Repository navigation

Add shared infrastructure for independently reviewed Python guides - #1914

Add shared infrastructure for independently reviewed Python guides#1914
n0tl3ss wants to merge 8 commits into
masterfrom
codex/python-guides-shared-infrastructure-20261006

n0tl3ss commented Oct 6, 2026 •

edited

Loading

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if (target.contains('_') \|\| target == target.toLowerCase(Locale.ENGLISH)) {
		return target

Conversation

n0tl3ss commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

n0tl3ss commented Oct 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

n0tl3ss commented Oct 6, 2026 •

edited

Loading