Skip to content

Reserve hydra.overlay.<lang>.* for all host-native code; reserve hydra.<lang>.* for translingual modules #501

Description

@joshsh

Summary

Establish a hard, project-wide namespace separation between translingual code (authored in Hydra, host-independent identity, including the generated coders, DSLs, and primitive definitions) and host-native hand-written code (impls, authoring DSLs, tooling — meaningful only in one host language).

  • hydra.* (incl. hydra.<lang>.*, hydra.dsl.*, hydra.lib.*) becomes exclusively translingual.
  • hydra.overlay.<lang>.* becomes the exclusive home for all hand-written host-native code, one sub-namespace per host.

This makes the two axes — translingual vs. host-specific and per host language — fully non-overlapping by construction, eliminating an entire class of latent module-name collisions.

Motivation

The codebase is already converging on this split but incompletely and inconsistently, which creates real and latent collisions:

  1. hydra.dsl.* is now a generated output namespace (Generate DSLs for term and primitive modules as well as type modules #467: every kernel module projects to hydra.dsl.<module>). Hand-written authoring helpers of the same name (Hydra.Dsl.Annotations, Hydra.Dsl.Literals) collide with the generated DSLs — the all-module widening already fails to compile with Duplicate modules: Hydra.Dsl.Annotations.
  2. Hydra.<Lang>.* mixes translingual and host-native. Hydra.Haskell.* holds both the generated translingual Haskell coder (Hydra.Haskell.Coder, .Syntax, .Language, .Testing, …) and hand-written impls (Hydra.Haskell.Lib.*). They avoid a clash today only by luck of non-overlapping leaf names, not by design.
  3. Host-native impl relocation is half-done. Per the architecture in bootstrap-from-json ("IMPLEMENTATIONS live at hydra.<lang>.lib.* … so hydra.lib.* is free for the generated PrimitiveDefinition def-modules"), Haskell (Hydra.Haskell.Lib.*), Python, Scala, and Clojure have relocated their impls — but Java is the laggard (263 impls still at hydra.lib.*, polluting the translingual namespace).

A single reserved root for host-native code resolves all three uniformly and is provably collision-free: no kernel module is named overlay, so the generator never emits into hydra.overlay.*.

Proposed scheme

Reserved roots:

  • hydra.* / hydra.<lang>.* / hydra.dsl.* / hydra.lib.* — translingual only (generated coders, generated DSLs, kernel defs, generated PrimitiveDefinitions).
  • hydra.overlay.<lang>.* — all hand-written host-native code.

Per-host conventions (consistent scheme across hosts; concrete files differ per host):

  • Authoring DSLs: hydra.overlay.<lang>.dsl.* — raw/value-level constructors sit flat; phantom-typed helpers go under a .typed sub-segment (hydra.overlay.<lang>.dsl.typed.*). No .raw segment.
  • Host-native primitive impls: hydra.overlay.<lang>.lib.*.
  • Tooling / util / runtime support / glue: hydra.overlay.<lang>.{tools,util,json,...}.
  • Runtime registry (Haskell Libraries/Bootstrap) sits flat under hydra.overlay.<lang>.* — it is not a DSL (it produces a runtime artifact, not an authoring vocabulary).

Migration map (big three; approximate counts)

Haskell → Hydra.Overlay.Haskell.*

  • Hydra.Dsl.{Terms,Types,Literals,Annotations,LiteralTypes,Tests,Prims} (11) → …Overlay.Haskell.Dsl.{…} (flat)
  • Hydra.Dsl.Meta.{Phantoms,Terms,Types,Literals,Core,Graph,Variants,Testing,Tabular,DeepCore,Common,Base} (12) → …Overlay.Haskell.Dsl.Typed.{…}
  • Hydra.Dsl.{AsTerm,AsType}…Overlay.Haskell.{AsTerm,AsType}
  • Hydra.Dsl.{Libraries,Bootstrap}…Overlay.Haskell.{Libraries,Bootstrap}
  • Hydra.Haskell.Lib.* (16) → …Overlay.Haskell.Lib.*
  • Hydra.{Kernel,Settings}…Overlay.Haskell.{Kernel,Settings}
  • Hydra.Dsl.Deep.Lib.MathDELETE (redundant with generated hydra.dsl.lib.math; migrate the one importer, Sources/Kernel/Terms/Differentiation.hs)

Java → hydra.overlay.java.*

  • hydra.lib.* (263) → hydra.overlay.java.lib.* (also resolves the Java laggard)
  • hydra.dsl.* + hydra.dsl.prims.*hydra.overlay.java.dsl.{…,prims.*}
  • hydra.tools.*, hydra.util.*, hydra.json.*, hydra.{Coders,Adapters}hydra.overlay.java.{tools,util,json,Coders,Adapters}

Python → hydra.overlay.python.*

  • hydra.python.lib.* (17) → hydra.overlay.python.lib.*
  • hydra.dsl.* (raw, flat), hydra.dsl.meta.*hydra.overlay.python.dsl.typed.*, hydra.dsl.prims, hydra.python.util, hydra.toolshydra.overlay.python.{…}

Other already-relocated hosts: hydra.scala.lib.*, hydra.clojure.lib.* (and the remaining Lisp dialects) → hydra.overlay.<lang>.lib.*.

Generator / coder changes (the non-mechanical part)

This is not a pure file-and-import rename — the generated coders reference host-native support and must emit updated paths:

  1. bootstrap-from-json impl-redirect (heads/haskell/src/exec/bootstrap-from-json/Main.hs, the redirectForSubs helper): change the redirect target "hydra." ++ langSeg ++ ".lib.""hydra.overlay." ++ langSeg ++ ".lib.", so generated consumer code resolves primitive calls to the relocated impls.
  2. Host runtime-support imports: wherever a coder emits an import of host-native support (e.g. Java generated code importing hydra.util.Optional, hydra.tools.PrimitiveFunction), the coder must emit the new hydra.overlay.java.{util,tools}.* paths.

Scope / non-goals

  • In scope: the namespace migration across all hosts + the coder-emission changes above.
  • Out of scope (enabled by this, tracked separately): the term-DSL generation widening (Generate DSLs for term and primitive modules as well as type modules #467 follow-on), and the Gap-1 / Gap-2 "self-contained shippable hydra-kernel" work (ship authoring DSLs + runtime primitive registry). This issue is the prerequisite that frees hydra.dsl.* for the generator and gives host-native code a clean home.

Execution notes

  • Carried out in a dedicated worktree/branch.
  • Large but mostly mechanical footprint (~2,000+ import-line edits across hosts); the coder changes are the substantive part.
  • Validate per host: full --local-host sync + stack test + /test (all targets) + /bootstrap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions