Skip to content

Conversation

@riturajFi
Copy link
Contributor

@riturajFi riturajFi commented Oct 1, 2025

Closes #746

Problem

The Python step runner was incorrectly setting module.__package__ to only the last directory name (e.g. "endpoints").
As a result, relative imports like:

from ..repositories import db_repo

failed with:

ImportError: attempted relative import beyond top-level package

This prevented Motia from correctly loading step files with nested package structures.


Solution

  • Updated the runner logic in get-config.py to compute the full dotted module path relative to the steps/ directory.

    • Example:

      • File: steps/endpoints/python_api_step.py
      • Old package: "endpoints"
      • New package: "steps.endpoints"
      • Full module name: "steps.endpoints.python_api_step"
  • Set module.__package__ and sys.modules[...] to this full dotted name before executing the module.

  • This ensures Python has the correct package context, so relative imports always resolve properly.

Screencast.from.2025-10-01.09-54-08.webm

Benefits

  • Relative imports (..repositories, ..utils, etc.) now work across all nested step files.
  • No need to rewrite individual step files.
  • Future-proof: supports arbitrary nesting depth under the steps/ directory.
  • Compatible with namespace packages (no __init__.py required).

Example

Before (broken):

module.__package__ = "endpoints"
ImportError: attempted relative import beyond top-level package

After (fixed):

module.__package__ = "steps.endpoints"
Relative import '..repositories' resolves correctly

@vercel
Copy link

vercel bot commented Oct 1, 2025

@riturajFi is attempting to deploy a commit to the motia Team on Vercel.

A member of the Team first needs to authorize it.

@rohitg00
Copy link
Contributor

rohitg00 commented Oct 1, 2025

Thanks @riturajFi, let me check with the team to review this quickly, thanks!

@sergiofilhowz
Copy link
Contributor

sergiofilhowz commented Oct 13, 2025

I'm trying to test it and doesn't seem to work, am I doing anything wrong?

Captura de Tela 2025-10-13 às 15 03 39

api2_step.py

from pydantic import BaseModel
from typing import Optional
from ..python.repo import Repo

class PetRequest(BaseModel):
    name: str
    photo_url: str

class FoodOrder(BaseModel):
    id: str
    quantity: int

class RequestBody(BaseModel):
    pet: PetRequest
    food_order: Optional[FoodOrder] = None

config = {
    "type": "api",
    "name": "PythonApiTrigger2",
    "description": "basic-tutorial api trigger",
    "flows": ["python-tutorial"],
    "method": "POST",
    "path": "/python-basic-tutorial2",
    "bodySchema": RequestBody.model_json_schema(),
    "emits": [],
}

async def handler(req, context):
    body = req.get("body", {})
    context.logger.info("Step 01 – Processing API Step", {"body": body})

    repo = Repo()
    context.logger.info("Repo", {"repo": repo.say_hello()})

    return {"status": 200, "body": {"traceId": context.trace_id}}

@riturajFi
Copy link
Contributor Author

Checking this asap

@riturajFi
Copy link
Contributor Author

riturajFi commented Oct 20, 2025

Hi @sergiofilhowz,

Here’s the detailed process to reproduce the bug and verify the fix:


🧪 Steps to Reproduce (on main branch)

  1. Build the project.

  2. Create the following folder structure:

    /steps
      ├── __init__.py
      ├── /endpoints
      │    ├── __init__.py
      │    └── python_api_step.py
      └── /repositories
           ├── __init__.py
           └── db_repo.py
    
  3. Add the following contents:

python_api_step.py

from ..repositories.db_repo import get_data

config = {
    "type": "api",
    "name": "nestedpython",
    "path": "/api",
    "method": "GET",
    "emits": []
}

async def handler(req, ctx):
    return {
        "status": 200,
        "body": get_data()
    }

db_repo.py

def get_data():
    return 1
  1. Run the project.
    You’ll see this error:

    Error running Python module: attempted relative import beyond top-level package
    

✅ Verification (on feature branch)

After switching to this feature branch and repeating the same steps, the project runs successfully:
You can also hit the api and check the successfull implementation of relative imports in python steps -

➜ [INFO] Activating Python environment 
➜ [INFO] Using Python <projectDir>/python_modules/bin/python
➜ [INFO] Site-packages path <projectDir>/python_modules/lib/python3.13/site-packages
➜ [REGISTERED] Step (API) steps/endpoints/python_api_step.py registered
[motia-plugins] ✓ Validated 3 plugin(s) successfully
[motia-plugins] Initialized with 3 plugin(s)
[motia-plugins] Dev server configured, HMR enabled
[motia-plugins] Build started
🚀 Server ready and listening on port 3000
🔗 Open http://localhost:3000 to open workbench 🛠️

@riturajFi riturajFi force-pushed the fix/python-nested-step-relative-import branch from 81b0750 to 592b032 Compare October 20, 2025 14:04
@sergiofilhowz
Copy link
Contributor

Thanks for providing this information, however, the issue happens on my project, we need to ensure it doesn't happen.

image

@riturajFi riturajFi force-pushed the fix/python-nested-step-relative-import branch from 592b032 to aeb978a Compare November 1, 2025 15:24
@github-actions github-actions bot added the size/S label Nov 1, 2025
@github-actions github-actions bot added the size/M label Nov 2, 2025
@riturajFi riturajFi force-pushed the fix/python-nested-step-relative-import branch from 1253bc8 to 6640f70 Compare November 2, 2025 16:43
@riturajFi riturajFi force-pushed the fix/python-nested-step-relative-import branch from d6aacf2 to 4325125 Compare November 4, 2025 10:54
@github-actions
Copy link

github-actions bot commented Nov 4, 2025

⚠️ This PR is quite large (>1000 lines). Consider splitting it into smaller PRs for easier review.

const logsPlugin = require('@motiadev/plugin-logs/plugin')
const observabilityPlugin = require('@motiadev/plugin-observability/plugin')

export default config({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry i placed these by mistake

@@ -0,0 +1,17 @@
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file necessary?

project_root = steps_dir.parent
project_parent = project_root.parent
if str(project_parent) not in sys.path:
sys.path.insert(0, str(project_parent))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include it at position 0? This will require a bit shift of all elements. If it's not necessary, can we use append?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants