Support Dev Dependencies

[biniona]

Hi! My name is Alek, and I am interested in using this Buildpack in a tool to create both Development and Production images of an application.

I did a little digging through the codebase (I don't know Rust, so I relied pretty heavily on Claude), and it seems I'm not currently able to override which set of dependencies the Buildpack pulls in (I.E. something like modifying the pip install -r requirements.txt command to pip install -r dev-requirements.txt or modifying the uv sync command to uv sync --group dev).

Would it be possible to add one or more configuration options to this Buildpack to support this feature?

Possible Implementation

If I understand the Buildpack Specification correctly, this could be done through one or more environment variables of the form BP_*.

Would something like this make sense:

BP_PIP_REQUIREMENTS=<file with your requirements>
BP_UV_GROUP=<your uv group>

I believe if BP_PIP_REQUIREMENTS was set, then you would need to detect that file to pass detection. If BP_UV_GROUP was set, you would fail detection if uv.lock was missing.

I am unfamiliar with poetry, so I'm unsure what the appropriate mechanism for that tool would be.

PS Hi @edmorley! It was a pleasure meeting you at PyCon!

[amureki]

Hey there,

I stumbled upon this issue while experiencing something similar.

Among other settings, I have the following lines in my pyproject.toml:

[dependency-groups]
prod = [
    "brotli",
]
dev = [
    "...",
]
lint = [
    "...",
]

So, the brotli package I only want to have installed in production. Ideally, I would run uv sync --locked --group prod on Heroku.

This is currently impossible, so instead, I have to use workarounds by removing brotli on development machines, which uv does not allow without relocking the dependencies (modifying the uv.lock file).

Adding one or several groups via environment variables does not sound like a bad idea, but maybe the maintainers have a better option to solve this.

Cheers,
Rust

[edmorley]

Hi Alek, sorry for the delayed reply. Was great meeting you at PyCon too!

Ah yes, so we do want to support development dependencies soon - upstream recently approved an RFC for "Execution Environments" (which will allow us to install dev deps if the environment is "test" etc):

• RFC: Execution Environments buildpacks/rfcs#274
• https://github.com/buildpacks/rfcs/blob/main/text/0134-execution-environments.md

...and now we're waiting on it to be implemented:

• [RFC #0134] - Execution Environments buildpacks/rfcs#327
• Proof of Concept for Execution Environment RFC buildpacks/pack#2324
• buildpacks/lifecycle@main...jab/exec-env-hack

At which point we can add support to Heroku's Fir internal build system, libcnb.rs, and then individual CNBs :-)

[edmorley]

@amureki Hi! We also spoke over in heroku/heroku-buildpack-python#1816 - which makes me think you might be using the classic Python buildpack (from that repo), rather than the Python CNB (this repo)? There's some more detail about the group behaviour in the PR description for heroku/heroku-buildpack-python#1791 if that helps give some context? In general, we'd discourage having things that you install for prod but not locally or in CI - it means you aren't testing what you deploy.

[amureki]

@amureki Hi! We also spoke over in heroku/heroku-buildpack-python#1816 - which makes me think you might be using the classic Python buildpack (from that repo), rather than the Python CNB (this repo)? There's some more detail about the group behaviour in the PR description for heroku/heroku-buildpack-python#1791 if that helps give some context? In general, we'd discourage having things that you install for prod but not locally or in CI - it means you aren't testing what you deploy.

Yes, thanks!

I only realized that this is a different buildpack after posting the comment. Nevertheless, thanks for clarifying everything, nothing hanging from my side. :)

[crestonbunch]

I had success writing a simple buildpack that runs before heroku/python that conditionally copies dev-requirements.txt over requirements.txt. This makes several assumptions, e.g. you don't need to install both requirements files. But it works for our use cases. Sharing in case someone else ends up wanting to do something similar.

[merwok]

[dependency-groups]
prod = [
    "brotli",
]

Dependency groups are meant for development, not deployment, so your example of dev/lint are good (assuming dev means developer’s machine, not development environment server), but for this use case* using an extra is more appropriate (so you would run pip install .[prod]).

*I had the same use case note! All environments install from runtime.txt and server environments also install deploy.txt

[edmorley]

Dependency groups are meant for development, not deployment

I don't think that's correct?

Dependency groups aren't for publishing (eg to PyPI), but publishing != deployment. So extras really only need to be used when publishing, but not when deploying.

[merwok]

Re-reading the intro of https://packaging.python.org/en/latest/specifications/dependency-groups/ I think I agree with you!