Clarify that poetry update only updates main dependencies

[hugobuddel]

Add a note that --with is required to let poetry update also update optional dependency groups.

This behavior of poetry update confused me, so I thought let's try to save other people the trouble by updating the documentation.

I do want to note that I find the behavior strange. If I do poetry install --with=dev followed by poetry update, then I expect all installed dependencies to be updated, also the dev ones.

Pull Request Check List

Resolves: #issue-number-here

• Added tests for changed code.
• Updated documentation for changed code.

[dimbleby]

update is a two-phase process: first locking, then installing. I think perhaps this is what the docs could be clearer about.

eg your first change is definitely wrong, the lockfile will be updated even for dependencies in optional groups

I think what has surprised you is that update has the same semantics as install during the install phase

[hugobuddel]

You are right, the lock file is indeed updated, but the optional dependencies that are installed are not updated; clarified that.

The behavior is rather confusing (to me at least), so it is good to be as explicit as possible.

[dimbleby]

I think this is pedantically not correct - non-optional dependency groups are also updated, right?

[hugobuddel]

That is probably right than non-optional dependency groups are also updated, although I have not tested this; updated, thanks.

[hugobuddel]

I was wondering whether the update behavior can leave the user with a broken state, and that seems indeed possible. E.g. see this example project https://github.com/hugobuddel/poetrytest

Maybe we should add a warning that the user can be left with an inconsistent environment if poetry update is called with different dependency groups specified than when the project was first installed?

[abn]

@hugobuddel unfortunately there not much Poetry can do in your example.

$ poetry show scipy
 name         : scipy                                                     
 version      : 1.15.1                                                    
 description  : Fundamental algorithms for scientific computing in Python 

dependencies
 - numpy >=1.23.5,<2.5

You will notice that the installed version of scipy will allow numpy >=1.23.5,<2.5. Which means your update numpy (1.26.0 -> 2.1.3) was "safe" from all the info Poetry has at the time.

[abn]

I do not think this is a 100% accurate. Dependencies specified in optional groups will also update iff a dependency in the activated group(s) has a dependency on any of them. Otherwise, it wont.

[hugobuddel]

I don't know how to formulate it better.

All I know is that I was very confused that I had to specify --with or poetry would not update several of my already installed packages (as in, it would not install newer versions). I found that behavior very counterintuitive, so I thought maybe others are confused too, so I tried to be a good citizen and improve the documentation.

Maybe

You need to specify the same optional dependency groups that you specified when you ran poetry install, otherwise poetry might not install newer versions of your already installed packages. Specifying no or different groups might leave your system in a broken state with incompatible packages.

That conveys more clearly what I want to warn about, and also adds a few 'might's in there in case the installed package whose group is not selected but that is a dependency of an active group is updated anyway.

[hugobuddel]

@hugobuddel unfortunately there not much Poetry can do in your example.

$ poetry show scipy
 name         : scipy                                                     
 version      : 1.15.1                                                    
 description  : Fundamental algorithms for scientific computing in Python 

The point is that poetry does not actually install a newer version of scipy, 1.10.1 is still installed. So the installation is broken after doing poetry update.

I don't think it is ever desirable to leave the system in a broken state. So it would be good to add a warning to the documentation that this is a possibility lest the user forgets which --with clauses were used during the initial installation of a project.

What poetry could do, is detect that it had previously installed scipy (through the optional test group), and that a newer version of scipy should now be installed. Or at least warn the user.