Discussion around linux/mac revision handling

[iMichka]

I am opening this as a separate issue from #7028 to give it more visibility and to focus on the specific topic.

We need to discuss the handling of revision bumps.
Some linux formulae have a different revision than the ones on Mac:

• this is due to various fixes that have been added over time and required a revision bump in linuxbrew-core
• or the revision bumps were done when a linux-only dependency was bumped, as the dependency tree is slightly different on linux.

We have two alternatives:

• Have only one revision for both platforms. This means some unnecessary revision bumps for Mac when linux stuff is bumped, and vice-versa. When migrating from linuxbrew-core to homebrew-core, we will need to align all the homebrew-core revisions with the linux ones to provide a smooth transition.
• Have separate revisions using a new DSL. No unnecessary revision bumps will be needed. We might need to adapt our audits and style tooling though.

We have time before this happens but I would like to have a clear strategy before we start bottling for linux in homebrew-core.

[reitermarkus]

Have only one revision for both platforms.

I'd say this is easier to maintain.

some unnecessary revision bumps

I guess this is already the case when fixing something specific to a given macOS version.

[Rylan12]

• Have only one revision for both platforms. This means some unnecessary revision bumps for Mac when linux stuff is bumped, and vice-versa. When migrating from linuxbrew-core to homebrew-core, we will need to align all the homebrew-core revisions with the linux ones to provide a smooth transition.

This is my preference (granted I'm not involved with Linux at all). The goal is to have the two repositories in sync as much as possible, so this is the obvious choice.

Keep in mind that any changes we make to the formula code to add Linux-specific stuff should probably still be run through macOS CI to make sure it's not breaking stuff on macOS. If we're already running CI on these, we may as well sync up the revisions. It does mean that users will experience an unnecessary upgrade but I don't think this is a significant deterrent.

Further, the revision numbers would only diverge until a new release of the formula is made. At that point, both revisions will reset back at zero. I don't think we should be bumping the revision for only one platform. My understanding is that most formula issues that are fixed by revision bumps would affect both macOS and Linux, so it makes sense to have them be synced. What I'm getting at is this: if we add a DSL for platform-specific revisions, I think it will only really be used as a temporary measure between the Linux-macOS merge for that formula and the next release. Sure, some formulae won't get new releases for quite some time (ever?) but I think most will. It feels like unnecessary work to temporarily have the revisions diverge. Keep in mind adding this DSL would require modifications to other tools like bump-revision and bump-formula-pr.

Plus, of course, having two separate numbers makes it more difficult to read and reduces clarity.

A note on implementation: I think there's currently an audit that ensures revision bumps are only by increments of 1. This would need to be ignored or modified to not trigger in these cases. I think this is a much easier change than adding a new DSL.

[carlocab]

We do get complaints about unexpected formula upgrades, so I would actually like to avoid unnecessary upgrades if possible. [*]

Do we have a sense of how often a formula gets a revision bump for Mac/Linux but not for Linux/Mac? If it’s not too often then I’d be fine with single revisions for both platforms. If it happens reasonably often, I may still be ok with it, but I’d want to start thinking about the alternative a lot more carefully too.

[*] Admittedly, this is usually a complaint about quantity and not frequency, so perhaps I am needlessly concerned.

[jonchang]

Do we have a sense of how often a formula gets a revision bump for Mac/Linux but not for Linux/Mac? If it’s not too often then I’d be fine with single revisions for both platforms

Well, any time the macOS version gets a revision bump, Linux gets one too. But we can see the divergence here:

% rg -l 'revision.*OS.mac\?' $(brew --repo homebrew/linux) | wc -l
     137

In practice, this only really happens for formulae that don't get a lot of updates, since a version bump will start from revision 0 and sync up macOS and Linux. The most divergent formulae are plotutils (it has bottles dating back to mavericks!) and freeglut (likely due to the xorg merge, as every runtime dependency change necessitates a revision increase).

[carlocab]

TL;DR: Yea, common revision bumps are a good idea.

Ok, some thoughts/observations:

I don't think there are too many complaints from Linux users regarding upgrade frequency. I might therefore be overstating my concern about upgrading too often. However, this may well be a function of a) the disposition of the average Linux user; and, b) the distribution of OSs running Homebrew.

It appears that the most common source of a revision bump is a change in runtime dependencies. It is certainly possible that these changes apply only to macOS or only to Linux. However, I believe the detection of whether these changes necessitate revision bumps comes from our tests, which have limited coverage. [*] It does not seem unlikely that if one OS requires a revision bump to formula because of this change then the other does as well, even if the tests do not detect it. This suggests that there may be an argument for common revision bumps beyond just keeping formulae, the DSL, and associated tooling simple.

[*] I might be underestimating brew linkage --test here.

[MikeMcQuaid]

• Have only one revision for both platforms. This means some unnecessary revision bumps for Mac when linux stuff is bumped, and vice-versa. When migrating from linuxbrew-core to homebrew-core, we will need to align all the homebrew-core revisions with the linux ones to provide a smooth transition.

I agree we should do this.

We do get complaints about unexpected formula upgrades, so I would actually like to avoid unnecessary upgrades if possible. [*]

I agree with this. We should have some written guidelines about when to revision bump and when to just pull new bottles.

I would expect that when homebrew-core and linuxbrew-core are merged most of the times when we'll justify a revision bump on what platform it'll apply elsewhere. See also: revision bumps for one version of macOS and not others (which is a case that comes up that we've ignored).

[sjackman]

👍 Have only one revision for both platforms.

[iMichka]

What I'm getting at is this: if we add a DSL for platform-specific revisions, I think it will only really be used as a temporary measure between the Linux-macOS merge for that formula and the next release

Not really; the revision divergence will stil be there, even after the merge of both repositories. So the DSL will stay forever and will be used.

upgrade frequency

I have the feeling that we have some divergence in linuxbrew-core because:

• a lot of formulae did not have linux bottles but built fine on some setups but not on others: we bumped the revision with the fix
• We forgot to remove the revision when merging from homebrew-core to linuxbrew-core. This is due to bad tooling or no merge conflict happening. Or lack of concentration when merging, as it is a semi-manual process.
• The xorg migration added a bunch of revision bumps

Tomorrow, we should have less revision bumps due to linux.

Of course, if zlib gets updated, that's 461 formulae with uses_from_macos "zlib" that might need an update if brew --linkage is broken: this is the typical revision bump that is only needed for linux but not for Mac, and where Mac users might complain about.

[MikeMcQuaid]

Of course, if zlib gets updated, that's 461 formulae with uses_from_macos "zlib" that might need an update if brew --linkage is broken: this is the typical revision bump that is only needed for linux but not for Mac, and where Mac users might complain about.

This is a good case I hadn't thought about.

[sjackman]

It's necessary to bump the revision of dependents if the new version breaks ABI compatibility. zlib does not typically break ABI compatibility. I've never seen it break ABI compatibility. boost intentionally breaks ABI compatibility with every single release (though it's not provided by macOS).

[iMichka]

What we could try is to start with the same revisions, and see over a few months if there are too many revision bumps.

[BrewTestBot]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[iMichka]

Let's close this. The conclusion was:

What we could try is to start with the same revisions, and see over a few months if there are too many revision bumps.