Why shallow clone is expensive to perform?

[xxyzz]

I find this commit that requires homebrew-core and homebrew-cask taps to be unshallow cloned but I don't why it's expensive.

[jonchang]

See CocoaPods/CocoaPods#4989 (comment) and https://public-inbox.org/git/[email protected]/T/. Every brew update that fetches into a shallow clone has a big performance and bandwidth penalty.

[SMillerDev]

The best way to update a shallow clone of homebrew is to unshallow it and use it that way.

[chrisfinazzo]

Huh.

Although I usually run update - and also fetch the upstream - into my copy of Core and/or Cask a couple times a day, I first encountered this late yesterday.

In that case, would it be unwise to alias brew update as git -C "/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask" fetch --unshallow && brew update or is there another way to get around this?

[SMillerDev]

You just need to unshallow it once.

[duhd1993]

/usr/local/Homebrew/Library/Taps/ is already almost 1GB on my mac. Although disk is not that expensive nowdays (I guess for Mac it's still expensive), this size will only increase with time. Do you have any plan for this? Most of the disk space used is actually not useful for normal users.

[carlocab]

Using an unshallow clone is the price for GitHub not imposing rate-limits on access to Homebrew repositories. I think rate-limits would create a terrible experience for normal users, so I think it absolutely is useful.

If you want to cut down on the storage space that your taps use, you can try git gc --aggressive. (Caveat: please read up on what this does before trying it yourself.)

[ghost]

Theoretically, for a pure linear commit history (without merges), following fetch from a shallow clone shouldn't be expensive.

Could something be improved for git itself?

[SMillerDev]

If you can find a way to optimize this in Git i'm sure GitHub engineers would be happy with it.

[carlocab]

I guess the difficulty is that git has no guarantee that a repo has a pure linear commit history. It also has no way of guaranteeing that a repo with such a history will stay that way.

[yichengchen]

https://blog.cocoapods.org/CocoaPods-1.7.2/ CocoaPods already support http CDN for years and greatly improve the user experience.
I guess it's the right way brew should follow.

[SMillerDev]

That's just over a year since the blog post. But if you have an idea for an implementation you can make a pull request.