Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache git templates locally #450

Open
yajo opened this issue Sep 24, 2021 · 6 comments · Fixed by #472
Open

Cache git templates locally #450

yajo opened this issue Sep 24, 2021 · 6 comments · Fixed by #472
Labels
enhancement good first issue Easy things for newbies

Comments

@yajo
Copy link
Member

yajo commented Sep 24, 2021

the fetch that copier does for this case is too slow.

Originally posted by @bburgin in #449 (comment)

And it's true. Copier should cache downloaded templates, so next fetches are faster.

@yajo yajo added the bug label Sep 24, 2021
@yajo yajo added this to the v6.0.0 milestone Sep 24, 2021
@pawamoy
Copy link
Contributor

pawamoy commented Sep 25, 2021

Or there might be possible optimizations, like partial clones, or single branch clones:

# clone only the remote primary HEAD (default: origin/master)
git clone <url> --single-branch

# as in:
git clone <url> --branch <branch> --single-branch [<folder>]

@pawamoy
Copy link
Contributor

pawamoy commented Sep 25, 2021

If we go with caching: I guess we'll just need to fetch/pull each time we generate/update a project again, and if it fails clone the template repo again, do I understand correctly?

yajo added a commit that referenced this issue Nov 2, 2021
When git 2.27 or newer is installed, we can add `--filter=blob:none` to avoid getting useless information from the git server. This makes clone much faster if your template has a big history.

Close #450 (it does not exactly fix it, but maybe makes it irrelevant).
yajo added a commit that referenced this issue Nov 2, 2021
When git 2.27 or newer is installed, we can add `--filter=blob:none` to avoid getting useless information from the git server. This makes clone much faster if your template has a big history.

Close #450 (it does not exactly fix it, but maybe makes it irrelevant).
yajo added a commit that referenced this issue Nov 2, 2021
When git 2.27 or newer is installed, we can add `--filter=blob:none` to avoid getting useless information from the git server. This makes clone much faster if your template has a big history.

Close #450 (it does not exactly fix it, but maybe makes it irrelevant).
@yajo yajo closed this as completed in #472 Nov 2, 2021
yajo added a commit that referenced this issue Nov 2, 2021
When git 2.27 or newer is installed, we can add `--filter=blob:none` to avoid getting useless information from the git server. This makes clone much faster if your template has a big history.

Close #450 (it does not exactly fix it, but maybe makes it irrelevant).
@ypid-work
Copy link

For me copier takes 5.5 seconds when updating from a template stored in a remote git repo vs. 0.7 seconds when the template is checked out locally. I still think caching the template repo makes a lot of sense. The cache could be created in ~/.cache/copier. Can you reopen because this issue was not solved in accordance with the original description. The cache could also be used for quick local development/template fixes before pushing them.

@yajo yajo reopened this Feb 3, 2023
@yajo
Copy link
Member Author

yajo commented Feb 3, 2023

Maybe we can just use https://pypi.org/project/git-autoshare/ behind the scenes automatically.

@yajo yajo modified the milestones: v6.0.0, Community contribution Feb 3, 2023
@yajo yajo added enhancement good first issue Easy things for newbies and removed bug labels Feb 3, 2023
@sisp
Copy link
Member

sisp commented Apr 7, 2024

Two arguments against using git-autoshare:

  1. It is licensed under GPLv3, which would poison our MIT license.
  2. It doesn't seem actively maintained.

@yajo
Copy link
Member Author

yajo commented Apr 7, 2024

Fair points. Then we can just cache a local git mirror and use temporary worktrees to speed things up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement good first issue Easy things for newbies
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants