Support Hashed repository storage type

[seriv]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet:
• Ruby:
• Distribution:
• Module version:

How to reproduce (e.g Puppet code you use)

Try creating a custom hook when the repository storage type is set to "hashed" which is recommended by default for Gitlab now.

What are you seeing

Module is trying to create a "custom_hooks" directory under /var/opt/gitlab/git-data/repositories/<namespace>/<repository_name> directory

What behavior did you expect instead

Since for this test project the "project ID" happens to be 3778, I'd like the directory "custom_hooks" to be created under

/var/opt/gitlab/git-data/repositories/@hashed/54/50/5450f0a658b52a4b3ecc517d84581bdffb2df2d5245de45bb9c8e78c4418f675.git/

because echo -n 3778 | sha256sum yilds 5450f0a658b52a4b3ecc517d84581bdffb2df2d5245de45bb9c8e78c4418f675

Output log

Error: Cannot create /var/opt/gitlab/git-data/repositories/seriv/csweb.git/custom_hooks; parent directory /var/opt/gitlab/git-data/repositories/seriv/csweb.git does not exist

Any additional information you'd like to impart

Per Gitlab "Storage Types" the way module behaves rely on using "legacy" storage types, while the default is not supported. To fix this issue module should query Gitlab getting Project ID and then create sha256sum of it for using as a hashed path instead of legacy <namespace>/<repo_name>

[seriv]

We are using the following patch as a workaround. I am not proposing this patch as a merge request, because, I guess, legacy repositories storage should also be supported until deprecation or at least until end of support:

diff --git a/manifests/custom_hook.pp b/manifests/custom_hook.pp
index e5f1be0..492d2a4 100644
--- a/manifests/custom_hook.pp
+++ b/manifests/custom_hook.pp
@@ -81,7 +81,32 @@ define gitlab::custom_hook(
     fail("gitlab::custom_hook[${name}]: Must specify either content or source, but not both")
   }
 
-  $hook_path = "${_repos_path}/${namespace}/${project}.git/custom_hooks"
+  $gitlab_private_token = hiera('gitlab::private_token')
+
+  $gitlab_external_url = $gitlab::external_url
+
+  $gitlab_getid = @(END)
+    <%=
+    require "net/http"
+    require "net/https"
+    require "json"
+    url = @gitlab_external_url + "/api/v4/projects/" + @namespace + "%2F" +@project
+    uri = URI.parse(url)
+    http = Net::HTTP.new(uri.host, uri.port)
+    http.use_ssl = true
+    http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+    headers = { "PRIVATE-TOKEN" => @gitlab_private_token }
+    request = Net::HTTP::Get.new(uri.request_uri, headers)
+    response = http.request(request)
+    body = response.body
+    json = JSON.parse(body)
+    json["id"]
+    -%>
+    | END
+
+  $hash = sha256(inline_template($gitlab_getid))
+  $prefixes = $hash.match(/(..)(..)/)
+  $hook_path = "${_repos_path}/@hashed/${prefixes[1]}/${prefixes[2]}/${hash}.git/custom_hooks"
 
   File {
     owner => $gitlab::service_user,

[LongLiveCHIEF]

between HA and storage pools (which come online when you turn on hashed storage), it could make our hooks feature nearly impossible to continue to support.

[yakatz]

I looked at using the gitlab-rails command runner in a custom function to get this (since the user has write access to the GitLab server data, can it definitely run the console?).

gitlab-rails runner "puts Project.find_by_full_path('@namespace/@project').repository.gitaly_repository.relative_path"

The only issue I see with this is each time the command is run takes 40-60 seconds on my machine which could substantially increase the puppet run time.

Thoughts on that approach?

[yakatz]

Gitlab has changed the storage path mechanism again - hashed paths are only for repos created before Gitlab 15.3. There is now a command line to create hooks - need to investigate that further: https://docs.gitlab.com/ee/administration/server_hooks.html#server-hooks-on-a-gitaly-cluster