Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use rclone instead of custom sync script to publish documentation to AWS S3 bucket #15155

Merged
merged 3 commits into from
Feb 14, 2025

Conversation

milzi234
Copy link
Contributor

Short description

Publishing documentation to AWS S3 has been slow, likely due to the custom publish.js script overwriting all files. This patch uses rclone instead, which uses digests to figure out which files to actually upload. This also means we don't have to use the javascript script anymore and don't need to install all the node dependencies for the aws sdk, which will likely also save time.

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)

Copy link
Member

@rgacogne rgacogne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's brilliant, nice work!

@coveralls
Copy link

coveralls commented Feb 14, 2025

Pull Request Test Coverage Report for Build 13326611785

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 52 unchanged lines in 9 files lost coverage.
  • Overall coverage increased (+0.2%) to 64.583%

Files with Coverage Reduction New Missed Lines %
pdns/backends/gsql/gsqlbackend.hh 2 97.71%
pdns/dnsdistdist/dnsdist-async.cc 2 81.51%
pdns/rcpgenerator.cc 2 90.55%
pdns/recursordist/test-syncres_cc2.cc 3 88.85%
pdns/remote_logger.cc 3 57.4%
pdns/dnsdistdist/dnsdist.cc 4 68.79%
pdns/dnsdistdist/dnsdist-carbon.cc 6 62.23%
ext/json11/json11.cpp 13 62.72%
modules/godbcbackend/sodbc.cc 17 70.8%
Totals Coverage Status
Change from base Build 13325621226: 0.2%
Covered Lines: 127829
Relevant Lines: 166866

💛 - Coveralls

@miodvallat
Copy link
Contributor

Stupid question, is it important to pass --progress to rclone? Shouldn't we aim for the tersest possible output?

@rgacogne
Copy link
Member

DNSdist CI failure is unrelated, see #15156

@rgacogne rgacogne added this to the common-soon milestone Feb 14, 2025
fi
# Cleanup credentials
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the VM gets destroyed right after this anyway, but it can't hurt

@Habbie Habbie merged commit 5270b6d into PowerDNS:master Feb 14, 2025
83 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants