-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automated fuzzing on Fuzzit #2835
Conversation
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here with What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
@googlebot signed it |
All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the ℹ️ Googlers: Go here for more info. |
Codecov Report
|
@dominikh Hi, I built on your good work for this PR. The googlebot is asking for consent to have your commits merged. Would you mind looking at it? |
I see there's a strange error in the build. A type error even though the types are identical.
Not sure what that's about, it works locally. Will be looking into this. |
@googlebot I consent. |
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
Thanks @dominikh. |
1ef5450
to
4205463
Compare
Applied the Go modules approach from dvyukov/go-fuzz#195 (comment) to get this working. Thanks @mvdan for the pattern. This is ready for review. |
This looks good but I'm hesitant to add 8 more minutes to our feedback loop. Can you please change it to a job instead of a stage so at least it can run in parallel with the other jobs? |
Will do, thanks for looking at it. |
Enables automated fuzzing on Fuzzit. Fuzz regression tests run every push and PR. Full fuzzing runs every push. Uses genetic fuzzer go-fuzz. Fuzz targets are: * parse-config - Configuration file parsing. * parse-reference - Docker image reference parsing. * control-api-tcp - Binary requests to control API server. * control-api-http - HTTP requests to control API server.
Rebased this and converted the fuzz stages to jobs. I'll check back to see how the build goes. If you'd like to sign in to Fuzzit to make an account I can add you to the org. Then final setup is like this:
|
@balopat do you think we should merge this (when it's rebased)? |
I'm not convinced. One thing is the slowness of the test, the other is that most of the test code seems to be HTTP specific instead of Skaffold's test code. It feels like this kind of code should live in a fuzz-http library or similar...I'm not sure I want to maintain this. |
@balopat I agree. I'm sorry @bookmoons but I think maintaining that would cost us more that it brings. |
Sounds good. Apologies for dragging this out this long. Closing. |
Integrates with Fuzzit for automated fuzzing. Proposed in #2670.
Under this setup Travis runs fuzzing on every build. It runs a fuzz regression test locally of past crashes and submits to Fuzzit for a full fuzzing run. The build ends as soon as the fuzzing run is in the queue. Fuzzit emails when it finds crashes.
This tries to fuzz the suggested targets plus some other identified parsers. Targets are:
control-api-http
- HTTP requests to the control API server.control-api-tcp
- Binary requests to the control API server.parse-config
- Fuzz the config parser.parse-reference
- Fuzz the Docker image reference parser.There's a JDWP specification parser in
pkg/skaffold/debug
that seems suitable for fuzzing. It's tough to get at because it's a private function. If you think it's worth fuzzing exposing it would make it possible.A
skaffold
org is preconfigured on Fuzzit. If whoever will be administering signs in to create an account, I can get you added to the org.Thank you for considering.