Skip to content

Optimize GDB prompt in CrashAnalyzer #1141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

maoyixie
Copy link
Collaborator

@maoyixie maoyixie commented Jul 3, 2025

This pr is used to optimize the gdb prompt. The language is more refined and unified, and the structure is clearer.

In particular, In GDB, do not use shell-style redirection like:
run < {AFTIFACT_PATH}
This won't work because GDB is not a shell. I have seen too many these cases, LLM model like GPT-4o also makes this mistake.

Also, avoid using:
run -- {AFTIFACT_PATH}
unless the fuzz driver explicitly expects that pattern. Relevant case: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-01-1133-pamusuo-comparison/sample/output-igraph-igraph_sparsemat_arpack_rssolve/03.html#:~:text=54%3A58%20%5BTrial%20ID%3A%2003%5D%20INFO%20%5Bbase_agent.-,chat_llm,-%3A96%5D%3A

Furthermore, while run {AFTIFACT_PATH} seems correct, it may execute the testcase multiple times, depending on how libFuzzer is configured.

✅ Instead, always use:
run -runs=1 {AFTIFACT_PATH}
to ensure the fuzz driver runs exactly once with the provided testcase — ideal for reproducible crash debugging.

@maoyixie maoyixie requested a review from DonggeLiu July 3, 2025 07:14
@maoyixie
Copy link
Collaborator Author

maoyixie commented Jul 3, 2025

/gcbrun exp -n my -m vertex_ai_gemini-2-5-pro-chat -ag

@maoyixie
Copy link
Collaborator Author

maoyixie commented Jul 3, 2025

/gcbrun exp -n my1 -m vertex_ai_gemini-2-5-pro-chat -ag

@maoyixie
Copy link
Collaborator Author

maoyixie commented Jul 3, 2025

There seems to be no successful build example in this experiment. Can we change the test set to conduct the experiment?
@DonggeLiu , exp report link: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-03-1141-my1-comparison/

@DonggeLiu
Copy link
Collaborator

https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-03-1141-my1-comparison/

There seems to be many now : )
It is because the experiment takes longer?

@maoyixie
Copy link
Collaborator Author

maoyixie commented Jul 4, 2025

@maoyixie
Copy link
Collaborator Author

maoyixie commented Jul 4, 2025

According to this case: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-03-1141-my1-comparison/sample/output-libfuse-af_gb_alloc_data/02.html

It can found the artifact. And the new prompt is ok. I think this pr is ok and it can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants