Turnoff the interim researcher logs

[muskaan1208]

Hi @assafelovic !
I have an interesting question,
i am looking out for help, in filtering out or sanitising the researcher logs are that are being generated in the middle, its a of logs that are getting stored unnecessarily consuming a lot of space and leaking a lot of senisitive information

we are using AZURE_API and TAVILY for our research.

I see there is no flag/param currently to pass and switch off interim logs that being produced.
Seeking for help, if anyone can suggest or can we make a contribution to the open source as this use case

[itxashancode]

I ran into this same thing last year when we were using Tavily for a big research run - those interim logs balloon fast and they definitely leak PII if you're not careful. The logs are coming from the researcher.py module where it dumps raw API responses before filtering. There's no config flag yet, but you can patch it locally.

In gpt_researcher/researcher.py, find the log_response() method (around line 180). Wrap the body in:

if not self.config.get("disable_interim_logs", False):
 # existing logging logic here

Then in config.py, add:

self.disable_interim_logs = False