Integrate an expert knowledge corpus containing specialized private or proprietary academic papers to inform and inspire the program's evolutionary process.

[yiyiyixia]

Most of the data that LLMs are trained on comes from public sources.However, there's a vast amount of valuable research locked away in paid or proprietary databases that's much harder to access.

I'm wondering if it's possible to create a dedicated knowledge base for this kind of high-value papers and data, and use it to actively guide the model's learning process. Essentially, how can we make sure this specialized knowledge has a greater impact on the model's development?

If we wanted to pursue this, what kind of steps or work would we need to plan for?

[codelion]

I think this can be done as part of an outer loop that incorporates domain-specific knowledge. An example of this is the work done by folks on GeoEvolve - https://algorithmicsuperintelligence.ai/blog/openevolve-geoevolve/index.html