On structured outputs

[dylanjcastillo]

Hey @souzatharsis,

I've read through the structured outputs chapter, and thought it was great. Very comprehensive!

I just had a couple of suggestions:

1. I think you do not explicitly mention Function/Tool calling, which might be the most popular method for generating structured outputs with proprietary models. It's actually the default approach of LangChain and instructor. I thought it might be good to mention it at least briefly.
2. Have you considered including instructor instead of LangChain? If I'm not mistaken they were the first library to push for this approach for proprietary models Pydantic Is All You Need.
3. In case it's useful, I've extended .txt's analysis to GPT-4o-mini and Gemini-Flash-1.5 and found that in both of those cases, it does seem to make a difference.
4. It might be interesting to mention somewhere that you should be careful not to think that a used to generate data from an LLM is the same as the one you use for your APIs. One thing that I've frequently noticed that people get wrong is that they don't order things correctly and often break CoT processes. There are even some examples of that in OpenAI's cookbook: Change order of justification key in eval schema openai/openai-cookbook#1619
5. There are some reports of latency related to structured outputs as well: https://python.useinstructor.com/blog/2024/08/20/should-i-be-using-structured-outputs/#unpredictable-latency-spikes

Anyhow, I really enjoyed the article's content. This is an area that I'm very interested in nowadays, so I will keep an eye on it.

[souzatharsis]

That's exactly the kind of feedback I am looking for, thank you so much for taking the time @dylanjcastillo .

1. Isn't Function calling a sub-type of fine-tuning? I've now added a side note mentioning function calling in addition to JSON mode as common fine-tuning use cases.

2. Added citation. I think LangChain needs to be covered just because of the sheer usage even though I agree it is not a tool for structured generation per se. Under that category I covered outlines.

3. Thanks for sharing. I actually had read them but decided not to include last minute since it would be a kind of rebuttal of the rebuttal. But since the very author is reaching out and I see this is good evidence this has been added. Thanks for reminding me.

4. I actually did mention this but in the Evals chapter! I have now added to structured generation which I agree makes more sense.

5. Added. Thanks for sharing. That's good evidence.

Finally, added your name to acknowledgments.

All in the pdf version with an updated version to be released soon.

[dylanjcastillo]

Thank you @souzatharsis. That's very kind.

Re function-calling a sub-type of fine-tuning, I think that's the case: https://community.openai.com/t/json-mode-vs-function-calling/476994/5. So I also think mentioning it as a side note sounds good!