.Net: Support for non-OpenAI models in StandardMagenticManager

[Celio Cidral (ccidral)]

If I understand correctly, it's only possible to use StandardMagenticManager with models that support the response_format parameter, which narrows down the options to OpenAI models only. I understand that this decision was facilitated by the "native" support that OpenAI offers through the response_format parameter, i.e. it would require some gymnastics to get it to work with models that don't have similar support through a specific parameter like that.

That said, is structured output support with non-OpenAI models something that's planned for the future? Or, more importantly, is there any viable workaround?

[John CSA (jluocsa)]

You're partially right — but the constraint is narrower than "OpenAI models only", and there's a clean workaround in the public API.

What StandardMagenticManager actually requires

It calls SupportsResponseFormat in its constructor:

public static bool SupportsResponseFormat(this PromptExecutionSettings settings)
{
    Type settingsType = settings.GetType();
    PropertyInfo? property = settingsType.GetProperty("ResponseFormat");
    return property != null && property.PropertyType == typeof(object);
}

It's a reflection check for a ResponseFormat property of type object on whatever concrete PromptExecutionSettings subclass you pass in. The manager then does executionSettings.SetResponseFormat<MagenticProgressLedger>(), which writes the CLR Type typeof(MagenticProgressLedger) into that property. What the connector does with that Type (build an OpenAI structured-output schema, translate it to Ollama format: "json", set Bedrock responseFormat, etc.) is up to the connector.

So the hard requirement is: the connector's PromptExecutionSettings subclass must expose public object? ResponseFormat { get; set; }. It does not have to be an OpenAI deployment.

Workarounds

1. Use a non-OpenAI connector whose settings already have ResponseFormat. This is the path of least resistance — verify the connector you want, and if <Connector>PromptExecutionSettings.ResponseFormat exists with type object, you can pass an instance into StandardMagenticManager directly. If SetResponseFormat<T> is a no-op or misinterprets typeof(T) for that connector, follow #2.

2. Adapt your connector's settings + service. Subclass your connector's PromptExecutionSettings and add an object? ResponseFormat property; in your model client adapter (or a small IChatCompletionService decorator), translate typeof(MagenticProgressLedger) into whatever JSON-mode flag your model API understands and re-emit the JSON schema in the prompt. The manager will then accept it and call _service.GetChatMessageContentAsync as usual.

3. Write a custom MagenticManager. The base class MagenticManager is abstract, not sealed — only StandardMagenticManager is sealed. It has exactly four abstract methods you must implement:

public abstract ValueTask<IList<ChatMessageContent>> PlanAsync(MagenticManagerContext context, CancellationToken ct);
public abstract ValueTask<IList<ChatMessageContent>> ReplanAsync(MagenticManagerContext context, CancellationToken ct);
public abstract ValueTask<MagenticProgressLedger>    EvaluateTaskProgressAsync(MagenticManagerContext context, CancellationToken ct = default);
public abstract ValueTask<ChatMessageContent>        PrepareFinalAnswerAsync(MagenticManagerContext context, CancellationToken ct = default);

Only EvaluateTaskProgressAsync actually needs JSON — and you have full control over how to coerce it. The two reliable patterns for non-structured-output models are:

• Function/tool calling — define a single tool whose parameters match MagenticProgressLedger and force the model to invoke it. Auto-invoke off, parse from Items.OfType<FunctionCallContent>().
• Prompt-then-extract — ask the model for JSON between explicit delimiters (<ledger>...</ledger>) and regex-extract; on a parse error, retry with an "your previous response was not valid JSON, here is the schema..." correction step. Robust enough in practice for Magentic's progress ledger because the ledger is small and bounded.

You can copy the prompts and structure straight out of StandardMagenticManager and swap only the response-format step — most of the logic (facts, plan, ledger templating) is reusable.

Roadmap

Whether first-class structured-output support for additional connectors is on the roadmap is a maintainer question — but the framework is already designed so that you can drop in a different MagenticManager without forking SK. For most non-OpenAI scenarios option 3 is what I'd reach for.