[Connectivity - UI-TARS] aiQuery & aiAssert 返回 Error: 422 Failed to deserialize the JSON body into the target type

[robin-miro]

If the error persists, tell us these information

• Where are you using Midscene.js: playwright

• The version of Midscene.js or Extension: 0.12.0

• The error message:

当使用aiQuery获取时，

  const avaiableCountries = await aiQuery(
    'string[], find all avaiable countires in the popup'
  );

同样的问题也在aiAssert上可以复现, 好像我现在只能用.ai()

你可以试这个网站 https://www.vinted.com， 他会弹一个国家选择popup

• The model name and endpoint (if could be public）

MIDSCENE_MODEL_NAME="UI-TARS-7B-SFT"
MIDSCENE_USE_VLM_UI_TARS=1

[robin-miro]

got issue from Unit test

passed when using gpt-4o, but failed using UI Tars

OPENAI_API_KEY="hf_<>"
OPENAI_BASE_URL="https://<>.huggingface.cloud/v1"

MIDSCENE_MODEL_NAME="UI-TARS-7B-SFT"
MIDSCENE_USE_VLM_UI_TARS=1
MIDSCENE_LANGSMITH_DEBUG=1

LANGSMITH_API_KEY=<>
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_TRACING_V2=true
LANGSMITH_PROJECT=<>