Skip to content

Fix prediction output readability and simplify inference installation#18

Open
khazic wants to merge 13 commits intogoogle-research:mainfrom
khazic:fix/predict-install-and-output
Open

Fix prediction output readability and simplify inference installation#18
khazic wants to merge 13 commits intogoogle-research:mainfrom
khazic:fix/predict-install-and-output

Conversation

@khazic
Copy link
Copy Markdown

@khazic khazic commented Mar 25, 2026

Summary

This PR fixes a few practical issues around inference and installation:

  • keep requirements.txt inference-only and document mt-metrics-eval as an extra dependency for meta-evaluation
  • make prediction output JSONL human-readable for non-ASCII text
  • add a padding collator and safer per-device batch size handling in metricx23.predict
  • handle output_file paths without a parent directory in both predict scripts

Why

Directly installing requirements.txt currently pulls in a VCS dependency that is only needed for meta-evaluation, which makes inference setup more fragile than necessary.

Also, both predict scripts currently write JSON using the default ensure_ascii=True, so multilingual content is emitted as Unicode escape sequences. In practice this makes output files harder to inspect.

Finally, metricx23.predict was missing a padding collator, which can break batched inference on variable-length inputs, and its per-device batch size could become zero when the global batch size is smaller than the GPU count.

Validation

  • python3 -m py_compile metricx23/predict.py metricx24/predict.py

@google-cla
Copy link
Copy Markdown

google-cla bot commented Mar 25, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@khazic khazic force-pushed the fix/predict-install-and-output branch from f311c93 to 48914c2 Compare March 25, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant