Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhancement]: convert model to 16-bit #4069

Open
1 task done
keturn opened this issue Jul 29, 2023 · 0 comments
Open
1 task done

[enhancement]: convert model to 16-bit #4069

keturn opened this issue Jul 29, 2023 · 0 comments
Labels
enhancement New feature or request model manager

Comments

@keturn
Copy link
Contributor

keturn commented Jul 29, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Contact Details

No response

What should this feature add?

Sometimes models are only distributed in 32-bit format, but we're only interested in running them at 16-bit.

To save disk space and loading time, model manager should have a "convert to 16-bit" button that writes out float16 safetensors and deletes the old files.

Alternatives

No response

Aditional Content

In order to know whether models need converting to 16-bit or not, we first need to be able to see their data type:
@keturn keturn added enhancement New feature or request model manager labels Jul 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request model manager
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@keturn