Adds GPTQ Quantization Documentation

[JyotinderSingh]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @JyotinderSingh, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new, detailed guide on GPTQ quantization for Keras and KerasHub models. The documentation aims to equip users with the knowledge and tools to apply weight-only quantization, significantly reducing model size and memory requirements while maintaining high model quality. It covers the theoretical background, practical implementation steps, and performance implications of using GPTQ.

Highlights

• New GPTQ Quantization Guide: A comprehensive guide has been added explaining how to perform GPTQ (Generative Pre-Training Quantization) on Keras and KerasHub models.
• Weight-Only Quantization Explained: The guide details GPTQ as a post-training, weight-only quantization method, emphasizing its benefits in reducing model size and memory footprint.
• Practical Implementation Steps: It provides practical steps for configuring and running GPTQ using keras.quantizers.GPTQConfig, including details on calibration datasets, tokenizers, and key parameters like weight_bits and group_size.
• Performance Benchmarking Included: The documentation includes benchmark results demonstrating significant reductions in disk storage and VRAM usage (around 50%) with only marginal increases in perplexity, showcasing the effectiveness of GPTQ.
• Model Export and Practical Tips: Instructions for exporting quantized models and a section with practical tips for effective GPTQ application are also included.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds comprehensive documentation for GPTQ quantization in Keras, including a Python guide, a Jupyter notebook, and a Markdown file. The additions are well-structured and informative. I've provided a few minor suggestions to improve clarity and consistency across the different documentation formats. Overall, this is a great contribution.

[hertschuh]

LGTM.

A couple of comments:

• GPTQConfig is not documented in the API, we should add it
• quantize is not documented in https://keras.io/api/models/model/ , we should add it

Also, do you want to mention a couple more of the config options for GPTQ in this tutorial?

[JyotinderSingh]

LGTM.

A couple of comments:

• GPTQConfig is not documented in the API, we should add it
• quantize is not documented in https://keras.io/api/models/model/ , we should add it

Also, do you want to mention a couple more of the config options for GPTQ in this tutorial?

I'll create another PR to add these symbols into the API docs as well.

I'm not sure if discussing other config keys will add value to the guide, since that would probably something that is more suited for the API reference.