Skip to content

Add AWQ Quantization for Blip-2 #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Vincent-La wants to merge 49 commits into
base: main
Choose a base branch
from
Open

Add AWQ Quantization for Blip-2 #5

Vincent-La wants to merge 49 commits into from

Conversation

@Vincent-La
Copy link
Collaborator

@Vincent-La Vincent-La commented Sep 26, 2024

  • AWQ implementation for Blip2ForConditionalGeneration (caption generation task)
  • See awq_demo.ipynb
  • TODO: compute model size and add support for Blip2ForImageTextRetrieval (retrieval task)

FoundedNahte and others added 29 commits February 23, 2025 17:01
@FoundedNahte
Add VQAv2 pipeline
42e5c25
@FoundedNahte
Fix distributed inference bugs
381166f
@FoundedNahte
Code cleanup
a9b7476
@FoundedNahte
Added slurm script for vqa
a545825
@FoundedNahte
Dataset trimming error handling and more cleanup
344e7e9
@FoundedNahte
Added removal of duplicate answers in VQA results
7b6c361
@FoundedNahte
Added command line arguments to vqav2 eval script
86b9b85
@FoundedNahte
Matched inference parameters to LAVIS repo
620350a
@FoundedNahte
Update VQAv2 slurm script
87ef245
@FoundedNahte
Updated VQAv2 metric code and other pipeline fixes
2027202
@FoundedNahte
Updated VQA demo notebook
35e0409
@FoundedNahte
Fix VQA demo notebook
b96c4b8
@FoundedNahte
Added GQA task
0e030b0
@FoundedNahte
Updated vqa_tools
9eb6870
@Vincent-La
add forward hooks
d7d1c08
@Vincent-La
notebook edits to start organizing things into classes
0dfce5e
@Vincent-La
progress on computing scales
557be67
@Vincent-La
restructure to gather first inputs for each group of layers
cafa9c6
@Vincent-La
group modules within layer to compute scaling for
a76e7d9
@Vincent-La
compute scales
55de279
@Vincent-La
apply scaling to modules and prev_op
96c5f47
@Vincent-La
sample awq captions versus full precision
dce8cad
@Vincent-La
start formalizing awq, split out to appropriate files
c4eb04a
@Vincent-La
support for awq qformer modules
77b4e32
@Vincent-La
file cleanup + commenting
5b1036f
@Vincent-La
start demo notebook
dcfd05a
@gautomdas @Vincent-La
Updated files + gptq code
3f7ffce
@Vincent-La
add forward hooks
f8f57ff
@Vincent-La
tidy up awq_demo and awq config support
eb64390
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Vincent-La @FoundedNahte @gautomdas