Skip to content

token id查词表性能优化 #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wilkingwang wants to merge 1 commit into
base: step-audio2-mini
Choose a base branch
from
Open

token id查词表性能优化 #5

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions vllm/model_executor/models/mm_step_audio.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
from .utils import (flatten_bn, init_vllm_registered_model, maybe_prefix,
merge_multimodal_embeddings)

AUDIO_START_TOKEN_ID = 151688
AUDIO_END_TOKEN_ID = 151689
AUDIO_PATCH_TOKEN_ID = 151690


Expand Down Expand Up @@ -107,10 +109,7 @@ def _get_audio_repl(
) -> tuple[str, list[int]]:
num_audio_tokens = self.get_num_audio_tokens(audio_feat_len)
text = "<audio_start>" + "<audio_patch>" * num_audio_tokens + "<audio_end>" # noqa: E501
token_ids = [self.tokenizer.convert_tokens_to_ids("<audio_start>")
] + [self.audio_token_id] * num_audio_tokens + [
self.tokenizer.convert_tokens_to_ids("<audio_end>")
]
token_ids = [AUDIO_START_TOKEN_ID] + [AUDIO_PATCH_TOKEN_ID] * num_audio_tokens + [AUDIO_END_TOKEN_ID]
return text, token_ids

def replace_placeholder(self, text: str, placeholder: str,
Expand Down Expand Up @@ -333,7 +332,7 @@ def _get_prompt_updates(
out_mm_kwargs: MultiModalKwargs,
) -> Sequence[PromptUpdate]:
processor = self.info.get_hf_processor(**hf_processor_mm_kwargs)
audio_token_id = processor.audio_token_id
audio_token_id = AUDIO_PATCH_TOKEN_ID

out_mm_data = out_mm_kwargs.get_data()

Expand Down