v0.1.7

This is a patchfix to:

• update the python version used for serving on vLLM.
• handle local environment variables

v0.1.6

This is patchfix against the skypilot update. Skypilot resource API changed so this version requires skypilot>=0.1.6

v0.1.5

This release is a patchfix namely

• Pin fastchat version. Old builds were breaking since fastchat and transformers were not pinned
• allow selection of regions/zones for finetuning
• Allow finetuning for more than just LLama2 (e.g. Mistral)

v0.1.4 fix file mount cli args

This is a hot fix so users can specify checkpoint paths to object stores rather than it being hardcoded to llm-atc since s3 buckets have to be globally unique.

v0.1.3

This patch includes some bugfixes as well enabling passing huggingface tokens to access gated/private models for serving and training. This update also enables tensor parallelism on all gpus of a given model to enable serving of larger models like llama-70b on a multigpu instance.

I promise to write a detailed changelog coming up in v0.1.4!

v0.1.2

Features

• Vicuna Finetuning (Llama base)
• Serving from HuggingFace or llm-atc finetuned models

Patch notes:

• fixed bug in serving with vLLM==0.1.3 by downgrading to ray==2.5.1