Demo LF programs with latency sensitive LLM inferences

[Deeksha-20-99]

This demo showcases time-sensitive Lingua Franca demos that include two LLM agents and measure both logical and physical time per user query for LLM-based inference. In this demo, we also present the federated execution of time-sensitive LLM-based inference.

• The Judge reactor takes in user input as a physical action, and this acts as a quiz question to the two LLM agents.
• The LlmA and LlmB reactors receive the query from the Judge and each invokes its respective LLM model to generate an answer.
• Both agents run concurrently. Whichever LLM responds first is declared the winner, and its answer is reported along with logical and physical execution times.

For this demo, the LLM models we have utilized are LLAMA chat models having a billion parameters, which are then quantized. These models are used for inference with the training knowledge, and no context is given.

We also explore federated execution, where each LLM agent could run as a separate federate, coordinated by the RTI. This follows the same principle of the game, except that we try to offload models on different devices (to support the models with higher parameters to be run on devices with greater GPU support).
This demo explores time-sensitive events and distributed execution of LLM-based systems using Lingua Franca.

[hokeun]

We need Python version information here (e.g., minimum version requirement).