Function calling and connections

[anderl80]

Do you have examples how go consume connections in a flow using function calling?

[0mza987]

Are you looking for this example?
Execute flow as a function — Prompt flow documentation

[anderl80]

No I refer to "function calling" feature https://github.com/microsoft/promptflow/blob/main/examples/flows/chat/use_functions_with_chat_models/flow.dag.yaml

[0mza987]

Could you please provide a sample code or more description to help us understand your scenario? Not sure about the question "how go consume connections in a flow using function calling".

[anderl80]

Sure, if I use the template from the repository above, I need to extend the code as follows (deleted some of the code to focus on the logic).

def run_db_lookup(message: str, db_conn: CustomConnection, aoai_conn: AzureOpenAIConnection) -> str:

    db_client = # database connection

    # Your initial question's embedding
    client = AzureOpenAI(
        api_key = aoai_conn.api_key,  
        api_version = aoai_conn.api_version,
        azure_endpoint = aoai_conn.api_base
    )

    response = client.embeddings.create(
        input = message,
        model= "text-embedding-ada-002"
    )
    input_vec = response.data[0].embedding

    # Get few-shot examples from vector database
    response = db_similarity_search(input_vec)

    return response


@tool
def run_function(response_message: dict, db_conn: CustomConnection, aoai_conn: AzureOpenAIConnection) -> str:
    function_call = response_message.get("function_call", None)
    if function_call and "name" in function_call and "arguments" in function_call:
        function_name = function_call["name"]
        function_args = json.loads(function_call["arguments"])

        # Pass db_conn and aoai_conn to run_db_lookup
        if function_name == "run_db_lookup":
            result = run_db_lookup(**function_args, db_conn=db_conn, aoai_conn=aoai_conn)
        else:
            result = globals()[function_name](**function_args)
    else:
        print("No function call")
        if isinstance(response_message, dict):
            result = response_message.get("content", "")
        else:
            result = response_message
    return result

Only @tool decorated functions can hold connections. Only one function can hold @tool decorator. LLM cannot invoke functions that are @tool decorated.
So I needed to add this to the run_function code:

  if function_name == "run_db_lookup":
      result = run_db_lookup(**function_args, db_conn=db_conn, aoai_conn=aoai_conn)

Is this a good practice? LLM so than can use run_db_lookup for function calling (it doesn't know of the connection settings and can obviously not pass them). And run_function adds via the added line the required credentials to access AOAI and the database.

[0mza987]

Basically I think your workaround is fine. If you don't like the ugly hard-coding, you can wrap the function with prepared connections:

from functools import partial

@tool
def run_function(response_message: dict, db_conn: CustomConnection, aoai_conn: AzureOpenAIConnection) -> str:
    functions = globals()
    functions["run_db_lookup"] = partial(run_db_lookup, db_conn=db_conn, aoai_conn=aoai_conn)
    ...

In this way you can use the original result = globals()[function_name](**function_args) to call the function with connection.