Skip to content

Latest commit

 

History

History
140 lines (101 loc) · 4.62 KB

SPRING_AI_INTEGRATION.md

File metadata and controls

140 lines (101 loc) · 4.62 KB

Spring AI Integration

Table of Contents

Introduction

This guide provides examples of how to use our Spring AI integration with our clients in SAP AI Core for chat completion tasks using the SAP AI SDK for Java.

First, add the Spring AI dependency to your pom.xml:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-core</artifactId>
    <version>1.0.0-M5</version>
</dependency>
...
<repository>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
    <id>spring-milestones</id>
    <name>Spring Milestones</name>
    <url>https://repo.spring.io/milestone</url>
</repository>

:::note Spring AI Milestone Version Note that currently no stable version of Spring AI exists just yet. The AI SDK currently uses the M6 milestone.

Please be aware that future versions of the AI SDK may increase the Spring AI version. :::

Orchestration Chat Completion

The Orchestration client is integrated in Spring AI classes:

ChatModel client = new OrchestrationChatModel();
OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
OrchestrationChatOptions opts = new OrchestrationChatOptions(config);

Prompt prompt = new Prompt("What is the capital of France?", opts);
ChatResponse response = client.call(prompt);

Please find an example in our Spring Boot application.

Orchestration Masking

Configure Orchestration modules withing Spring AI:

ChatModel client = new OrchestrationChatModel();
OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);

val masking =
    DpiMasking.anonymization()
        .withEntities(DPIEntities.EMAIL, DPIEntities.ADDRESS, DPIEntities.LOCATION);

val opts = new OrchestrationChatOptions(config.withMaskingConfig(masking));
val prompt =
    new Prompt(
        "Please write 'Hello World!' to me via email. My email address is [email protected]",
        opts);

ChatResponse response = client.call(prompt);

Please find an example in our Spring Boot application.

Stream chat completion

It's possible to pass a stream of chat completion delta elements, e.g. from the application backend to the frontend in real-time.

ChatModel client = new OrchestrationChatModel();
OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
OrchestrationChatOptions opts = new OrchestrationChatOptions(config);

Prompt prompt =
    new Prompt(
        "Can you give me the first 100 numbers of the Fibonacci sequence?", opts);
Flux<ChatResponse> flux = client.stream(prompt);

// also possible to keep only the chat completion text
Flux<String> responseFlux = 
    flux.map(chatResponse -> chatResponse.getResult().getOutput().getContent());

Note: A Spring endpoint can return Flux instead of ResponseEntity.

Please find an example in our Spring Boot application.

Tool Calling

First define a function that will be called by the LLM:

class WeatherMethod {
  enum Unit {C,F}
  record Request(String location, Unit unit) {}
  record Response(double temp, Unit unit) {}

  @Tool(description = "Get the weather in location")
  Response getCurrentWeather(@ToolParam Request request) {
    int temperature = request.location.hashCode() % 30;
    return new Response(temperature, request.unit);
  }
}

Then add your tool to the options:

ChatModel client = new OrchestrationChatModel();
OrchestrationModuleConfig config = new OrchestrationModuleConfig().withLlmConfig(GPT_35_TURBO);
OrchestrationChatOptions opts = new OrchestrationChatOptions(config);

options.setToolCallbacks(List.of(ToolCallbacks.from(new WeatherMethod())));

options.setInternalToolExecutionEnabled(false);// tool execution is not yet available in orchestration

Prompt prompt = new Prompt("What is the weather in Potsdam and in Toulouse?", options);

ChatResponse response = client.call(prompt);

Please find an example in our Spring Boot application.