[Solutions] Ollama integration

[jstanden]

See: https://ollama.com

This needs a solutions integration page and social media image.

It probably doesn't need a Cerb connected service because there's no API auth by default. This would be used locally, or with a proxy like Caddy or Nginx in front it for auth.

The code sample could do a code completion with a local model. We also support it in llm.agent:.

Embeddings

start:
  http.request/embeddings:
    output: http_response
    inputs:
      method: POST
      url: http://host.docker.internal:11434/api/embeddings
      headers:
        Content-Type: application/json
      body:
        model: nomic-embed-text
        prompt: This is some text I need to embed

Chat completions with tool calling

You can clean this one up. It's a rough example and shared for completeness.

I'd probably just do a user message (w/o tools) and let the LLM respond. Tool use is much easier with llm.agent: than doing it by hand like this.

start:
  http.request/chat:
    output: http_response
    inputs:
      method: POST
      url: http://host.docker.internal:11434/api/chat
      headers:
        Content-Type: application/json
      body:
        model: llama3.2
        stream@bool: false
        messages:
          0:
            role: user
            content: What is the weather in Los Angeles, CA?
          1:
            role: assistant
            content@text:
            tool_calls:
              0:
                function:
                  name: get_current_weather
                  arguments:
                    format: fahrenheit
                    location: Los Angeles, CA
          2:
            role: tool
            name: get_current_weather
            content@text:
              Weather Today in Los Angeles, CA
              Feels Like
              50°
              Rain
              Sunrise
              6:35 am
              Sunset
              5:32 pm
              High / Low
              --/46°
              Wind
              8 mph
              Humidity
              94%
              Dew Point
              51°
              Pressure
              29.94
              UV Index
              0 of 11
              Visibility
              3 mi
              Moon Phase
              Waning Gibbous
        tools:
          0:
            type: function
            function:
              name: get_current_weather
              description: Get the current weather for a location
              parameters:
                type: object
                properties:
                  location:
                    type: string
                    description: The location to get the weather for, e.g. San Francisco, CA
                  format:
                    type: string
                    description: The format to return the weather in e.g. 'celsius' or 'fahrenheit'
                    enum@csv: celsius, fahrenheit
                required@csv: location, format
    #on_error:  
    on_success:
      set:
        json_response@key,json: http_response:body
        http_response@json: null