Skip to content

Conversation

@alvidofaisal
Copy link

This PR introduces a Kubernetes operator that enables declarative management of Eino applications on Kubernetes. The operator automates deployment, scaling, and lifecycle management of LLM orchestrations built with the Eino framework.

What type of PR is this?

feat: A new feature

Check the PR title.

  • This PR title match the format: <type>(optional scope): <description>
  • The description of this PR title is user-oriented and clear enough for others to understand.
  • Attach the PR updating the user documentation if the current PR requires user awareness at the usage level. User docs repo

(Optional) Translate the PR title into Chinese.

feat(operator): 添加用于 EinoChainApp 管理的 Kubernetes 操作器

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
This commit introduces a Kubernetes operator for managing EinoChainApp custom resources. Key features include:

  • EinoChainApp CRD: Defines custom resource with fields for image, replicas, deployment template, service configuration, and autoscaling parameters
  • Automated Resource Management: Controller reconciles EinoChainApp resources by managing Deployments, Services, and HorizontalPodAutoscalers
  • Custom Autoscaling: Built-in HPA support using eino_token_per_sec_avg_per_pod metric, optimized for LLM workload performance
  • Status Reporting: Updates EinoChainApp status with managed resource states and autoscaling activity
  • Comprehensive Testing: Includes unit and integration tests covering CRD lifecycle, resource management, and status updates
  • RBAC Ready: Defines necessary permissions and deployment manifests

The operator enables users to declaratively define and manage Eino applications on Kubernetes with intelligent autoscaling based on token processing throughput, a key performance indicator for LLM workloads.

zh(optional):
此提交为管理 EinoChainApp 自定义资源引入了 Kubernetes 操作器。主要功能包括 EinoChainApp CRD 定义、自动化资源管理、基于令牌处理吞吐量的自定义自动扩缩容,以及全面的测试覆盖。

(Optional) Which issue(s) this PR fixes:

(optional) The PR that updates user documentation:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant