Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][flytepropeller][flyteagent] Rename Agent Service To Connector Service #6320

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Future-Outlier
Copy link
Member

@Future-Outlier Future-Outlier commented Mar 10, 2025

Tracking issue

Why are the changes needed?

Given there's a name collision between LLM agent and flyte agent, we are going to rename flyte agent to flyte connectors.

What changes were proposed in this pull request?

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

  • added: For new features.
  • changed: For changes in existing functionality.
  • deprecated: For soon-to-be-removed features.
  • removed: For features being removed.
  • fixed: For any bug fixed.
  • security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR renames the Flyte Agent Service to Flyte Connector Service to eliminate naming collisions and improve clarity. The changes update service identification across the codebase by replacing hard-coded values with dynamic references, while also updating type names, function signatures, package names, and documentation to enhance maintainability.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 5

@flyte-bot
Copy link
Collaborator

flyte-bot commented Mar 10, 2025

Code Review Agent Run #476623

Actionable Suggestions - 8
  • flyteplugins/go/tasks/plugins/webapi/agent/plugin.go - 3
    • Consider consistency between struct field names · Line 48-48
    • Parameter type doesn't match function name · Line 348-348
    • Parameter type mismatch in RegisterConnectorPlugin · Line 432-432
  • flyteplugins/go/tasks/pluginmachinery/core/plugin.go - 1
    • Method receiver type changed without updating references · Line 74-75
  • flyteplugins/go/tasks/plugins/webapi/agent/config.go - 1
  • flyteplugins/go/tasks/plugins/webapi/connector/plugin.go - 1
    • Missing output handling for State_SUCCEEDED · Line 311-311
  • flyteplugins/go/tasks/plugins/webapi/connector/config.go - 1
  • flyteplugins/go/tasks/plugins/webapi/connector/integration_test.go - 1
    • Missing registry field in plugin initialization · Line 339-339
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • flyteplugins/go/tasks/plugins/webapi/connector/client.go - 1
    • Potential nil pointer dereference in error handling · Line 116-116
Review Details
  • Files reviewed - 17 · Commit Range: ea8caab..ea8caab
    • flyteplugins/go/tasks/pluginmachinery/core/plugin.go
    • flyteplugins/go/tasks/plugins/webapi/agent/client.go
    • flyteplugins/go/tasks/plugins/webapi/agent/client_test.go
    • flyteplugins/go/tasks/plugins/webapi/agent/config.go
    • flyteplugins/go/tasks/plugins/webapi/agent/config_test.go
    • flyteplugins/go/tasks/plugins/webapi/agent/integration_test.go
    • flyteplugins/go/tasks/plugins/webapi/agent/plugin.go
    • flyteplugins/go/tasks/plugins/webapi/agent/plugin_test.go
    • flyteplugins/go/tasks/plugins/webapi/connector/client.go
    • flyteplugins/go/tasks/plugins/webapi/connector/client_test.go
    • flyteplugins/go/tasks/plugins/webapi/connector/config.go
    • flyteplugins/go/tasks/plugins/webapi/connector/config_test.go
    • flyteplugins/go/tasks/plugins/webapi/connector/integration_test.go
    • flyteplugins/go/tasks/plugins/webapi/connector/plugin.go
    • flyteplugins/go/tasks/plugins/webapi/connector/plugin_test.go
    • flytepropeller/pkg/controller/nodes/task/handler.go
    • flytepropeller/pkg/controller/nodes/task/handler_test.go
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

@Future-Outlier Future-Outlier changed the title [flytepropeller][flyteagent] Rename Agent Service To Connector Service [WIP][flytepropeller][flyteagent] Rename Agent Service To Connector Service Mar 10, 2025
Copy link

codecov bot commented Mar 10, 2025

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 57.90%. Comparing base (b779bed) to head (400e702).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...lytepropeller/pkg/controller/nodes/task/handler.go 78.94% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6320      +/-   ##
==========================================
- Coverage   58.49%   57.90%   -0.60%     
==========================================
  Files         937      774     -163     
  Lines       71088    57321   -13767     
==========================================
- Hits        41583    33189    -8394     
+ Misses      26353    21639    -4714     
+ Partials     3152     2493     -659     
Flag Coverage Δ
unittests-datacatalog 59.06% <ø> (ø)
unittests-flyteadmin 56.30% <ø> (ø)
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 64.76% <ø> (+0.05%) ⬆️
unittests-flyteidl 76.12% <ø> (ø)
unittests-flyteplugins ?
unittests-flytepropeller 54.80% <78.94%> (+<0.01%) ⬆️
unittests-flytestdlib 64.02% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Future-Outlier <[email protected]>
Copy link
Member Author

@Future-Outlier Future-Outlier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo:

  • make enabled-plugins: agent-service to connector service
  • implement the merge config funciton in GetConfig
  • test in internal cluster
  • discuss how to merge the webapi config with @pingsutw

@flyte-bot
Copy link
Collaborator

flyte-bot commented Mar 10, 2025

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Feature Improvement - Connector Service Renaming Enhancement

plugin.go - Renamed AgentService to ConnectorService and updated corresponding method references.

client.go - Updated package declaration and refactored client functions, connection logic, and registry constructions from agent to connector.

client_test.go - Modified test cases to reference connector clients instead of agent clients.

config.go - Refactored configuration settings by renaming default agent fields to default connector fields along with polling intervals and deployment mappings.

config_test.go - Adjusted configuration tests to validate new connector settings and ensure consistency with the renaming.

integration_test.go - Updated integration test suite to align with the connector service naming.

plugin.go - Replaced agent-specific types and functions with their connector counterparts for consistency.

plugin_test.go - Changed package declaration from agent to connector to reflect new naming.

client.go - Introduced connector client with comprehensive gRPC setup and connection management.

client_test.go - Added tests verifying the initialization and functionality of connector client sets.

config.go - Defined new configuration structure for connector service with updated default settings and deployment mappings.

plugin.go - Implemented connector plugin with task execution, client management, and output writing functionalities.

handler.go - Updated plugin registration and task resolution to use connector service, replacing agent references.

Testing - Connector Service Testing Additions

config_test.go - Adds unit tests for connector configuration, ensuring proper setting and retrieval.

integration_test.go - Introduces integration tests for async and sync task flows, covering error handling and task retries.

plugin_test.go - Updated test cases to verify various task statuses and error conditions for the connector plugin.

handler_test.go - Modified tests to use connector service and validate the refactored plugin interfaces.

Message string
LogLinks []*flyteIdl.TaskLog
CustomInfo *structpb.Struct
ConnectorError *admin.AgentError
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider consistency between struct field names

The field name AgentError has been renamed to ConnectorError in ResourceWrapper struct. This change seems to be part of a larger refactoring from 'agent' to 'connector' terminology, but the corresponding field in the Resource struct (line 838 in agent.pb.go) is still named AgentError. This inconsistency might cause issues when mapping between these structures.

Code suggestion
Check the AI-generated fix before applying
Suggested change
ConnectorError *admin.AgentError
AgentError *admin.AgentError

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

}
return client, nil
}

func (p *Plugin) watchAgents(ctx context.Context, agentService *core.AgentService) {
func (p *Plugin) watchConnectors(ctx context.Context, connectorService *core.AgentService) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameter type doesn't match function name

The function name has been changed from watchAgents to watchConnectors, but there's a parameter type mismatch. The parameter connectorService is still of type *core.AgentService instead of *core.ConnectorService. This could lead to confusion and potential issues if the code expects a different service type.

Code suggestion
Check the AI-generated fix before applying
Suggested change
func (p *Plugin) watchConnectors(ctx context.Context, connectorService *core.AgentService) {
func (p *Plugin) watchConnectors(ctx context.Context, connectorService *core.ConnectorService) {

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

return plugin, nil
},
}
}

func RegisterAgentPlugin(agentService *core.AgentService) {
func RegisterConnectorPlugin(connectorService *core.AgentService) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameter type mismatch in RegisterConnectorPlugin

The function parameter connectorService is of type *core.AgentService but based on the context, it should likely be *core.ConnectorService. This inconsistency could lead to type mismatch issues when the function is called.

Code suggestion
Check the AI-generated fix before applying
Suggested change
func RegisterConnectorPlugin(connectorService *core.AgentService) {
func RegisterConnectorPlugin(connectorService *core.ConnectorService) {

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +74 to +75
// SetSupportedTaskType set supportTaskType in the connector service.
func (p *ConnectorService) SetSupportedTaskType(taskTypes []TaskType) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method receiver type changed without updating references

The method signature has been changed from AgentService to ConnectorService, but there are references to this method in other files that might still be using AgentService. Consider updating all references to ensure consistency.

Code suggestion
Check the AI-generated fix before applying
Suggested change
// SetSupportedTaskType set supportTaskType in the connector service.
func (p *ConnectorService) SetSupportedTaskType(taskTypes []TaskType) {
// SetSupportedTaskType set supportTaskType in the connector service.
func (p *ConnectorService) SetSupportedTaskType(taskTypes []TaskType) {

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +140 to +141
// Maps task types to their connectors. {TaskType: connectorDeploymentID}
ConnectorForTaskTypes map[string]string `json:"connectorForTaskTypes" pflag:"-,"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider migration path for renamed field

The field name has been changed from AgentForTaskTypes to ConnectorForTaskTypes to better reflect its purpose. However, any code that previously referenced AgentForTaskTypes will now break. Consider adding a migration path or deprecation notice if this is a breaking change to existing configurations.

Code suggestion
Check the AI-generated fix before applying
Suggested change
// Maps task types to their connectors. {TaskType: connectorDeploymentID}
ConnectorForTaskTypes map[string]string `json:"connectorForTaskTypes" pflag:"-,"`
// Maps task types to their connectors. {TaskType: connectorDeploymentID}
ConnectorForTaskTypes map[string]string `json:"connectorForTaskTypes" pflag:"-,"`
// Deprecated: Use ConnectorForTaskTypes instead. Maps task types to their agents. {TaskType: agentDeploymentID}
AgentForTaskTypes map[string]string `json:"agentForTaskTypes" pflag:"-," deprecated:"use connectorForTaskTypes instead"`

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

return core.PhaseInfoFailure(pluginErrors.TaskFailedWithError, "failed to run the job.\n"+resource.Message, taskInfo), nil
case admin.State_RETRYABLE_FAILURE:
return core.PhaseInfoRetryableFailure(pluginErrors.TaskFailedWithError, "failed to run the job.\n"+resource.Message, taskInfo), nil
case admin.State_SUCCEEDED:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing output handling for State_SUCCEEDED

It looks like there's a missing implementation for handling the State_SUCCEEDED case in the Status function. The function should write outputs and return a success phase when the state is State_SUCCEEDED, similar to how it's done for the TaskExecution_SUCCEEDED phase. This could lead to tasks not completing properly when using the legacy state field.

Code suggestion
Check the AI-generated fix before applying
 @@ -311,0 +312,7 @@
 	case admin.State_SUCCEEDED:
 		err = writeOutput(ctx, taskCtx, resource.Outputs)
 		if err != nil {
 			logger.Errorf(ctx, "failed to write output with err %s", err.Error())
 			return core.PhaseInfoUndefined, fmt.Errorf("failed to write output with err %s", err.Error())
 		}
 		return core.PhaseInfoSuccess(taskInfo), nil

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

Comment on lines +176 to +177
func SetConfig(cfg *Config) error {
return connectorConfigSection.SetConfig(cfg)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type mismatch in SetConfig function

The SetConfig function is trying to set a *Config type to the connectorConfigSection which expects a *ConnectorConfig. This type mismatch could lead to runtime errors. Consider updating the function to accept *ConnectorConfig instead.

Code suggestion
Check the AI-generated fix before applying
Suggested change
func SetConfig(cfg *Config) error {
return connectorConfigSection.SetConfig(cfg)
func SetConfig(cfg *ConnectorConfig) error {
return connectorConfigSection.SetConfig(cfg)

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

},
registry: agentRegistry,
}, nil
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing registry field in plugin initialization

The plugin initialization in newMockAsyncAgentPlugin() is missing required fields. The current implementation creates a Plugin with empty maps for clients but doesn't set the registry field which is used in the plugin's logic. Consider initializing the registry field with the agentRegistry that's already defined in the function.

Code suggestion
Check the AI-generated fix before applying
Suggested change
},
},
registry: agentRegistry,

Code Review Run #476623


Should Bito avoid suggestions like this for future reviews? (Manage Rules)

  • Yes, avoid them

@flyte-bot
Copy link
Collaborator

flyte-bot commented Mar 10, 2025

Code Review Agent Run #d220e2

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: ea8caab..400e702
    • flyteplugins/go/tasks/plugins/webapi/connector/plugin.go
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants