Architecture: Fleet

Fleet Service (orb-fleet)

This microservice is responsible for:

• Agent communications (RPC list here)
  • process client RPC requests (e.g. request for policies)
  • send control plane RPC requests (e.g. list of groups an agent belongs to)
  • process heartbeats and capabilities from agents, maintain information about their state such as which policies are active and whether any are in an error state
  • consume system events from policy service to send agents policy updates in real time according to their group membership
• Agent management
  • /agents User facing CRUD management of Agents
  • especially bootstrapping a new agent to receive connection information (id, channel, key)
  • interact with mainflux things service (1:1 between agent and thing, and between agent and channel), maintaining things, channels, and their connection
• Agent Backend capabilities management
  • GET /agents/backends endpoint for getting a list of agent backends (initially just pktvisor)

[
  {"backend": "pktvisor",
   "description": "pktvisor observability agent from pktvisor.dev"
  }
]

• GET /agents/backends/{backend}/ custom endpoints per backend
  • GET /agents/backends/pktvisor/taps retrieve list of Taps available from current agents (see https://github.com/ns1labs/pktvisor/blob/develop/RFCs/2021-04-16-75-taps.md)

[
  {"name": "anycast_dns",
   "input_type": "pcap",
   "config_predefined": [ "iface" ],
   "agents": {
     "total": 12
   }
  }
]

• GET /agents/backends/pktvisor/inputs retrieve list of Inputs available across current agents (see https://github.com/ns1labs/pktvisor/blob/develop/RFCs/2021-04-16-77-module-reflection.md)

• GET /agents/backends/pktvisor/handlers retrieve list of Handlers available across current agents (see https://github.com/ns1labs/pktvisor/blob/develop/RFCs/2021-04-16-77-module-reflection.md)

• Agent Group management

  • /agent_groups User facing CRUD management of Agents Groups
  • provide gRPC service that allows policy service to verify agent groups during policy creation
  • interact with mainflux things service (1:1 between agent group and channel), maintaining connections between group channel and agents in the group
  • able to report on which and how many agents currently match the group (optionally limited to those agents which are currently online)

Agent Data Model

DB Field	JSON Field	Public	ReadOnly	Description
mf_thing_id	id	X	X	UUIDv4 (known as mf_thing_id in the internal models because it comes from the mainflux thing id)
mf_owner_id				UUIDv4 tenant owner ID
name	name	X		A name label field
mf_channel_id	channel_id		X	Communication channel ID (UUIDv4), unique to this agent and created at agent creation
agent_tags	agent_tags	X	X	Orb tags field: sent in by the agent when it connects
orb_tags	orb_tags	X		Orb tags field: defined through the API or UI
ts_created	ts_created	X	X	A timestamp of creation
agent_metadata	agent_metadata	X	X	JSON object sent in by the agent representing its Capabilities
state	state	X	X	Current connection status of the agent, one of: 'new', 'online', 'offline', 'stale', 'removed'
error_state	error_state	X	X	Boolean which indicates whether the agent is in an error state or not. Heartbeat data contains error information.
last_hb_data	last_hb_data	X	X	JSON object sent in by the agent as its last heartbeat
ts_last_hb	ts_last_hb	X	X	A time stamp of the last heartbeat that was received

Agent Group Data Model

DB Field	JSON Field	Public	ReadOnly	Description
id	id	X	X	UUIDv4
mf_owner_id				UUIDv4 tenant owner ID
name	name	X		A name label field
description	description	X		Description
mf_channel_id	channel_id		X	Communication channel ID (UUIDv4), unique to this group and created at group creation
tags	tags	X		Orb tags field: defined through the API or UI
ts_created	ts_created	X	X	A timestamp of creation

Agent Capabilities

            "agent_metadata": {
                "backends": {
                    "pktvisor": {
                        "data": {
                            "taps": {
                                "mydefault": {
                                    "config": {
                                        "iface": "en0"
                                    },
                                    "input_type": "pcap",
                                    "interface": "visor.module.input/1.0"
                                }
                            }
                        },
                        "version": "3.3.0-develop"
                    }
                },
                "orb_agent": {
                    "version": "1.0.0-develop"
                }
            },

Core to Agent communication paths for synchronized state

These actions are provided by the fleet AgentCommsService.

Control Plane Action	RPC	Sent To	Agent Action
Agent Group created	NotifyAgentNewGroupMembership	Agent Channel (for each agent in group)	Agent subscribes to group channel specified in RPC
x	NotifyAgentAllDatasets	x	x
x	NotifyGroupNewDataset	x	x
x	NotifyGroupRemoval	x	x
x	NotifyGroupPolicyRemoval	x	x
x	NotifyGroupDatasetRemoval	x	x
x	NotifyGroupPolicyUpdate	x	x

Agent Action	RPC	Sent To	Agent Action
Bootstrap	NotifyAgentGroupMemberships	Agent Channel	Agent subscribes to all groups listed in RPC