Fetch useful info from clients and store it as client metadata

[misje]

Client metadata and asset management

Although I imagine everyone using Velociraptor mostly to fetch data en masse using hunts, personally I usually look up individual clients for investigations. Later on I will run targeted hunts based on metadata/tags, and rarely hunt across the whole fleet.

One of the perhaps biggest upgrades for myself has been to be able to look up clients based on their serial number or assigned owner (fetched from an asset management system). I have since extended this system to tie the client to external systems and agents like Defender/Intune/Entra and Action1/NinjaOne based on its serial or agent ID.

I have used most of these artifacts for years now, but I have not had the capacity to document how to use them. predictible has since written a good knowledge base artifact that covers most if not all the concepts, so releasing the following artifacts may not require a lot of additional documentation. There should be at least a complete example showing

• How to extend Custom.Generic.Client.Info with Generic.Client.HW.Identification
• Configure Server.Monitor.StoreClientHWInfo (illustrate how to use SerialColumn and HWInfoMetadata)
• Configure Server.Monitor.StoreClientInfo (illustrate how to use InfoMetadata)

An excellent showcase would use Defender, ADStatus FIXME for more useful fields.

Perhaps a knowledge base article or blog post (whichever is the most suitable format) referring to predictible's existing article, then simply setting up all artifacts and showing how to look up various indexed metadata provided by these new artifacts is enough?

Client artifacts

Gather information from the client system.

These are helper artifacts fetching information about the client. The aim is to extract some sort of identifier, a computer serial or some sort of ID, that will later be used to fetch a lot of telemetry from the computer from another agent. – Rather than implementing all of that in Veloriraptor. However, when the useful data is already there as part of fetching this identifier, it will be included in the artifact.

The intended way to use of these artifacts is by calling them from Custom.Generic.Client.Info, but they may be used on their own.

Generic.Client.HW.Identification

This artifact fetches useful hardware identification values from all OSes and has served me well for years. It would be nice to gather experience from manufacturers that I have never run clients on. The only changes it has seen recently is a very rudimentary attempt to detect whether the client is running in a VM.

Generic.Client.ADStatus

The primary use case for this artifact is to get a simple answer to whether the computer is joined to an Active Directory domain or not. Depending on the OS, a lot more details are provided.

Generic.Client.Defender.Health

The primary use case for this artifact is to get the MDATP machine ID for all three OSes, but an extensive list of health data from the agent is fetched.

TODO: A good fourth example

I have some ideas, but I am very open for suggestions that goes beyond just my own particular needs.

Server event artifacts

Store information from interrogation as client metadata.

Storing data from the Custom.Generic.Client.Info is easy and almost does not require its own published artifact. However, the following two artifacts saves the user from having to write this themselves, and lets them configure their desired result from parameters.

Server.Monitor.StoreClientHWInfo

This is a relatively simple artifact, but it is very useful. The parameter SerialColumn lets you choose which metadata columns (defaults assuming Generic.Client.HW.Identification is used) to use for serial value, in a prioritised order. The artifact comes with common patterns to ignore. It could be extended with some transforms for manufacturers who store serials in a weird way.

Server.Monitor.StoreClientInfo

Given a specified interrogation artifact, this artifacts stores specified columns from specified sources as optionally renamed metadata values. Sources should only contain one row, but if they contain more, the first row is picked.

Example use of the InfoMetadata parameter:

Source	Field	Alias
BasicInformation	OS	os
BasicInformation	Architecture	arch
ADStatus	DomainJoined	ad_joined
MDATPHealth	MDATPHealth.edrMachineId	defender_id
ComputerSetup	Details.computer setup.execution.timestamp	cs_executed
ComputerSetup	Details.computer setup.git.hash	cs_version

(The ComputerSetup source here is an artifact fetching metadata produced by an ansible playbook run, telling us when it was last executed, and what version.)

This artifact could probably be over-engineered to add some sort of transformation logic to support multiple rows. However, I think it is fine as it is.

I am not sure if it is fixed, but attempting to store empty metadata created issues in some version, hence the conditional at the end.

Asset management

Now that clients have various IDs set in metadata, it would be great to pull useful data (not stored on the endpoint) from external sources using APIs. Examples:

• Fetch assigned owner and assigned use from an asset management system
• Look up account name / account ID
• Fetch vulnerabilities and security issues/recommendations
• TODO: plenty of examples to fill

Use cases where this additional information is fetched and then used in a notebook are topics for a chapter of its own. Right now we want to fetch data to store as additional indexed metadata. We will use one particular example: Fetching information about the assigned owner from an asset management system.

Velociraptor will give us plenty of information about logged-in users, but I want to know who actually owns the computer. This can then be used to determine whether the last-logged in user shouldn't even be logged into the system in the first place. Other information, like assigned department, location, or other useful custom fields, could also be valuable.

We will use Snipe-IT as our example asset management system. It is most likely not used in large enterprises, but is open-source and although a bit cumbersome, still powerful and useful. I have been planning to write one for Atlassian Assets. The system in question matters little, since they will have have some sort of REST API and very common concepts.

We want to do the following:

• Fetch assets (perhaps matching certain filters)
• Match assets by serial
• Transform some of the information
• Synchronise selected data from the asset and save as client metadata

We want this data updated, so the data will be synchronised either

• On demand (running the server artifact)
• Whenever the client is interrogated, giving us immediate updates for new clients (done by a server event artifact)
• Periodically, so that changes in the assets are updated (e.g. new owner) (done by a server event artifact)

The server event artifacts simply call the server artifacts. Since the server artifact has a lot of important parameters, and since we cannot expect the administrator to have to fill these in in three separate places, the following is done:

• The server artifact saves its arguments to a server metadata value
• The event artifacts fetches all arguments from the same value

This may be elegant or ugly, depending on who you ask, but it works and certainly beats the alternative.

Server artifacts

Server.Assets.SnipeIT.Sync

Querying Snipe-IT is trivial (at least with the QueryAPI server artifact helper). However, in order to create a user-friendly way to select which fields to synchronise, with support for custom fields and aliases, additional logic is needed. The artifact may be used in three ways:

• Simply fetch all assets (matching filters)
• Fetch assets with a serial found in Velociraptor clients
• Synchronise client metadata

The first two are "read-only" actions and can be used with notebook suggestions to

• Find clients missing serial numbers
• Find clients not in assets
• Find assets not running Velociraptor

Server event artifacts

Server.Monitor.SnipeIT.Sync.Interrogation

Update client metadata whenever it is interrogated. Shows what metadata changed (from/to).

Server.Monitor.SnipeIT.Sync.Periodic

Update client metadata at a specified interval. Shows what metadata changed (from/to).

Artifact table

Complete artifact list (with dependencies, if any):

Name	Type	Deps.	Description
Generic.Client.HW.Identification	Client	-	Pull hardware identification information from the system, like serial number and computer model
Generic.Client.ADStatus	Client	-	Fetch Active Directory status
Generic.Client.Defender.Health	Client	-	Get Microsoft Defender health status
Server.Monitor.StoreClientHWInfo	Server	-	Store client HW info as metadata
Server.Monitor.StoreClientInfo	Server	(HW.Identification)	Store client info as metadata
Server.Utils.QueryAPI	Server	-	Query an HTTP API with login and pagination support
Server.Assets.SnipeIT.Sync	Server	(QueryAPI)	Save key Snipe-IT asset information as client metadata
Server.Monitor.SnipeIT.Sync.Interrogation	Server event	SnipeIT.Sync	Update asset information when client is interrogated
Server.Monitor.SnipeIT.Sync.Periodic	Server event	SnipeIT.Sync	Update asset information regularly at a specified interval