Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSDK-9870: Check for schema changes all the time. Allow for FTDC to consume maps. #4756

Merged
merged 13 commits into from
Jan 31, 2025

Conversation

dgottlieb
Copy link
Member

No description provided.

@viambot viambot added the safe to test This pull request is marked safe to test from a trusted zone label Jan 29, 2025
@@ -136,6 +138,11 @@ func isNumeric(kind reflect.Kind) bool {
}

func flattenStruct(item reflect.Value) ([]float32, error) {
_, b, c := flattenStructNew(item)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new flattenStructNew does everything flattenStruct does and more (gets the fields for a schema). To help test correctness of the new code, I've started this PR where everything funnels into the new code.

return fields, numbers, nil
}

func getFieldsForStruct(item reflect.Value) ([]string, error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new flattenStructNew does everything getFieldsForStruct[Old] does and more (gets the data values). To help test correctness of the new code, I've started this PR where everything funnels into the new code.

ftdc/ftdc.go Outdated
@@ -356,6 +357,131 @@ func (ftdc *FTDC) constructDatum() datum {
return datum
}

func schemaDiff(prev []string, curr []string) bool {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I use this anymore

ftdc/ftdc.go Outdated
return !slices.Equal(prev, curr)
}

func walk(data map[string]any, inputSchema *schema) (*schema, []float32, error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the new method that combines getting a schema and data values.

ftdc/ftdc.go Outdated
return inputSchema, values, nil
}

func (ftdc *FTDC) writeDatumNew(datum datum) error {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will become the new method to use.

ftdc/ftdc.go Outdated
// The first iteration when `currSchema` is nil, it's guaranteed that the generations will not
// be equal. In the happy path where the schema hasn't changed, the `walk` function is
// guaranteed to return the same schema object.
if datum.generationID != ftdc.outputGenerationID || ftdc.currSchema != newSchema {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think generations will go away entirely. But I'll save that for a later commit in this PR

@@ -124,6 +211,7 @@ func (badStatser badStatser) Stats() any {
}

func TestRemoveBadStatser(t *testing.T) {
t.Skip("It is now legal to change schemas without a corresponding `Add`/`Remove` statster call.")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asserts on a behavior that is no longer true

@@ -248,6 +336,7 @@ func TestNestedStructs(t *testing.T) {
// TestStatsWriterContinuesOnSchemaError asserts that "schema errors" are handled by removing the
// violating statser, but otherwise FTDC keeps going.
func TestStatsWriterContinuesOnSchemaError(t *testing.T) {
t.Skip("It is now legal to change schemas without a corresponding `Add`/`Remove` statster call.")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asserts on a behavior that is no longer true

@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 29, 2025
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 29, 2025
@dgottlieb dgottlieb requested a review from cheukt January 30, 2025 18:26
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 30, 2025
@dgottlieb dgottlieb changed the title Check for schema changes all the time RSDK-9870: Check for schema changes all the time. Allow for FTDC to consume maps. Jan 30, 2025
@dgottlieb dgottlieb closed this Jan 30, 2025
@dgottlieb dgottlieb deleted the dynamic-ftdc branch January 30, 2025 18:29
@dgottlieb dgottlieb restored the dynamic-ftdc branch January 30, 2025 18:30
@dgottlieb dgottlieb reopened this Jan 30, 2025
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 30, 2025
@dgottlieb dgottlieb added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 30, 2025
@viambot viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Jan 30, 2025
@dgottlieb dgottlieb merged commit 8acdd7f into viamrobotics:main Jan 31, 2025
16 checks passed
@dgottlieb dgottlieb deleted the dynamic-ftdc branch January 31, 2025 13:55
vijayvuyyuru pushed a commit to vijayvuyyuru/rdk that referenced this pull request Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test This pull request is marked safe to test from a trusted zone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants