feat(go/genkit): add DefineSchema and Dotprompt reference support #2769

ihan211 · 2025-04-16T01:23:21Z

This PR adds a schema definition system to genkit with Dotprompt reference support.

Key Components

Schema registration with Dotprompt framework
DefineSchema API for defining prompt data structures
Enhanced initialization process for schema registration
Schema resolution within templates

Example Usage

// Define a product schema
type ProductSchema struct {
    Name        string  `json:"name"`
    Description string  `json:"description"`
    Price       float64 `json:"price"`
    Category    string  `json:"category"`
    InStock     bool    `json:"inStock"`
}

// Register the schema with genkit
genkit.DefineSchema("ProductSchema", ProductSchema{})

// Reference the schema in a Dotprompt template
// promptContent example:
// ---
// input:
//   schema:
//     theme: string
// output:
//   schema: ProductSchema
// ---
// Generate a product that fits the {{theme}} theme.
// Make sure to provide a detailed description and appropriate pricing.

The implementation enables:
* Automatic schema registration during Genkit initialization
* Structured output parsing from model responses to Go structs
* Schema reference resolution within Dotprompt templates
* JSON Schema support for standardized type definitions

Testing
Unit tests in schema_test.go
Working example in samples directory

Checklist (if applicable):
- [X] PR title is following https://www.conventionalcommits.org/en/v1.0.0/
- [X] Tested (manually, unit tested, etc.)
- [ ] Docs updated (updated docs or a docs bug required)

AbeJLazaro · 2025-04-16T14:08:34Z

go/genkit/schema.go

+//
+//	personSchema := genkit.DefineSchema("Person", Person{})
+func DefineSchema(name string, schema Schema) Schema {
+	if name == "" {


Doubt, if core.RegisterSchema already has those validations, is it okay to have them also here? Just learning :D

You raise an interesting point, genkit.DefineSchema not only calls core.RegisterSchema but also adds to pendingSchemas. refactor to remove duplication of validation is a good idea, I'm thinking to refactor pendingSchemas to core package.

AbeJLazaro · 2025-04-16T14:12:10Z

go/genkit/schema.go

+}
+
+// registerSchemaResolver registers a schema resolver with Dotprompt to handle schema lookups
+func registerSchemaResolver(dp *dotprompt.Dotprompt) {


what is the purpose of the private function? Maybe the Public one (RegisterGlobalSchemaResolver) could instance the private one (registerSchemaResolver) to factorize code, what are your thoughts about it?

Good observation! and your suggestion follows DRY (Don't Repeat Yourself) principle. I'll expose it out by adding func RegisterGlobalSchemaResolver(dp *dotprompt.Dotprompt) { registerSchemaResolver(dp) }

AbeJLazaro · 2025-04-16T14:15:57Z

go/internal/registry/schema.go

+	}
+
+	r.Dotprompt.DefineSchema(name, jsonSchema)
+	r.RegisterValue("schema/"+name, structType)


what about adding 'schema/' as a const?

EditGood suggestion! I'll add a const SchemaPrefix = "schema/" .

AbeJLazaro · 2025-04-16T14:17:58Z

go/internal/registry/schema.go

+
+	r.Dotprompt.DefineSchema(name, jsonSchema)
+	r.RegisterValue("schema/"+name, schema)
+	r.setupSchemaLookupFunction()


maybe RegisterSchemaWithDotprompt also could call DefineSchema and only ejecute r.setupSchemaLookupFunction() after calling it and validating there is no error.

Good idea! I'll update the code to call setupSchemaLookupFunction() only after successful schema registration to ensure proper error handling.

AbeJLazaro · 2025-04-16T14:25:49Z

go/internal/registry/schema.go

+		schema.Properties = orderedmap.New[string, *jsonschema.Schema]()
+		schema.Required = []string{}
+
+		for i := 0; i < t.NumField(); i++ {


Maybe this part of code could be factorized in another function to be recursively? Well, supporting more nesting levels could be another issue for itself. Let me know what you think! :D

I agree this section could benefit from refactoring! I'll create a helper function to handle the field-to-schema conversion recursively, which would improve code organization and support nested struct types more elegantly.

apascal07 · 2025-04-17T21:02:18Z

Tests are not passing, can you resolve that first?

hugoaguirre · 2025-04-21T17:36:20Z

go/core/schema.go

+// SchemaType is the type identifier for schemas in the registry.
+const SchemaType = "schema"
+
+// schemaRegistry maintains registry of schemas.


Is this comment needed here?

hugoaguirre · 2025-04-21T19:33:28Z

go/core/schema.go

+	schemasMu.RLock()
+	defer schemasMu.RUnlock()
+
+	// First check local registry


This code is pretty clear to read, these comments are not needed at all.
A good suggestion from Alex was that we should remove the comments that explains what and only keep the comments that explains why

hugoaguirre · 2025-04-21T19:34:39Z

go/core/schema.go

+
+// GetPendingSchemas returns a copy of pending schemas that need to be
+// registered with Dotprompt.
+func GetPendingSchemas() map[string]Schema {


Suggested change

func GetPendingSchemas() map[string]Schema {

func PendingSchemas() map[string]Schema {

hugoaguirre · 2025-04-21T19:34:55Z

go/core/schema.go

+}
+
+// GetSchemas returns a copy of all registered schemas.
+func GetSchemas() map[string]any {


Suggested change

func GetSchemas() map[string]any {

func Schemas() map[string]any {

In general, avoid having function names with Get/Set prefixes 👍🏽

hugoaguirre · 2025-04-21T19:39:31Z

go/genkit/schema.go

+
+// GetSchema retrieves a registered schema by name.
+// It returns an error if no schema exists with that name.
+func GetSchema(name string) (Schema, error) {


What's the reason behind having GetSchema if it internally calls LookupSchema?

hugoaguirre · 2025-04-21T19:40:25Z

go/genkit/schema.go

+
+// LookupSchema retrieves a registered schema by name.
+// It returns nil and false if no schema exists with that name.
+func LookupSchema(name string) (Schema, bool) {


Wouldn't be cleaner and easier to use if the return value would be to return Schema if found and nil if not found?

hugoaguirre · 2025-04-21T19:41:11Z

go/genkit/schema.go

+			return nil
+		}
+
+		// Convert the schema to a JSON schema


nit: not needed, same for comment in line 113

hugoaguirre · 2025-04-21T19:44:51Z

go/internal/registry/schema.go

+
+	schema := &jsonschema.Schema{
+		Type:       "object",
+		Properties: orderedmap.New[string, *jsonschema.Schema](),


Curious, why do we need an ordered map?

hugoaguirre · 2025-04-21T20:03:37Z

go/core/schema.go

+
+// ClearSchemas removes all registered schemas.
+// This is primarily for testing purposes.
+func ClearSchemas() {


Where's this function used?

hugoaguirre · 2025-04-21T20:08:08Z

go/internal/registry/schema.go

+}
+
+// convertStructToJsonSchema converts a Go struct to a JSON schema
+func convertStructToJsonSchema(structType any) (*jsonschema.Schema, error) {


Wouldn't it be better if we use map[string]any as the type instead of just any?

github-project-automation bot added this to Genkit Backlog Apr 16, 2025

github-actions bot added the go label Apr 16, 2025

AbeJLazaro reviewed Apr 16, 2025

View reviewed changes

hugoaguirre reviewed Apr 21, 2025

View reviewed changes

hugoaguirre requested changes Apr 21, 2025

View reviewed changes

ihan211 force-pushed the ivenh/2702 branch from 1b10304 to d75d77e Compare April 21, 2025 22:08

ihan211 closed this Apr 21, 2025

ihan211 force-pushed the ivenh/2702 branch from d75d77e to 1d9fe19 Compare April 21, 2025 22:29

github-project-automation bot moved this to Done in Genkit Backlog Apr 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(go/genkit): add DefineSchema and Dotprompt reference support #2769

feat(go/genkit): add DefineSchema and Dotprompt reference support #2769

ihan211 commented Apr 16, 2025

AbeJLazaro Apr 16, 2025

ihan211 Apr 16, 2025

AbeJLazaro Apr 16, 2025

ihan211 Apr 16, 2025

AbeJLazaro Apr 16, 2025

ihan211 Apr 16, 2025

AbeJLazaro Apr 16, 2025

ihan211 Apr 16, 2025

AbeJLazaro Apr 16, 2025

ihan211 Apr 16, 2025

apascal07 commented Apr 17, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

hugoaguirre Apr 21, 2025

	func GetPendingSchemas() map[string]Schema {
	func PendingSchemas() map[string]Schema {

	func GetSchemas() map[string]any {
	func Schemas() map[string]any {

feat(go/genkit): add DefineSchema and Dotprompt reference support #2769

feat(go/genkit): add DefineSchema and Dotprompt reference support #2769

Conversation

ihan211 commented Apr 16, 2025

Key Components

Example Usage

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apascal07 commented Apr 17, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment