SymDB: Component / Remote / Uploader integration

docs/DevelopmentGuide.md

@@ -388,6 +388,14 @@ Then [open a pull request](../CONTRIBUTING.md#have-a-patch) and be sure to add t
  - Screenshots showing a sample trace
  - Any additional code snippets, sample apps, benchmarks, or other resources that demonstrate its implementation are a huge plus!
 
+### Testing the Symbol Database without Remote Configuration
+
+Symbol Database upload normally activates via Remote Configuration when the DI
+UI is opened for the service. For local development, bypass RC and trigger the
+upload directly on tracer startup with:
+
+    export DD_INTERNAL_FORCE_SYMBOL_DATABASE_UPLOAD=true
+
 ### Generating GRPC proto stubs for tests
 
 If you modify any of the `.proto` files under `./spec/datadog/tracing/contrib/grpc/support/proto` used for

docs/DynamicInstrumentation.md

@@ -4,7 +4,7 @@
 
 Dynamic Instrumentation for Ruby is in **limited preview**.
 While the core functionality is stable, some features available in other
-languages (Java, Python, .NET) are not yet available for Ruby.
+languages (Java, Python, .NET) are not available for Ruby.
 
 > **New to Dynamic Instrumentation?**
 > This document covers Ruby-specific setup and limitations. For an
@@ -26,7 +26,7 @@ practices for using Dynamic Instrumentation.
 - Rack-based applications only
   - Includes Rails, Sinatra, and other Rack-compatible frameworks
   - Non-Rack applications are not supported
-  - Background processes and jobs (including Sidekiq, Resque, etc.) are not yet supported
+  - Background processes and jobs (including Sidekiq, Resque, etc.) are not supported
 - [Remote Configuration Management](https://docs.datadoghq.com/remote_configuration/) enabled
   - Remote Configuration is enabled by default.
   - If it's disabled, follow the [instructions to enable it](https://docs.datadoghq.com/remote_configuration/#enable-remote-configuration).
@@ -118,9 +118,9 @@ the entire method execution.
 - You're debugging method-level behavior
 - You need to track method execution time
 
-### Not Yet Supported
+### Not Supported
 
-The following probe types available in other languages are not yet
+The following probe types available in other languages are not
 supported for Ruby:
 
 - Metric probes
@@ -223,7 +223,7 @@ generated.
 ### Application Must Be Processing Requests
 - Dynamic Instrumentation is initialized via Rack middleware when
   processing HTTP requests
-- An application that has just booted but has not yet served any requests
+- An application that has just booted but has not served any requests
   will not have Dynamic Instrumentation activated
 - Dynamic Instrumentation will be automatically activated when the first
   HTTP request is processed
@@ -267,7 +267,7 @@ per-probe in the probe definition.
   data at the default depth of 3
 - Their attributes are often nested deeper than 3 levels
 - Custom serializers are available for internal Datadog use but the API
-  is not yet finalized for customer use
+  is not finalized for customer use
 - **Workaround:** Increase the capture depth for probes targeting code
   that works with complex objects
 
@@ -286,8 +286,9 @@ The value will fall back to default serialization.
 
 ## What Data Is Captured
 
-When a probe fires, Dynamic Instrumentation captures a snapshot of
-application state and sends it to Datadog. The snapshot includes:
+Dynamic instrumentation sends some of the application data to Datadog.
+
+**Probe snapshots** (captured when probes fire):
 
 - **Variable values** — local variables, method arguments, and return
   values, subject to the capture depth and collection size limits
@@ -305,6 +306,32 @@ application state and sends it to Datadog. The snapshot includes:
     redacted.
 - **Stack traces** — the call stack at the point the probe fires.
 
+**Symbol Database** (uploaded once at startup, see below):
+
+- Class, module, and method names from user application code
+- Method parameter names (not values)
+- Source file paths and line ranges
+- File content hashes (for source code version matching)
+- No runtime values, variable contents, or application data
+
+## Symbol Database
+
+The Symbol Database powers auto-completion in the Dynamic Instrumentation
+UI. When enabled, the tracer extracts symbol information (classes,
+modules, methods, parameters) from your running application and uploads
+it to Datadog via the Agent. This allows the DI UI to suggest class
+names, method names, and method parameters when creating probes.
+
+### Enabling the Symbol Database
+
+Symbol Database upload is enabled by default when Dynamic Instrumentation
+is enabled. No additional configuration is required. It activates via
+Remote Configuration when the DI UI is opened for your service.
+
+To explicitly disable it:
+
+    export DD_SYMBOL_DATABASE_UPLOAD_ENABLED=false
+
 ## Rate Limiting and Performance
 
 ### Default Rate Limits

docs/GettingStarted.md

@@ -2946,6 +2946,42 @@ Alternatively, set error tracking parameters inside a `Datadog.configure` block
 - Relative path: For example, for a program running in the `app` directory, use `./lib/mypackage/main.rb` to instrument the `main.rb` file, or`./lib/mypackage/` to instrument every Ruby file in that folder.
 
 
+### Dynamic Instrumentation
+
+Dynamic Instrumentation allows you to add log probes, metric probes, and span probes to running applications without restarts or redeployments. For setup instructions, see [Dynamic Instrumentation](DynamicInstrumentation.md).
+
+#### Configuration
+
+| Environment variable | Type | Description | Default |
+|---|---|---|---|
+| `DD_DYNAMIC_INSTRUMENTATION_ENABLED` | `Boolean` | Enable or disable Dynamic Instrumentation. | `false` |
+| `DD_DYNAMIC_INSTRUMENTATION_REDACTED_IDENTIFIERS` | `Array` | Comma-separated list of variable/key names to redact in addition to the built-in list. Names are normalized (underscores, dashes, `@`, `$` removed) and matched case-insensitively. | `[]` |
+| `DD_DYNAMIC_INSTRUMENTATION_REDACTION_EXCLUDED_IDENTIFIERS` | `Array` | Comma-separated list of variable/key names to exclude from the built-in redaction list, allowing their values to be captured. | `[]` |
+| `DD_DYNAMIC_INSTRUMENTATION_REDACTED_TYPES` | `Array` | Comma-separated list of class names whose values will be redacted. Suffix with `*` for wildcard matching (e.g. `Foo*` redacts `Foo`, `FooBar`, `Foo::Bar`). | `[]` |
+
+Alternatively, set DI parameters inside a `Datadog.configure` block:
+
+| Setting | Type | Description | Default |
+|---|---|---|---|
+| `c.dynamic_instrumentation.enabled` | `Boolean` | Enable or disable Dynamic Instrumentation. | `false` |
+| `c.dynamic_instrumentation.redacted_identifiers` | `Array` | Variable/key names to redact in addition to the built-in list. | `[]` |
+| `c.dynamic_instrumentation.redaction_excluded_identifiers` | `Array` | Variable/key names to exclude from the built-in redaction list. | `[]` |
+| `c.dynamic_instrumentation.redacted_type_names` | `Array` | Class names whose values will be redacted. Suffix with `*` for wildcard. | `[]` |
+
+#### Symbol Database
+
+When Dynamic Instrumentation is enabled, the tracer can extract and upload symbol information (class names, method signatures, parameter names) from your application to enable auto-completion in the DI UI. Symbol Database upload is enabled by default and activates automatically via Remote Configuration.
+
+| Environment variable | Type | Description | Default |
+|---|---|---|---|
+| `DD_SYMBOL_DATABASE_UPLOAD_ENABLED` | `Boolean` | Enable or disable symbol database upload. | `true` |
+
+| Setting | Type | Description | Default |
+|---|---|---|---|
+| `c.symbol_database.enabled` | `Boolean` | Enable or disable symbol database upload. | `true` |
+
+Symbol Database requires MRI Ruby 2.6+ and Remote Configuration (enabled by default). For details on what is extracted, which code is included, and behavior differences across Ruby versions, see [Dynamic Instrumentation — Symbol Database](DynamicInstrumentation.md#symbol-database).
+
 ## Known issues and suggested configurations
 
 ### Payload too large

lib/datadog/core/configuration/components.rb

@@ -19,6 +19,8 @@
 require_relative '../../appsec/component'
 require_relative '../../ai_guard/component'
 require_relative '../../di/component'
+require_relative '../../symbol_database'
+require_relative '../../symbol_database/component'
 require_relative '../../open_feature/component'
 require_relative '../../error_tracking/component'
 require_relative '../crashtracking/component'
@@ -120,6 +122,7 @@ def build_data_streams(settings, agent_settings, logger, agent_info)
           :ai_guard,
           :agent_info,
           :data_streams,
+          :symbol_database,
           :open_feature
 
         def initialize(settings)
@@ -171,6 +174,7 @@ def initialize(settings)
           @ai_guard = Datadog::AIGuard::Component.build(settings, logger: @logger, telemetry: telemetry)
           @open_feature = OpenFeature::Component.build(settings, agent_settings, logger: @logger, telemetry: telemetry)
           @dynamic_instrumentation = Datadog::DI::Component.build(settings, agent_settings, @logger, telemetry: telemetry)
+          @symbol_database = Datadog::SymbolDatabase::Component.build(settings, agent_settings, @logger)
           @error_tracking = Datadog::ErrorTracking::Component.build(settings, @tracer, @logger)
           @data_streams = self.class.build_data_streams(settings, agent_settings, @logger, @agent_info)
           @environment_logger_extra[:dynamic_instrumentation_enabled] = !!@dynamic_instrumentation
@@ -241,6 +245,9 @@ def shutdown!(replacement = nil)
           # Shutdown DI after remote, since remote config triggers DI operations.
           dynamic_instrumentation&.shutdown!
 
+          # Shutdown Symbol Database
+          symbol_database&.shutdown!
+
           # Shutdown OpenFeature component
           open_feature&.shutdown!
 

lib/datadog/core/configuration/supported_configurations.rb

@@ -103,6 +103,8 @@ module Configuration
           "DD_RUNTIME_METRICS_RUNTIME_ID_ENABLED",
           "DD_SERVICE",
           "DD_SITE",
+          "DD_INTERNAL_FORCE_SYMBOL_DATABASE_UPLOAD",
+          "DD_SYMBOL_DATABASE_UPLOAD_ENABLED",
           "DD_SPAN_SAMPLING_RULES",
           "DD_SPAN_SAMPLING_RULES_FILE",
           "DD_SYMBOL_DATABASE_UPLOAD_ENABLED",

lib/datadog/core/remote/client/capabilities.rb

@@ -3,6 +3,8 @@
 require_relative '../../utils/base64'
 require_relative '../../../appsec/remote'
 require_relative '../../../tracing/remote'
+require_relative '../../../di/remote'
+require_relative '../../../symbol_database/remote'
 require_relative '../../../open_feature/remote'
 
 module Datadog
@@ -37,6 +39,13 @@ def register(settings)
               register_capabilities(Datadog::DI::Remote.capabilities)
               register_products(Datadog::DI::Remote.products)
               register_receivers(Datadog::DI::Remote.receivers(@telemetry))
+
+              # Symbol Database
+              if settings.respond_to?(:symbol_database) && settings.symbol_database.enabled
+                register_capabilities(Datadog::SymbolDatabase::Remote.capabilities)
+                register_products(Datadog::SymbolDatabase::Remote.products)
+                register_receivers(Datadog::SymbolDatabase::Remote.receivers(@telemetry))
+              end
             end
 
             if settings.respond_to?(:open_feature) && settings.open_feature.enabled