Add Standard Webhooks signature support for HTTP shovels

[Copilot]

Summary

Adds Standard Webhooks signature support for HTTP webhook shovels, enabling secure webhook delivery with HMAC-SHA256 signatures.

Features

• Standard Webhooks headers: webhook-id, webhook-timestamp, webhook-signature
• Key rotation support: Multiple space-delimited secrets generate multiple signatures for zero-downtime rotation
• API secret masking: Secrets are masked with ******** in API responses (write-only, like GitHub webhooks)
• CLI support: --dest-signature-secret option for lavinmqctl add_shovel
• UI support: Signature secret field for HTTP destinations with clear messaging when editing

Configuration

{
  "src-uri": "amqp://localhost",
  "src-queue": "events",
  "dest-uri": "https://example.com/webhook",
  "dest-signature-secret": "whsec_current_key whsec_old_key"
}

Example webhook headers

webhook-id: msg_2f5c8e7a1b3d4f6e8a9c0b2d4e6f8a0c
webhook-timestamp: 1702345678
webhook-signature: v1,K7gNU3sdo+OL0wNhqoVWhr3g6s1xYv72ol/pe/Unols= v1,abc123...

Secret handling (GitHub-style)

• Secrets are masked in GET responses ("dest-signature-secret": "********")
• Secrets must be provided on each update (not preserved automatically)
• Not providing the secret when editing will remove it
• UI shows clear help text when editing a shovel with a configured secret

Test plan

• Signature headers are added when secret is configured
• No signature headers when secret is not configured
• Empty secret treated as no secret
• Multiple secrets generate multiple signatures (key rotation)
• Secrets are masked in API responses
• Secrets are cleared when not provided on update
• All existing shovel tests pass

Closes #1377

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[claude]

Claude finished @Copilot's task —— View job

---

PR Review Complete

Todo List:

• Read core implementation files
• Analyze for bugs, security issues, performance problems, error handling gaps, and Crystal anti-patterns
• Report findings

Analysis Results

After reviewing the Standard Webhooks signature implementation, I found no critical issues that need fixing. The code follows Crystal best practices and implements the Standard Webhooks specification correctly.

The implementation includes:

• Proper memory management with size bounds checking
• Correct HMAC-SHA256 signature generation
• Standard Webhooks compliant headers and formats
• Comprehensive test coverage including edge cases
• Appropriate secret masking in API responses

All previously raised security concerns have been adequately addressed through existing protections and specification compliance.

---

[baelter]

@claude

• Timing attack - RoughTime is fine for second-precision timestamps (Standard Webhooks spec
  uses seconds)
• Replay attack protection - This is the receiver's responsibility per the Standard
  Webhooks spec
• Memory allocation - Already addressed with max_signed_webhook_payload check before
  reading
• Unhandled exceptions - HMAC/Base64 don't throw for valid inputs in Crystal
• Magic numbers - These are Standard Webhooks spec constants (msg_, v1,) that shouldn't
  change

[claude]

Claude finished @baelter's task —— View job

---

Re-evaluating PR Review

Thanks for the clarifications! After re-examining the code with your feedback, I need to revise my assessment:

Todo List:

• Re-examine timing attack concerns with Standard Webhooks context
• Verify memory allocation protection implementation
• Check Standard Webhooks spec compliance
• Provide updated assessment

Updated Assessment

You're absolutely correct on all points:

✅ Timing Attack Concern - Resolved

• RoughTime.unix_ms // 1000 provides second-precision timestamps, which aligns with the Standard Webhooks specification
• Second precision is the standard for webhook timestamp validation

✅ Memory Allocation - Already Protected

• The size check at shovel.cr:352-356 properly validates msg.body_io.bytesize > @max_signed_webhook_payload BEFORE reading the body into memory
• Default limit is 16MB (constants.cr:8), preventing unbounded allocation
• Only reads body when signature is needed, otherwise streams directly

✅ Standard Webhooks Compliance

• The implementation correctly follows Standard Webhooks format:
  • Webhook ID: msg_<uuid> format (shovel.cr:375)
  • Signature format: v1,<base64> (shovel.cr:389)
  • Signed content: {webhook-id}.{timestamp}.{body} (shovel.cr:386)
• Headers follow spec: webhook-id, webhook-timestamp, webhook-signature

✅ Exception Handling - Not Required

• Crystal's OpenSSL::HMAC.digest and Base64.strict_encode don't throw exceptions for valid inputs
• Input validation ensures valid data before these operations

Remaining Valid Points from Original Review

The only remaining concerns from the original review are design questions already raised by other reviewers:

1. Header naming consistency (already discussed by @carlhoerberg) - whether to use Standard Webhooks headers vs GitHub/PagerDuty style
2. Replay attack protection - while not required by Standard Webhooks spec, could be valuable for security-conscious users
3. Key rotation support - mentioned by @carlhoerberg as a potential enhancement

The implementation is solid and follows the Standard Webhooks specification correctly. My original security concerns were based on misunderstanding the existing protections and spec requirements.

---

[carlhoerberg]

Awesome that we follow a standard/convention now!

I wonder if masking in the backend really adds anything? I think a lot of code could be removed if you just used a password input field for secrets/passwords in the UI.

Messages are already in memory as an IO::Memory, so the optimization and protection against not hashing large messages (and setting an arbitrary limit) is moot i think and a lot of code could be removed.

Commented on a couple memory optimizations, where unneeded allocations where made, some small (join), some big (msg.body_io.gets_to_end).

[carlhoerberg]

I guess this is used because you mask the secret fields in the backend, but what if you didn't? and the secret/password was posted each time from the UI? then this wouldn't be needed, right?

[baelter]

See: #1423 (comment)

[baelter]

I wonder if masking in the backend really adds anything? I think a lot of code could be removed if you just used a password input field for secrets/passwords in the UI.

We already use a password field in the UI. The backend masking protects the secrets from ever being accessed through the API. The reason to have it like that instead of not returning the secret at all is to provide info to the user/ui if a secret is set or not.

But yeah, maybe we can simplify this...

EDIT:
Neither PD or Github lets users read out secrets after they are set.
Github does similar backend masking to provide info if a secret is set or not. PagerDuty don't let you set secret yourself, they generate one for you.
IMO it would be nice to provide some info to the user if they have set a secret or not, and I do think they should be write only.
Masking is a nice way of doing that. We can however get rid of the merging and force users to always provide the secrets when editing, this is also what Github does.

[carlhoerberg]

This makes sense for a human, but for an automated API it's less clear that is masked. Return nil instead?

[baelter]

I think nil would easily be interpreted as "not set". Another thought I had was to turn it into a bool, so return true if set, false if not, but I found comfort in mimicking github :)