Skip to content

feat: add statsd counters for signup funnel (account created + email verified, by device)#13190

Open
Sadashii wants to merge 1 commit into
internetarchive:masterfrom
Sadashii:4369/feature/statsd-email-verification
Open

feat: add statsd counters for signup funnel (account created + email verified, by device)#13190
Sadashii wants to merge 1 commit into
internetarchive:masterfrom
Sadashii:4369/feature/statsd-email-verification

Conversation

@Sadashii

@Sadashii Sadashii commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #

This PR adds statsd counters to measure signup funnel dropoff — specifically, how many users who create an account actually complete email verification. We also track device type (mobile vs desktop) at both steps to test a hypothesis that the link-only email verification flow causes disproportionate mobile dropoff.

Background / Hunch: When a user signs up on mobile and hasn't logged into their email client on that device, clicking a verification link opens a new browser session where they may not have context, or worse, they may have to switch apps entirely. This friction likely causes a meaningful portion of signups to bounce before verifying. Adding device-aware counters lets us validate (or disprove) this before investing in an OTP-based alternative verification flow.

Technical

Added to openlibrary/plugins/upstream/account.py:

  • _MOBILE_UA_RE — compiled regex to classify User-Agent as mobile or desktop (no new dependencies)
  • get_device_label() — thin helper that reads HTTP_USER_AGENT from web.ctx.env

Counters (via openlibrary.core.stats.increment):

Counter Fires when
ol.account.created IA account created successfully
ol.account.created.mobile / .desktop same, split by UA
ol.account.email_verified email link clicked and activation succeeds
ol.account.email_verified.mobile / .desktop same, split by UA

In Grafana: email_verified / created gives the overall funnel %. Comparing .mobile vs .desktop sub-rates will show whether mobile users are dropping off disproportionately at the verification step.

Note: previously, admin_range__members in admin/numbers.py counted new accounts daily, but only post-verification (a /type/user record isn't created until first login after activation). There was no counter for attempted signups, so there was no way to compute a funnel rate. These counters close that gap.

Testing

No new behaviour — purely additive observability. Existing tests pass unchanged.

To manually verify locally:

  1. Register a new account at /account/create
  2. Check statsd/Graphite for ol.account.created and ol.account.created.{mobile,desktop}
  3. Click the verification link in the email
  4. Check for ol.account.email_verified and ol.account.email_verified.{mobile,desktop}

Screenshot

N/A — no UI changes.

Stakeholders

@mek

Copilot AI review requested due to automatic review settings July 21, 2026 17:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Track ol.account.created and ol.account.email_verified via statsd so
we can compute the signup->verification funnel rate in Grafana.

Also split each counter by device type (mobile/desktop) derived from
the User-Agent header, so we can see whether mobile users drop off at
a higher rate than desktop users at the verification step.

Counters added:
  ol.account.created           - fires on successful IA account creation
  ol.account.created.mobile    - same, mobile UA
  ol.account.created.desktop   - same, desktop UA
  ol.account.email_verified           - fires on successful email link activation
  ol.account.email_verified.mobile    - same, mobile UA
  ol.account.email_verified.desktop   - same, desktop UA
@Sadashii
Sadashii force-pushed the 4369/feature/statsd-email-verification branch from 24b5792 to 6210717 Compare July 21, 2026 17:12
@mekarpeles

Copy link
Copy Markdown
Member

As noted in our 1:1 conversation, statsd + devices might be challenging

It might make more sense to use matomo (which already has device type)

The question becomes much simpler; how can we trigger a matomo event on the page or redirect triggered by the activate endpoint

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants