MIssing bad captures (est. 2025-01-11)

[Mr0grog]

This is a running list of example bad captures that we are failing to flag/ignore.

• The left-hand side of this comparison is a probably temporary 500 error from Oracle APEX: https://monitoring.envirodatagov.org/page/d1620a7d-557c-4517-89f7-53577d5d4e34/be872cfc-d937-4eb0-a283-d4dc2beb8d73..bb2b117f-cefb-4a61-92a1-a69fde42bda2

  

  Unfortunately there’s not much useful there to classify as different from a normal/legit 500 error. We may need to look contextually at versions before/after it to see if it’s temporary.

  Here’s another one from the same server: https://monitoring.envirodatagov.org/page/888408ab-b717-4b05-b6e3-4253c1e66833/2c3fd690-32de-481a-ad15-6420275c90c0..9cd58eba-3e2b-43b5-91df-92a489e7109b

[Mr0grog]

Unfortunately there’s not much useful there to classify as different from a normal/legit 500 error. We may need to look contextually at versions before/after it to see if it’s temporary.

Ironically this is a particular case that the old method (which tried to skip over any errors for the baseline version) handled better. Maybe we could bring that back in a slightly modified form:

• We currently look up to 3 versions/30 days back when the baseline version would otherwise be a bad/questionable capture
• We could additionally/alternatively look up to, say, 5 days back (something much shorter than 30) when the baseline would otherwise be an error (but otherwise looks like a good/valid capture).

But the contextuality thing is good, too. To iterate through versions and their surrounding context, Itertools has a nice pairwise iterator, and we probably want something like that but with N items (3 or 5) instead of 2. Toolz has sliding_window, which is close to ideal for this (but we probably want to prefix the list with N/2 items, e.g. in the sequence (1,2,3,4) we should start with (None, 1, 2) instead of (1, 2, 3)). See also the official Itertools recipes in the Python docs for a nice example of a simple implementation.

[Mr0grog]

I think we are catching these challenge interstitials from AWS WAF/CloudFront as bad versions, but we should also probably be marking their effective status as 429 or 503: (right side in this view) https://monitoring.envirodatagov.org/page/d439db77-448a-4a00-a741-1f970c547450/9533551f-ef28-4d0a-9400-ab9aa28142d1..30619214-1677-4b4c-b244-7bace9bd127e