tests: kdoc: handle buggy "line number only" warnings

Some versions of kernel-doc currently report certain warnings with only
their line number. This breaks the logic of the kdoc test, as such lines do
not pass the expected format defined by the KdocWarning class.

Such warnings appear to indicate duplicate section headers in a kernel doc
entry, but the text of the warning is not properly displayed.

As a result, the kdoc test produces false positive failures for patches
which touch files that have such warnings. The line contents are compared
without ignoring the line numbers. This produces output failures:

   Warnings before: 7 after: 7 (add: 2 del: 2)

   Warnings removed:
   Warning: 174
   Warning: 184
   Per-file breakdown:
        2 None

   New warnings added:
   Warning: 180
   Warning: 196
   Per-file breakdown:
        2 None

Since the warnings are compared directly, the changing line numbers are
detected as differences.

This is a bug in the kernel-doc script, and a proper fix requires updating
the script to stop producing such warnings. In the mean time, the NIPA
tests continue to fail. To fix that, use a separate regular expression to
identify lines of the form 'Warning: <number>'. Set the content of such
warnings to None.

This will make all such warnings compare as identical in the existing
algorithm, and thus the changing line numbers will no longer cause such a
false positive.

To avoid introducing further regressions, extend the check to count the
total number of unparsed warnings. If it increases, then also fail the kdoc
test. This way, we will correctly fail when we see more empty warnings
after applying the current patch.

This also should help catch issues for other types of unrecognized warnings
in the future. Note that all warnings still get compared in the usual way,
where fully unrecognized warnings will be compare the lines as-is without
eliding the line number comparison. This just adds an additional safety
guard in addition to handling the empty warning case.

Signed-off-by: Jacob Keller <[email protected]>

Author: jacob-keller

tests/patch/kdoc/test.py

@@ -30,7 +30,7 @@ class KdocWarning:
     # Note: *not* part of comparison, or hash!
     line : Optional[int] = dataclasses.field(repr=True, compare=False)
     # The content of the warning (excluding kind, file, line)
-    content : str = dataclasses.field(repr=True, compare=True)
+    content : Optional[str] = dataclasses.field(repr=True, compare=True)
     parsed : bool = dataclasses.field(repr=True, compare=True)
 
     @classmethod
@@ -65,6 +65,29 @@ def from_text(self, line, extra=None):
             return KdocWarning(message, kind=kind, file=file, line=line,
                                content=content, parsed=True)
 
+        # Check for line-number only warnings produced by certain buggy
+        # versions of kernel-doc. These will get treated as unparsed but
+        # all such warnings will compare identically to each other.
+        line_only_parser = re.compile(
+            r"""
+            ^                         # Start of string
+            (?P<kind>warning|error):  # Severity
+            \s+                       # Spacing
+            (?P<line>[0-9]+)          # Line number
+            $                         # End of string
+            """,
+            re.VERBOSE | re.IGNORECASE)
+
+        m = line_only_parser.match(line)
+        if m:
+            kind = m['kind']
+            line = m['line']
+
+            return KdocWarning(message, kind=kind, file=None, line=line,
+                               content=None, parsed=False)
+
+        # The warning text didn't match a known format. Such warnings will be
+        # counted to ensure that we don't increase this count over time.
         return KdocWarning(message, kind='Unknown', file=None, line=None,
                            content=line, parsed=False)
 
@@ -188,6 +211,10 @@ def kdoc(tree, patch, _result_dir) -> Tuple[int, str, str]:
     new_count = len(new_warnings)
     rm_count = len(rm_warnings)
 
+    # Count the number of warnings which we failed to parse
+    current_unparsed = len([x for x in current_warnings if not x.parsed])
+    incumbent_unparsed = len([x for x in incumbent_warnings if not x.parsed])
+
     desc = f'Warnings before: {incumbent_count} after: {current_count}'
     brac = []
     if new_count:
@@ -220,4 +247,8 @@ def kdoc(tree, patch, _result_dir) -> Tuple[int, str, str]:
         for f, count in file_breakdown.items():
             log += [f'{count:6} {f}']
 
+    if current_unparsed > incumbent_unparsed:
+        ret = 1
+        log += ["", f'Number of parse failures increased from {incumbent_unparsed} to {current_unparsed}.']
+
     return ret, desc, "\n".join(log)