reporter: add unit tests for the usage report payload and missing licenses

Pin down the JSON that the management server POSTs to usage.report.uri so
any change to the wire format has to be a deliberate one.

UsageReporterTest mocks the seven DAOs the report is built from and drives
the real report builders through the real AtomicGsonAdapter, comparing the
result against a checked-in fixture, usage-report-expected.json. Keys are
sorted on both sides before comparing: AtomicLongMap is backed by a
ConcurrentHashMap and the report itself by a HashMap, so key order on the
wire is not deterministic and must not be part of the contract. Sorting
also makes a mismatch print a readable diff.

The tests document three things that are not obvious from reading the code:

 - provisioning_type keys are lowercase ("thin"/"fat"). Storage.Provisio-
   ningType overrides toString() and the adapter keys on String.valueOf(),
   so the payload does not carry the enum constant names that Gson would
   emit by default.
 - boolean counters reach the wire as the string keys "true" and "false",
   used by ha_enabled, dynamically_scalable, compute_only and
   use_local_storage.
 - a report from an empty install still carries all eight sections with
   empty counter objects, and avg_disk_size falls back to 0 rather than
   dividing by zero.

AtomicGsonAdapterTest covers the adapter on its own: null and empty maps,
counts as numbers, boolean and enum keys, and that read() consumes a null.

Also add the ASF license headers that apache-rat flags on reporter/README.md
and reporter/requirements.txt.

Author: wido

reporter/README.md

@@ -1,3 +1,22 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
 # CloudStack Usage Reporter
 
 This directory contains the server-side webservice for the Apache CloudStack usage reporting feature. When enabled, CloudStack management servers periodically send an anonymized report to the Apache CloudStack project. This data helps the community understand how CloudStack is deployed and used in the field.

reporter/requirements.txt

@@ -1 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
 flask>=2.2,<4

server/src/test/java/org/apache/cloudstack/report/AtomicGsonAdapterTest.java

@@ -0,0 +1,129 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.report;
+
+import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+import com.cloud.storage.Storage;
+import com.google.common.util.concurrent.AtomicLongMap;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import com.google.gson.stream.JsonReader;
+import com.google.gson.stream.JsonWriter;
+
+/**
+ * The adapter that turns every counter in the usage report into a JSON object.
+ * Its handling of keys decides most of the payload's wire format.
+ */
+public class AtomicGsonAdapterTest {
+
+    private final AtomicGsonAdapter adapter = new AtomicGsonAdapter();
+
+    private String write(AtomicLongMap<Object> value) throws IOException {
+        StringWriter out = new StringWriter();
+        try (JsonWriter writer = new JsonWriter(out)) {
+            writer.setSerializeNulls(true);
+            adapter.write(writer, value);
+        }
+        return out.toString();
+    }
+
+    @Test
+    public void testNullMapIsWrittenAsJsonNull() throws IOException {
+        Assert.assertEquals("null", write(null));
+    }
+
+    @Test
+    public void testEmptyMapIsWrittenAsEmptyObject() throws IOException {
+        Assert.assertEquals("{}", write(AtomicLongMap.create()));
+    }
+
+    /**
+     * AtomicLongMap.asMap() is backed by a ConcurrentHashMap, so the order of keys
+     * within a counter object is not deterministic. The receiving service must treat
+     * these as unordered objects; only the key/value pairs are part of the contract.
+     */
+    @Test
+    public void testCountsAreWrittenAsNumbers() throws IOException {
+        AtomicLongMap<Object> counter = AtomicLongMap.create();
+        counter.getAndIncrement("KVM");
+        counter.getAndIncrement("KVM");
+        counter.getAndIncrement("VMware");
+
+        JsonObject json = JsonParser.parseString(write(counter)).getAsJsonObject();
+        Assert.assertEquals(2, json.size());
+        Assert.assertEquals(2, json.get("KVM").getAsLong());
+        Assert.assertEquals(1, json.get("VMware").getAsLong());
+    }
+
+    /**
+     * Boolean keys reach the wire as the strings "true" and "false"; the report uses
+     * these for ha_enabled, dynamically_scalable, compute_only and use_local_storage.
+     */
+    @Test
+    public void testBooleanKeysBecomeStringKeys() throws IOException {
+        AtomicLongMap<Object> counter = AtomicLongMap.create();
+        counter.getAndIncrement(Boolean.TRUE);
+        counter.getAndIncrement(Boolean.FALSE);
+        counter.getAndIncrement(Boolean.FALSE);
+
+        String json = write(counter);
+        Assert.assertTrue(json, json.contains("\"true\":1"));
+        Assert.assertTrue(json, json.contains("\"false\":2"));
+    }
+
+    /**
+     * Keys go through String.valueOf(), i.e. toString(), so an enum that overrides
+     * toString() is serialized by that override and not by its constant name.
+     */
+    @Test
+    public void testEnumKeysUseToStringNotConstantName() throws IOException {
+        AtomicLongMap<Object> counter = AtomicLongMap.create();
+        counter.getAndIncrement(Storage.ProvisioningType.THIN);
+
+        Assert.assertEquals("{\"thin\":1}", write(counter));
+    }
+
+    @Test
+    public void testNullKeysCannotReachThePayload() throws IOException {
+        AtomicLongMap<Object> counter = AtomicLongMap.create();
+        counter.getAndIncrement("KVM");
+
+        // AtomicLongMap rejects null keys outright, so a "null" key can only ever
+        // appear if a caller stringifies before counting. Guard the assumption.
+        try {
+            counter.getAndIncrement(null);
+            Assert.fail("AtomicLongMap unexpectedly accepted a null key");
+        } catch (NullPointerException expected) {
+            // expected
+        }
+
+        Assert.assertEquals("{\"KVM\":1}", write(counter));
+    }
+
+    @Test
+    public void testReadConsumesNullAndReturnsNull() throws IOException {
+        try (JsonReader reader = new JsonReader(new StringReader("null"))) {
+            Assert.assertNull(adapter.read(reader));
+        }
+    }
+}