WIP: Add possibily to group stat results by field names

CHANGELOG.md

@@ -4,7 +4,7 @@
 
 ### Added
 
-- nothing added
+- Add possibily to group stat results by field names
 
 ### Changed
 

concrete_datastore/api/v1/views.py

@@ -6,6 +6,7 @@
 import sys
 import re
 import os
+from collections import Counter
 from urllib.parse import urljoin, unquote, urlparse, urlunparse
 from importlib import import_module
 from itertools import chain
@@ -1315,6 +1316,9 @@ def get_stats(self, request, timestamp_start=None, timestamp_end=None):
 
         dict_pages = dict()
 
+        url = remove_query_param(url, 'group_by')
+        url = remove_query_param(url, 'combine')
+
         for page_number in range(1, _num_pages + 1):
             if page_number == 1:
                 dict_pages['page{}'.format(page_number)] = unquote(
@@ -1335,6 +1339,44 @@ def get_stats(self, request, timestamp_start=None, timestamp_end=None):
             ],
             'page_urls': dict_pages,
         }
+        group_by_fields = request.GET.get('group_by', None)
+        combine_results = request.GET.get('combine', 'false').lower()
+        if combine_results not in ('true', 'false'):
+            return Response(
+                data={
+                    'message': f'"combine" value ("{combine_results}") is not valid. Should be either "true" or "false"',
+                    '_errors': ['INVALID_QUERY'],
+                },
+                status=HTTP_400_BAD_REQUEST,
+            )
+        if group_by_fields is not None:
+            group_by_fields = group_by_fields.split(',')
+            if any(map(lambda x: x not in self.fields, group_by_fields)):
+                return Response(
+                    data={
+                        'message': 'lookup query is not a valid field',
+                        '_errors': ['INVALID_QUERY'],
+                    },
+                    status=HTTP_400_BAD_REQUEST,
+                )
+
+            grouped_data = {}
+            if combine_results == 'false':
+                for field_name in group_by_fields:
+                    grouped_data[field_name] = dict(
+                        Counter(queryset.values_list(field_name, flat=True))
+                    )
+            else:
+                grouped_data[','.join(group_by_fields)] = dict(
+                    Counter(
+                        map(
+                            lambda x: ','.join(map(str, x)),
+                            queryset.values_list(*group_by_fields),
+                        )
+                    )
+                )
+            data.update({"results": grouped_data})
+
         return Response(data)
 
     def _get_queryset_filtered_since_timestamp(

docs/api-routes.md

@@ -14,7 +14,7 @@ For each model, Concrete Datastore exposes two routes accepting different method
 
 #### List all instances of model MyModel
 
-A `GET` on the root url of the model MyModel will retrieve all instances of this model.
+A `GET` on the root url of the model MyModel will retrieve all instances of this model. This endpoint accepts filtering (see [filters section](filters.md) for more information)
 
 - **Method**: `GET`
 
@@ -120,7 +120,7 @@ curl \
 
 #### Update a specific instance of model MyModel by its UID
 
-#### Update some of the fields with `PATCH`
+##### Update some of the fields with `PATCH`
 
 A `PATCH` on the url of a given instance of model MyModel will update the fields of this given instance.
 
@@ -142,7 +142,7 @@ curl \
 
 **Response**: with status code HTTP `200 (OK)`, the response body is a JSON containing all the fields of the given instance, updated.
 
-#### Update all the fields with `PUT`
+##### Update all the fields with `PUT`
 
 A `PUT` on the url of a given instance of model MyModel will update the fields of this given instance.
 
@@ -187,6 +187,160 @@ curl \
 
 This operation could fail. If the instance is related to a protected instance, it cannot be deleted. In this case, the HTTP status code is `412 (PRECONDITION FAILED)` with the error code `"PROTECTED_RELATION"` in the response.
 
+#### Get stats on model MyModel
+
+A `GET` on the endpoint `https://<webapp>/api/v1.1/my-model/stats/` allows to get stats on the given model. This endpoint accepts filtering (see [filters section](filters.md) for more information)
+
+*N.B* Some query parameters in the [filters section](filters.md) are not needed for this endpoint (hence ignored) such as `c_resp_page_size`, `page` and `c_resp_nested`. As for the `timestamp_start` and `timestamp_end`, they are used differently for this endpoint. (Please refer to the [option parameters section](#OptionalParameters) to see more information about the `timestamp_start` and `timestamp_end`, as well as the itroduction of two other query parameters)
+
+- **Method**: `GET`
+
+- **ENDPOINT**: `https://<webapp>/api/v1.1/my-model/stats/`
+
+- **Example**: 
+
+**Request**
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/"
+```
+
+**Response**:with status code HTTP `200 (OK)`, the response body is a JSON containing stats on the instances of the model such as:
+
+- `objects_count`: the number of instances of this model.
+
+- `num_total_pages`: the number of total pages.
+
+- `page_urls`: An object containing all the pages.
+
+Example of a response:
+
+```json
+{
+  "objects_count": 125,
+  "num_total_pages": 2,
+  "max_allowed_objects_per_page": 100,
+  "timestamp_start": 0.0,
+  "timestamp_end": 701184650.0,
+  "page_urls": {
+    "page1": "https://<webapp>/api/v1.1/my-model/",
+    "page2": "https://<webapp>/api/v1.1/my-model/?page=2"
+  }
+}
+```
+
+<a name="OptionalParameters"></a>##### Optional parameters for the stats endpoint
+
+- **Path parameters:** The stats endpoint accepts two path parameters: `timestamp_start` and `timestamp_end` to get stats between two timestamps. You can either specify only a `timestamp_start` (the end will be the current timestamp) or both start and end:
+
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/timestamp_start=123456789.123"
+```
+
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/timestamp_start=123456789.0-timestamp_end:123456832.0"
+```
+
+- **Query parameters:** Two additional query parameters are accepted for this endpoint: `group_by` and `combine`:
+
+  + `group_by`: Accepts one or more comma separated field names. If the value of this param contains an invalid field name, the API responds with a `400 BAD REQUEST` Used to group the stats by the field values. The results will appear as an object within a `results` field. Example:
+
+**Request:**
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/?group_by=status"
+```
+
+**Response:**
+```json
+{
+  "objects_count": 125,
+  "num_total_pages": 2,
+  "max_allowed_objects_per_page": 100,
+  "timestamp_start": 0.0,
+  "timestamp_end": 701184650.0,
+  "page_urls": {
+    "page1": "https://<webapp>/api/v1.1/my-model/",
+    "page2": "https://<webapp>/api/v1.1/my-model/?page=2"
+  },
+  "results":{
+    "status":{
+      "COMPLETED": 113,
+      "FAILED": 12
+    }
+  }
+}
+```
+
+  + `combine`: Accepts `true` or `false` (default to `false`). if any other value is given, the API responds with a `400 BAD REQUEST`. This param is used when more than one field name is given in the `group_by` parameter, otherwise it is ignored. If `combine` is `true`, the results will be a combination of the possible values of the two fields. Otherwise the results will be given separately for each field
+
+**Request with combine to false:**
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/?group_by=status,archived"
+```
+
+**Response:**
+```json
+{
+  "objects_count": 125,
+  "num_total_pages": 2,
+  "max_allowed_objects_per_page": 100,
+  "timestamp_start": 0.0,
+  "timestamp_end": 701184650.0,
+  "page_urls": {
+    "page1": "https://<webapp>/api/v1.1/my-model/",
+    "page2": "https://<webapp>/api/v1.1/my-model/?page=2"
+  },
+  "results":{
+    "status":{
+      "COMPLETED": 113,
+      "FAILED": 12
+    },
+    "archived":{
+      "true": 53,
+      "false": 72
+    }
+  }
+}
+```
+
+**Request with combine to true:**
+```shell
+curl \
+  -H "Authorization: Token <auth_token>" \
+  "https://<webapp>/api/v1.1/my-model/stats/?group_by=status,archived&combine=true"
+```
+
+**Response:**
+```json
+{
+  "objects_count": 125,
+  "num_total_pages": 2,
+  "max_allowed_objects_per_page": 100,
+  "timestamp_start": 0.0,
+  "timestamp_end": 701184650.0,
+  "page_urls": {
+    "page1": "https://<webapp>/api/v1.1/my-model/",
+    "page2": "https://<webapp>/api/v1.1/my-model/?page=2"
+  },
+  "results":{
+    "status,archived":{
+      "COMPLETED,true": 45,
+      "COMPLETED,false": 68,
+      "FAILED,true": 8
+      "FAILED,false": 4
+    }
+  }
+}
+```
 
 ### Specific API endpoints
 

docs/filters.md

@@ -22,6 +22,7 @@ example:
 example: `?creation_date__range=2018-01-01,2018-12-31` returns all objects with creation date is between 1st Jan 2018 and 31st Dec 2018
 
 - `c_resp_page_size`: The API also features pagination by the use of the query parameter `c_resp_page_size` that takes an integer representing the number of results per page that sould be returned
+- `page`: If this query param is specified, it returns the results of requested page. Returns `404 NOT FOUND` is the page is not found
 - `c_resp_nested`: If there are relation between objects, by default the API shows the relation completely, it is nested.
 Example: