Merge branch 'gsoc25' into issue/show-mass-upgrade-progress

Author: nemesifier

docs/index.rst

@@ -38,6 +38,7 @@ within the OpenWISP architecture.
 
     ./user/intro.rst
     ./user/quickstart.rst
+    ./user/upgrade-status.rst
     ./user/automatic-device-firmware-detection.rst
     ./user/custom-firmware-upgrader.rst
     ./user/rest-api.rst

docs/user/rest-api.rst

@@ -299,6 +299,18 @@ Get Upgrade Operation Details
 
     GET /api/v1/firmware-upgrader/upgrade-operation/{id}
 
+Cancel Upgrade Operation
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: text
+
+    POST /api/v1/firmware-upgrader/upgrade-operation/{id}/cancel/
+
+.. note::
+
+    This endpoint may return a 409 status code if the operation cannot be
+    canceled.
+
 List Device Upgrade Operations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/user/upgrade-status.rst

@@ -0,0 +1,162 @@
+Upgrade Status Reference
+========================
+
+.. contents:: **Table of contents**:
+    :depth: 2
+    :local:
+
+Overview
+--------
+
+OpenWISP Firmware Upgrader tracks the progress of firmware upgrade
+operations through different status values. Understanding these statuses
+is essential for monitoring upgrade operations and troubleshooting issues.
+
+In Progress
+~~~~~~~~~~~
+
+**Status**: ``in-progress``
+
+**Description**: The upgrade operation is currently running. This includes
+all phases of the upgrade process: device connection, firmware validation,
+file upload, and firmware flashing.
+
+**What happens during this status:**
+
+- Device identity verification
+- Firmware image validation
+- Some non-critical services may be stopped to free up memory, if needed
+- Image upload to the device
+- Firmware flashing process
+
+**User Actions**: Users can cancel upgrade operations that are in
+progress, but only before the firmware flashing phase begins (typically
+when progress is below 60%).
+
+Success
+~~~~~~~
+
+**Status**: ``success``
+
+**Description**: The firmware upgrade completed successfully. The device
+has been upgraded to the new firmware version and is functioning properly.
+
+**What this means:**
+
+- The firmware was successfully flashed to the device
+- The device rebooted with the new firmware
+- Connectivity was restored after the upgrade
+- All verification checks passed
+
+**Next Steps**: No action required. The upgrade is complete and the device
+is running the new firmware.
+
+Failed
+~~~~~~
+
+**Status**: ``failed``
+
+**Description**: The upgrade operation completed, but the system could not
+reach the device again after the upgrade.
+
+**Common causes:**
+
+- Hardware failures
+- Unexpected system errors
+- The network became unreachable after flashing the new firmware
+
+**Recommended Actions**:
+
+- Check network connectivity
+- Physical inspection and/or serial console debugging
+
+Aborted
+~~~~~~~
+
+**Status**: ``aborted``
+
+**Description**: The upgrade operation was stopped due to pre-requisites
+not being met. The system determined it was unsafe or impossible to
+proceed with the upgrade.
+
+**Common causes:**
+
+- Device UUID mismatch (wrong device targeted)
+- Insufficient memory on the device
+- Invalid or corrupted firmware image
+
+**What happens when aborted:**
+
+- The upgrade stops immediately
+- If services were stopped to free up memory, they are automatically
+  restarted
+- No firmware changes are made to the device
+- Device remains in its original state
+
+**Recommended Actions**:
+
+- Verify the correct device is selected
+- Check firmware image compatibility
+- Ensure device has sufficient memory
+
+Cancelled
+~~~~~~~~~
+
+**Status**: ``cancelled``
+
+**Description**: The upgrade operation was manually stopped by the user
+before completion. This is a deliberate action taken through the admin
+interface or REST API.
+
+Users can cancel upgrades through the admin interface using the "Cancel"
+button that appears next to in-progress operations.
+
+**When cancellation is possible:**
+
+- During the early stages of upgrade (typically before 60% progress)
+- Before the new firmware image is written to the flash memory of the
+  network device
+- While the operation status is still "in-progress"
+
+**What happens when the upgrade operation cancels:**
+
+- The upgrade process stops immediately
+- If services were stopped during the upgrade, they are automatically
+  restarted
+- No firmware changes are made to the device
+- Device remains in its original state
+
+Status Flow
+-----------
+
+The typical flow of upgrade statuses follows this pattern:
+
+.. code-block:: none
+
+    in-progress → success
+               ↓
+               failed/aborted/cancelled
+
+**Typical successful upgrade:**
+
+1. ``in-progress``
+2. ``success``
+
+**Typical problematic upgrade:**
+
+1. ``in-progress`` 3. ``failed``: an unexpected error occurs during
+upgrade 2. **OR** ``aborted``: the system detects pre-condition failure
+and stops safely 4. **OR** ``cancelled``: the user manually stops the
+upgrade
+
+Monitoring Upgrades
+-------------------
+
+**Real-time Progress**: The admin interface provides real-time updates of
+upgrade operations, including progress percentages and detailed logs.
+
+**Upgrade Logs**: Each status change is logged with detailed information
+about what occurred during the upgrade process.
+
+**Batch Operations**: When performing mass upgrades, you can monitor the
+status of individual device upgrades within the batch operation.

openwisp_firmware_upgrader/admin.py

@@ -312,6 +312,7 @@ class BatchUpgradeOperationAdmin(ReadonlyUpgradeOptionsMixin, ReadOnlyAdmin, Bas
         "success_rate",
         "failed_rate",
         "aborted_rate",
+        "cancelled_rate",
         "readonly_upgrade_options",
         "created",
         "modified",
@@ -322,6 +323,7 @@ class BatchUpgradeOperationAdmin(ReadonlyUpgradeOptionsMixin, ReadOnlyAdmin, Bas
         "success_rate",
         "failed_rate",
         "aborted_rate",
+        "cancelled_rate",
         "readonly_upgrade_options",
     ]
     change_form_template = (
@@ -471,6 +473,9 @@ def failed_rate(self, obj):
     def aborted_rate(self, obj):
         return self.__get_rate(obj.aborted_rate)
 
+    def cancelled_rate(self, obj):
+        return self.__get_rate(obj.cancelled_rate)
+
     def __get_rate(self, value):
         if value:
             return f"{value}%"
@@ -480,6 +485,7 @@ def __get_rate(self, value):
     success_rate.short_description = _("success rate")
     failed_rate.short_description = _("failure rate")
     aborted_rate.short_description = _("abortion rate")
+    cancelled_rate.short_description = _("cancellation rate")
 
 
 class DeviceFirmwareForm(forms.ModelForm):

openwisp_firmware_upgrader/api/serializers.py

@@ -115,6 +115,7 @@ class BatchUpgradeOperationSerializer(BatchUpgradeOperationListSerializer):
     success_rate = serializers.IntegerField(read_only=True)
     failed_rate = serializers.IntegerField(read_only=True)
     aborted_rate = serializers.IntegerField(read_only=True)
+    cancelled_rate = serializers.IntegerField(read_only=True)
     upgradeoperations = UpgradeOperationSerializer(
         read_only=True, source="upgradeoperation_set", many=True
     )

openwisp_firmware_upgrader/api/urls.py

@@ -57,6 +57,11 @@
                     views.upgrade_operation_detail,
                     name="api_upgradeoperation_detail",
                 ),
+                path(
+                    "upgrade-operation/<uuid:pk>/cancel/",
+                    views.upgrade_operation_cancel,
+                    name="api_upgradeoperation_cancel",
+                ),
                 path(
                     "device/<uuid:pk>/upgrade-operation/",
                     views.device_upgrade_operation_list,

openwisp_firmware_upgrader/api/views.py

@@ -1,7 +1,12 @@
+import logging
+
 import swapper
 from django.core.exceptions import ValidationError
 from django.http import Http404
+from django.utils.translation import gettext_lazy as _
 from django_filters.rest_framework import DjangoFilterBackend
+from drf_yasg import openapi
+from drf_yasg.utils import swagger_auto_schema
 from rest_framework import filters, generics, pagination, serializers, status
 from rest_framework.exceptions import NotFound, PermissionDenied
 from rest_framework.request import clone_request
@@ -26,6 +31,8 @@
     UpgradeOperationSerializer,
 )
 
+logger = logging.getLogger(__name__)
+
 BatchUpgradeOperation = load_model("BatchUpgradeOperation")
 UpgradeOperation = load_model("UpgradeOperation")
 Build = load_model("Build")
@@ -344,6 +351,75 @@ def get_object_or_none(self):
                 raise
 
 
+class UpgradeOperationCancelView(ProtectedAPIMixin, generics.GenericAPIView):
+    queryset = UpgradeOperation.objects.all()
+    serializer_class = serializers.Serializer
+    lookup_field = "pk"
+    organization_field = "device__organization"
+
+    @swagger_auto_schema(
+        operation_description=_("Cancel an upgrade operation"),
+        operation_summary=_("Cancel upgrade operation"),
+        responses={
+            200: openapi.Response(
+                description=_("Upgrade operation cancelled successfully"),
+                schema=openapi.Schema(
+                    type=openapi.TYPE_OBJECT,
+                    properties={
+                        "message": openapi.Schema(
+                            type=openapi.TYPE_STRING, description=_("Success message")
+                        )
+                    },
+                ),
+            ),
+            409: openapi.Response(
+                description=_("Operation cannot be cancelled"),
+                schema=openapi.Schema(
+                    type=openapi.TYPE_OBJECT,
+                    properties={
+                        "error": openapi.Schema(
+                            type=openapi.TYPE_STRING,
+                            description=_(
+                                "Error message explaining why cancellation is not allowed"
+                            ),
+                        )
+                    },
+                ),
+            ),
+        },
+    )
+    def post(self, request, pk):
+        """Cancel an upgrade operation if conditions are met."""
+        try:
+            operation = self.get_object()
+        except Http404:
+            return self._error_response(
+                "Upgrade operation not found", status.HTTP_404_NOT_FOUND
+            )
+        try:
+            operation.cancel()
+        except ValueError as e:
+            return self._error_response(str(e), status.HTTP_409_CONFLICT)
+        except Exception as e:
+            logger.error(f"Failed to cancel upgrade operation {pk}: {str(e)}")
+            return self._error_response(
+                "Failed to cancel upgrade operation",
+                status.HTTP_500_INTERNAL_SERVER_ERROR,
+            )
+        else:
+            logger.info(
+                f"Upgrade operation {pk} cancelled successfully by user {request.user}"
+            )
+            return Response(
+                {"message": "Upgrade operation cancelled successfully"},
+                status=status.HTTP_200_OK,
+            )
+
+    def _error_response(self, message, status_code):
+        """Helper method to create consistent error responses."""
+        return Response({"error": message}, status=status_code)
+
+
 build_list = BuildListView.as_view()
 build_detail = BuildDetailView.as_view()
 api_batch_upgrade = BuildBatchUpgradeView.as_view()
@@ -358,3 +434,4 @@ def get_object_or_none(self):
 upgrade_operation_detail = UpgradeOperationDetailView.as_view()
 device_upgrade_operation_list = DeviceUpgradeOperationListView.as_view()
 device_firmware_detail = DeviceFirmwareDetailView.as_view()
+upgrade_operation_cancel = UpgradeOperationCancelView.as_view()