Merge pull request #89 from nautobot/develop

Bump Version to 0.9.4

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## v0.9.4 - 2021-06
+
+### Added
+
+- #87 Added the ability to map arbitraty slug to proper netutils expected network_os
+
+### Fixed
+
+- #87 Dispatcher docs and update dependency for nautobot-plugin-nornir
+
 ## v0.9.3 - 2021-06-19
 
 ### Added

README.md

@@ -86,7 +86,7 @@ PLUGINS_CONFIG = {
         "enable_intended": True,
         "enable_sotagg": True,
         "sot_agg_transposer": None,
-        "default_drivers_mapping": None,
+        "platform_slug_map": None,
     },
 }
 
@@ -100,14 +100,16 @@ The plugin behavior can be controlled with the following list of settings.
 | enable_compliance | True | True | A boolean to represent whether or not to run the compliance process within the plugin. |
 | enable_intended | True | True | A boolean to represent whether or not to generate intended configurations within the plugin. |
 | enable_sotagg | True | True | A boolean to represent whether or not to provide a GraphQL query per device to allow the intended configuration to provide data variables to the plugin. |
-| default_drivers_mapping | {"newos": "dispatcher.newos"} | None | A dictionary in which the key is a platform slug and the value is the import path of the dispatcher in string format|
+| platform_slug_map | {"cisco_wlc": "cisco_aireos"} | None | A dictionary in which the key is the platform slug and the value is what netutils uses in any "network_os" parameter. |
 | sot_agg_transposer | mypkg.transposer | - | A string representation of a function that can post-process the graphQL data. |
 | per_feature_bar_width | 0.15 | 0.15 | The width of the table bar within the overview report | 
 | per_feature_width | 13 | 13 | The width in inches that the overview table can be. | 
 | per_feature_height | 4 | 4 | The height in inches that the overview table can be. | 
 
 > Note: Over time the intention is to make the compliance report more dynamic, but for now allow users to configure the `per_*` configs in a way that fits best for them.
 
+> Note: Review [`nautobot_plugin_nornir`](https://pypi.org/project/nautobot-plugin-nornir/) for Nornir and dispatcher configuration options. 
+
 # Contributing
 
 Pull requests are welcomed and automatically built and tested against multiple versions of Python and Nautobot through TravisCI.

docs/FAQ.md

@@ -38,19 +38,47 @@ _Why not predefine the configuration feature map?_
 The process is based on an opinion on what defines a feature, for one organization BGP may include the prefix configuration and another it would not.
 Understanding that there will never be consensus on what should go into a feature it becomes obvious why the users must maintain such configuration.
 
-_What are the supported platforms? How do I configure a device with a specific OS?_
+_What are the supported platforms for Compliance jobs? How do I configure a device with a specific OS?_
 
-The current supported platform and the associated *required* platform slug names are the following:
+The current supported platform and the associated *default* platform slug names are the following for:
 
 * arista_eos
+* cisco_asa
 * cisco_ios
 * cisco_nxos
 * cisco_aireos
 * linux
 * bigip_f5
 * juniper_junos
 
-In many use cases, this can be extended with a custom dispatcher, which is controlled in the [nornir-nautobot](https://github.com/nautobot/nornir-nautobot) repository.
+The expected "network_os" parameter must be as defined by netutils and golden config uses the platform slug to map from the device to the appropriate
+"network_os" that netutils expects. However, there an ability to map the actual platform slug for compliance and parsing tasks via the plugin settings in your
+"nautobot_config.py", and documented on the primary Readme.
+
+To provide a concrete example of this, note the following example that demonstrates how you can transpose any platform slug name to the expected one, as well as
+map multiple keys to a single netutils expected key.
+```json
+{
+    "platform_slug_map":  {
+        "cisco_aireos": "cisco_wlc",
+        "ios": "cisco_ios",
+        "iosxe": "cisco_ios"
+    }
+}
+```
+
+_What are the supported platforms for Backup and Intended Configuratino jobs? How do I configure a device with a specific OS?_
+
+The current supported platform and the associated *default* platform slug names are the following for:
+
+* arista_eos
+* cisco_ios
+* cisco_ios_xr
+* cisco_nxos
+* juniper_junos
+
+In many use cases, this can be extended with a custom dispatcher for nornir tasks, which is controlled in the [nornir-nautobot](https://github.com/nautobot/nornir-nautobot) repository. Additionally the [`nautobot_plugin_nornir`](https://pypi.org/project/nautobot-plugin-nornir/) provide the ability to leverage the
+`dispatcher_mapping` configuration parameter to map and/or extend for your environment. Please see the instructions there for further details.
 
 _Why not provide the corrective configurations?_
 

docs/navigating-compliance.md

@@ -120,6 +120,8 @@ what a line starts with only. Meaning, there is an implicit greediness to the ma
 
 > Note: If accidentally the data is "corrupted" with a bad tested match, simply delete the devices an re-run the compliance process.
 
+> Note: The mapping of "network_os" as defined by netutils is provided via the plugin settings in your nautobot_config.py, and documented on the primary Readme.
+
 # Compliance View
 
 The compliance overview will provide a per device and feature overview on the compliance of your network devices. From here you can navigate to the details view.
@@ -142,7 +144,9 @@ Please note the following about the compliance details page.
 
 # Supported Platforms
 
-Platforms support technically come from the options provided by nornir_nautobot. However, for reference, the valid slug's of the platforms are provided in the [FAQ](./FAQ.md).
+Platforms support technically come from the options provided by [nornir-nautobot](https://github.com/nautobot/nornir-nautobot) for nornir dispatcher tasks and
+[netutils](https://github.com/networktocode/netutils) for configuration compliance and parsing. However, for reference, the valid slug's of the platforms are
+provided in the [FAQ](./FAQ.md).
 
 # Overview Report
 

docs/navigating-golden.md

@@ -153,4 +153,5 @@ garbage collection and it is up to the operator to remove such data.
 
 # Network Operating System Support
 
-The version of OS's supported is documented in the [FAQ](./FAQ.md) and is controlled the platform slug. The platform slug must be exactly as expected for the plugin to work.
+The version of OS's supported is documented in the [FAQ](./FAQ.md) and is controlled the platform slug. The platform slug must be exactly as expected or leverage
+a configuration option--which is described the the FAQ--for the plugin to work.

nautobot_golden_config/__init__.py

@@ -1,6 +1,6 @@
 """Plugin declaration for nautobot_golden_config."""
 
-__version__ = "0.9.3"
+__version__ = "0.9.4"
 
 from nautobot.extras.plugins import PluginConfig
 

nautobot_golden_config/models.py

@@ -18,7 +18,7 @@
 from netutils.config.compliance import feature_compliance
 
 from nautobot_golden_config.choices import ComplianceRuleTypeChoice
-
+from nautobot_golden_config.utilities.utils import get_platform
 
 LOGGER = logging.getLogger(__name__)
 GRAPHQL_STR_START = "query ($device_id: ID!)"
@@ -205,7 +205,7 @@ def save(self, *args, **kwargs):
             "name": self.rule.feature.name,
             "section": self.rule.match_config.splitlines(),
         }
-        value = feature_compliance(feature, self.actual, self.intended, self.device.platform.slug)
+        value = feature_compliance(feature, self.actual, self.intended, get_platform(self.device.platform.slug))
         self.compliance = value["compliant"]
         if self.compliance:
             self.compliance_int = 1

nautobot_golden_config/nornir_plays/config_backup.py

@@ -12,10 +12,11 @@
 
 from nautobot_plugin_nornir.plugins.inventory.nautobot_orm import NautobotORMInventory
 from nautobot_plugin_nornir.constants import NORNIR_SETTINGS
+from nautobot_plugin_nornir.utils import get_dispatcher
+
 
 from nautobot_golden_config.utilities.helper import (
     get_job_filter,
-    get_dispatcher,
     verify_global_settings,
     check_jinja_template,
 )

nautobot_golden_config/nornir_plays/config_compliance.py

@@ -24,6 +24,7 @@
     check_jinja_template,
 )
 from nautobot_golden_config.nornir_plays.processor import ProcessGoldenConfig
+from nautobot_golden_config.utilities.utils import get_platform
 
 
 InventoryPluginRegister.register("nautobot-inventory", NautobotORMInventory)
@@ -95,8 +96,8 @@ def run_compliance(  # pylint: disable=too-many-arguments,too-many-locals
         logger.log_failure(obj, f"There is no `user` defined feature mapping for platform slug {platform}.")
         raise NornirNautobotException()
 
-    if platform not in parser_map.keys():
-        logger.log_failure(obj, f"There is currently no parser support for platform slug {platform}.")
+    if get_platform(platform) not in parser_map.keys():
+        logger.log_failure(obj, f"There is currently no parser support for platform slug {get_platform(platform)}.")
         raise NornirNautobotException()
 
     backup_cfg = _open_file_config(backup_file)
@@ -109,8 +110,8 @@ def run_compliance(  # pylint: disable=too-many-arguments,too-many-locals
             device=obj,
             rule=feature["obj"],
             defaults={
-                "actual": section_config(feature, backup_cfg, platform),
-                "intended": section_config(feature, intended_cfg, platform),
+                "actual": section_config(feature, backup_cfg, get_platform(platform)),
+                "intended": section_config(feature, intended_cfg, get_platform(platform)),
             },
         )
 

nautobot_golden_config/nornir_plays/config_intended.py

@@ -15,11 +15,11 @@
 
 from nautobot_plugin_nornir.plugins.inventory.nautobot_orm import NautobotORMInventory
 from nautobot_plugin_nornir.constants import NORNIR_SETTINGS
+from nautobot_plugin_nornir.utils import get_dispatcher
 
 from nautobot_golden_config.models import GoldenConfigSetting, GoldenConfig
 from nautobot_golden_config.utilities.helper import (
     get_job_filter,
-    get_dispatcher,
     verify_global_settings,
     check_jinja_template,
 )

nautobot_golden_config/utilities/helper.py

@@ -5,12 +5,10 @@
 from jinja2.exceptions import TemplateError, TemplateSyntaxError
 
 from nornir_nautobot.exceptions import NornirNautobotException
-from nornir_nautobot.plugins.tasks.dispatcher import _DEFAULT_DRIVERS_MAPPING
 from nautobot.dcim.filters import DeviceFilterSet
 from nautobot.dcim.models import Device
 
 from nautobot_golden_config import models
-from nautobot_golden_config.utilities.constant import PLUGIN_CFG
 
 
 FIELDS = {
@@ -46,13 +44,6 @@ def get_job_filter(data=None):
     return DeviceFilterSet(data=query, queryset=base_qs).qs
 
 
-def get_dispatcher():
-    """Helper method to load the dispatcher from nautobot nornir or config if defined."""
-    if PLUGIN_CFG.get("dispatcher_mapping"):
-        return PLUGIN_CFG["dispatcher_mapping"]
-    return _DEFAULT_DRIVERS_MAPPING
-
-
 def null_to_empty(val):
     """Convert to empty string if the value is currently null."""
     if not val:

nautobot_golden_config/utilities/utils.py

@@ -0,0 +1,10 @@
+"""Utility functions."""
+
+from nautobot_golden_config.utilities.constant import PLUGIN_CFG
+
+
+def get_platform(platform):
+    """Utility method to map user defined platform slug to netutils named entity."""
+    if PLUGIN_CFG.get("platform_slug_map", {}).get(platform):
+        return PLUGIN_CFG["platform_slug_map"][platform]
+    return platform