diff --git a/README.md b/README.md index f566eaa9..4b1cd4ce 100644 --- a/README.md +++ b/README.md @@ -33,38 +33,7 @@ More detailed information and examples about the specific usage of the additiona ## Related MITRE Work -### CTI - -[Cyber Threat Intelligence repository](https://github.com/mitre/cti) of the ATT&CK catalog expressed in STIX 2.0 JSON. -This repository also contains [our USAGE document](https://github.com/mitre/cti/blob/master/USAGE.md) which includes -additional examples of accessing and parsing our dataset in Python. - -### ATT&CK - -ATT&CK® is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of -an adversary’s lifecycle, and the platforms they are known to target. -ATT&CK is useful for understanding security risk against known adversary behavior, -for planning security improvements, and verifying defenses work as expected. - - - -### STIX - -Structured Threat Information Expression (STIX™) is a language and serialization format used to exchange cyber threat intelligence (CTI). - -STIX enables organizations to share CTI with one another in a consistent and machine-readable manner, -allowing security communities to better understand what computer-based attacks they are most likely to -see and to anticipate and/or respond to those attacks faster and more effectively. - -STIX is designed to improve many capabilities, such as collaborative threat analysis, automated threat exchange, automated detection and response, and more. - - - -### ATT&CK scripts - -One-off scripts and code examples you can use as inspiration for how to work with ATT&CK programmatically. Many of the functionalities found in the mitreattack-python package were originally posted on attack-scripts. - - +Go to [this link](https://mitreattack-python.readthedocs.io/en/latest/related_work.html) for related MITRE work. ## Contributing diff --git a/docs/attacktoexcel.rst b/docs/attacktoexcel.rst new file mode 100644 index 00000000..d09fb62c --- /dev/null +++ b/docs/attacktoexcel.rst @@ -0,0 +1,155 @@ +ATT&CK to Excel +============================================== + +ATT&CK to Excel contains a module for converting `ATT&CK STIX data `_ to Excel spreadsheets. +It also provides a means to access ATT&CK data as `Pandas `_ DataFrames for data analysis. + +Usage: +----- + +Command Line +----- + +Print full usage instructions: + +.. code:: bash + + python3 attackToExcel.py -h + + +Example execution: + +.. code:: bash + + python3 attackToExcel.py + + +Build a excel files corresponding to a specific domain and version of ATT&CK: + +.. code:: bash + + python3 attackToExcel -domain mobile-attack -version v5.0 + + +Module +----- + +Example execution targeting a specific domain and version: + +.. code-block:: python + + import mitreattack.attackToExcel.attackToExcel as attackToExcel + + attackToExcel.export("mobile-attack", "v5.0", "/path/to/export/folder") + + +Interfaces: +----- + +attackToExcel +----- + +attackToExcel provides the means by which to convert/extract the ATT&CK STIX data to Excel spreadsheets. A brief +overview of the available methods follows. + + +.. list-table:: Title + :widths: 33 33 34 + :header-rows: 1 + + * - method name + - arguments + - usage + * - get_stix_data + - `domain`: the domain of ATT&CK to fetch data from
`version`: optional parameter indicating which version to fetch data from (such as "v8.1"). If omitted retrieves the most recent version of ATT&CK.
`remote`: optional parameter that provides a URL of a remote ATT&CK Workbench instance to grab data from. + - Retrieves the ATT&CK STIX data for the specified version and returns it as a MemoryStore object + * - build_dataframes + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Builds a Pandas DataFrame collection as a dictionary, with keys for each type, based on the ATT&CK data provided + * - write_excel + - `dataframes`: pandas DataFrame dictionary (generated by build_dataframes)
`domain`: domain of ATT&CK that `dataframes` corresponds to
`version`: optional parameter indicating which version of ATT&CK is in use
`output_dir`: optional parameter specifying output directory + - Writes out DataFrame based ATT&CK data to excel files + * - export + - `domain`: the domain of ATT&CK to download
`version`: optional parameter specifying which version of ATT&CK to download
`output_dir`: optional parameter specifying output directory + - Downloads ATT&CK data from MITRE/CTI and exports it to Excel spreadsheets + +stixToDf +----- + +stixToDf provides various methods to process and manipulate the STIX data in order to create `Pandas `_ DataFrames for +processing. A brief overview of these methods follows. + +.. list-table:: + :widths: 33 33 34 + :header-rows: 1 + + * - method name + - arguments + - usage + * - techniquesToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX techniques from the provided data and returns corresponding Pandas DataFrames. + * - tacticsToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX tactics from the provided data and returns corresponding Pandas DataFrames. + * - softwareToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX software from the provided data and returns corresponding Pandas DataFrames. + * - groupsToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX groups from the provided data and returns corresponding Pandas DataFrames. + * - mitigationsToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX mitigations from the provided data and returns corresponding Pandas DataFrames. + * - relationshipsToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX relationships from the provided data and returns corresponding Pandas DataFrames. + * - matricesToDf + - `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to + - Parses STIX matrices from the provided data and returns a parsed matrix structure of the form `[{matrix, name, description, merge, border}, ...]` + + +Spreadsheet format +----- + +The Excel representation of the ATT&CK dataset includes both master spreadsheets, +containing all object types, and individual spreadsheets for each object type. +The individual type spreadsheets break out relationships (e.g procedure examples connecting groups to techniques) +into separate sheets by relationship type, while the master spreadsheet includes all relationship types in a single sheet. +Otherwise, the representation is identical. + +A citations sheet can be used to look up the in-text citations which appear in some fields. +For domains that include multiple matrices, such as Mobile ATT&CK, each matrix gets its own named sheet. +Unlike the STIX dataset, objects that have been revoked or deprecated are not included in the spreadsheets. + +Accessing the Pandas DataFrames +----- + +Internally, attackToExcel stores the parsed STIX data as `Pandas `_ DataFrames. +These can be retrieved for use in data analysis. + +Example of accessing `Pandas `_ DataFrames: + +.. code-block:: python + + import mitreattack.attackToExcel.attackToExcel as attackToExcel + import mitreattack.attackToExcel.stixToDf as stixToDf + + # download and parse ATT&CK STIX data + attackdata = attackToExcel.get_stix_data("enterprise-attack") + techniques_data = stixToDf.techniquesToDf(attackdata, "enterprise-attack") + + # show T1102 and sub-techniques of T1102 + techniques_df = techniques_data["techniques"] + print(techniques_df[techniques_df["ID"].str.contains("T1102")]["name"]) + # 512 Web Service + # 38 Web Service: Bidirectional Communication + # 121 Web Service: Dead Drop Resolver + # 323 Web Service: One-Way Communication + # Name: name, dtype: object + + # show citation data for LOLBAS Wmic reference + citations_df = techniques_data["citations"] + print(citations_df[citations_df["reference"].str.contains("LOLBAS Wmic")]) + # reference citation url + # 1010 LOLBAS Wmic LOLBAS. (n.d.). Wmic.exe. Retrieved July 31, 2... https://lolbas-project.github.io/lolbas/Binari... diff --git a/docs/collections.rst b/docs/collections.rst new file mode 100644 index 00000000..cfaae6a8 --- /dev/null +++ b/docs/collections.rst @@ -0,0 +1,113 @@ + +Collections +============================================== + +Collections contains modules and scripts for working with ATT&CK collections. +Collections are sets of ATT&CK STIX objects, grouped for user convienence. +For more information about ATT&CK collections, see the corresponding +`ATT&CK documentation `_. + +Collections Scripts +------------------------ + + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - script + - description + * - `index_to_markdown `_ + - Provides a means by which to convert a `collection index `_ into a human-readable markdown file. More information can be found in the corresponding :ref:`index_to_markdown.py` below. + * - `collection_to_index `_ + - Provides a means by which to convert a `collection `_ into a easy-to-share `index file `_. More information can be found in the corresponding :ref:`section` below. + * - `stix_to_collection `_ + - Provides a means by which to convert raw stix (in the form of `bundles `_) into a `collection `_. More information can be found in the corresponding :ref:`section` below. + +index_to_markdown.py +------------------------ + +``index_to_markdown.py`` provides the ``IndexToMarkdown`` class, which provides a way to transform an existing +`collection index file `_ +into a markdown file for easy of use and reference. +The ``IndexToMarkdown`` class is very simple, and provides a single method, ``index_to_markdown``, +which in turn only requires a single parameter - a dictionary representation of the desired index file to convert to markdown. +An example of how to use the class, and method, can be found below. + +Example Usage +------------------------ + + .. code-block:: python + + import json + from mitreattack.collections import IndexToMarkdown + + with open('collection_index.json', 'r') as input_file: + with open('collection_index.md', 'w') as output_file: + input_index = json.load(input_file) + generated_md = IndexToMarkdown.index_to_markdown(input_index) # Convert index to markdown + output_file.write(generated_md) + print(generated_md) + + +collection_to_index.py +------------------------ + +`collection_to_index.py` provides the `CollectionToIndex` class, which proves a means by which to summarize existing +`collections `_ +into a single `collection index `_ file. +The `CollectionToIndex` class contains the `generate_index` function, which when provided with a name, description, root url (pointing to where the raw collections are stored), +and a list of either files, folders, or already loaded bundles in the form of dictionaries, will create a summarizing index. + +Example Usage +--------------- + + .. code-block:: python + + import json + from mitreattack.collections import CollectionToIndex + + output_indexA = CollectionToIndex.generate_index(name='example', description='example index', + root_url='www.example.com', + files=['/path/to/collection1.json', '/path/to/collection2.json'], + folders=None, sets=None) + output_indexB = CollectionToIndex.generate_index(name='example2', description='demonstration index', + root_url='www.example.com', + files=None, folders=['/path/to/folder/with/collections'], sets=None) + with open('path/to/bundle/bundleC.json', 'r') as f: + data = json.load(f) + output_indexC = CollectionToIndex.generate_index(name='example3', description='exhibit index', + root_url='www.example.com', + files=None, folders=None, sets=[data]) + print(output_indexA) + print(output_indexB) + print(output_indexC) + + +stix_to_collection.py +------------------------ + +`stix_to_collection.py` provides the `STIXToCollection` class, which proves a means by which to convert +existing stix bundles into ones containing a +`collection `_ object. +The `STIXToCollection` class contains the `stix_to_collection` function, which when provided with a starter bundle, +a name, a version, and an optional description, will output a modified bundle that contains a summary collection object. + +Example Usage +-------------- + +.. code-block:: python + + import json + from mitreattack.collections import STIXToCollection + + with open('path/to/bundle/bundle2_0.json', 'r') as f: + data = json.load(f) + output_bundleA = STIXToCollection.stix_to_collection(bundle=data, name='collectionA', version='9.1', description="demo bundle (2.0)") + + with open('path/to/bundle/bundle2_1.json', 'r') as f: + data = json.load(f) + output_bundleB = STIXToCollection.stix_to_collection(bundle=data, name='collectionB', version='9.0', description="demo bundle (2.1)") + + print(output_bundleA) + print(output_bundleB) diff --git a/docs/diffinstix.rst b/docs/diffinstix.rst new file mode 100644 index 00000000..b7c88f81 --- /dev/null +++ b/docs/diffinstix.rst @@ -0,0 +1,171 @@ +Diff Stix +============================================== + +Diff Stix contains a module for creating markdown, HTML, JSON and/or ATT&CK Navigator layers +reporting on the changes between two versions of the STIX2 bundles representing the ATT&CK content. +Run ``diff_stix -h`` for full usage instructions. + +Usage +----- + +Command Line +----- + +Print full usage instructions: + +.. code:: bash + + # You must run `pip install mitreattack-python` in order to access the diff_stix command + diff_stix --help + usage: diff_stix [-h] [--old OLD] [--new NEW] [--domains {enterprise-attack,mobile-attack,ics-attack} [{enterprise-attack,mobile-attack,ics-attack} ...]] [--markdown-file MARKDOWN_FILE] [--html-file HTML_FILE] [--html-file-detailed HTML_FILE_DETAILED] + [--json-file JSON_FILE] [--layers [LAYERS ...]] [--site_prefix SITE_PREFIX] [--unchanged] [--use-mitre-cti] [--show-key] [--contributors] [--no-contributors] [-v] + + Create changelog reports on the differences between two versions of the ATT&CK content. Takes STIX bundles as input. For default operation, put enterprise-attack.json, mobile-attack.json, and ics-attack.json bundles in 'old' and 'new' folders for the script to compare. + + options: + -h, --help show this help message and exit + --old OLD Directory to load old STIX data from. + --new NEW Directory to load new STIX data from. + --domains {enterprise-attack,mobile-attack,ics-attack} [{enterprise-attack,mobile-attack,ics-attack} ...] + Which domains to report on. Choices (and defaults) are enterprise-attack, mobile-attack, ics-attack + --markdown-file MARKDOWN_FILE + Create a markdown file reporting changes. + --html-file HTML_FILE + Create HTML page from markdown content. + --html-file-detailed HTML_FILE_DETAILED + Create an HTML file reporting detailed changes. + --json-file JSON_FILE + Create a JSON file reporting changes. + --layers [LAYERS ...] + Create layer files showing changes in each domain expected order of filenames is 'enterprise', 'mobile', 'ics', 'pre attack'. If values are unspecified, defaults to output/January_2023_Updates_Enterprise.json, + output/January_2023_Updates_Mobile.json, output/January_2023_Updates_ICS.json, output/January_2023_Updates_Pre.json + --site_prefix SITE_PREFIX + Prefix links in markdown output, e.g. [prefix]/techniques/T1484 + --unchanged Show objects without changes in the markdown output + --use-mitre-cti Use content from the MITRE CTI repo for the -old data + --show-key Add a key explaining the change types to the markdown + --contributors Show new contributors between releases + --no-contributors Do not show new contributors between releases + -v, --verbose Print status messages + + +Example execution: + +.. code:: bash + + diff_stix -v --show-key --html-file output/changelog.html --html-file-detailed output/changelog-detailed.html --markdown-file output/changelog.md --json-file output/changelog.json --layers output/layer-enterprise.json output/layer-mobile.json output/layer-ics.json --old path/to/old/stix/ --new path/to/new/stix/ + + +Changelog JSON format +----- + +The changelog helper script has the option to output a JSON file with detailed differences between ATT&CK releases. +This is the overall structure you can expect to find in the file. +A brief explanation of key pieces can be found below. + +.. code-block:: json + { + "enterprise-attack": { + "techniques": { + "additions": [], + "major_version_changes": [], + "minor_version_changes": [], + "other_version_changes": [], + "patches": [], + "revocations": [], + "deprecations": [], + "deletions": [], + }, + "software": {}, + "groups": {}, + "campaigns": {}, + "mitigations": {}, + "datasources": {}, + "datacomponents": {} + }, + "mobile-attack": {}, + "ics-attack": {}, + "new-contributors": [ + "Contributor A", + "Contributor B", + "Contributor C" + ] + } + + +* The top-level objects include information about specific domains as well as ``new-contributors``, which are only found in the newer ATT&CK release. +* For domain changes, they are broken down by object type, e.g. ``techniques`` or ``mitigations``. +* The following table helps break down the change types that are currently tracked. + +.. list-table:: Title + :widths: 33 33 34 + :header-rows: 1 + + * - field + - type + - description + * - ``additions`` + -array[object] + - ATT&CK objects which are only present in the new STIX data. + * - ``major_version_changes`` + - array[object] + - ATT&CK objects that have a major version change. (e.g. 1.0 → 2.0). + * - ``minor_version_changes`` + - array[object] + - ATT&CK objects that have a minor version change. (e.g. 1.0 → 1.1). + * - ``other_version_changes`` + - array[object] + - array[object] | ATT&CK objects that have a version change of any other kind. (e.g. 1.0 → 1.3). These are unintended, but can be found in previous releases. + * - ``patches`` + - array[object] + - ATT&CK objects that have been patched while keeping the version the same. + * - ``revocations`` + - array[object] + - ATT&CK objects which are revoked by a different object. + * - ``deprecations`` + - array[object] + - ATT&CK objects which are deprecated and no longer in use, and not replaced. + * - ``deletions`` + - array[object + - ATT&CK objects which are no longer found in the STIX data. This should almost never happen. + + +Changed Objects +----- + +The bulk of the changelog file consists of lists of JSON objects. +If you are familiar with reading the STIX format, they may look famliar, yet a little "off". +That is because there are a few fields that have been added in some cases depending on what section they appear in. +For example, objects that are brand new do not have ``previous_version`` available to them. +The following table lists the extra fields that can be found in objects in the changelog. + +.. list-table:: + :widths: 25 25 25 25 + :header-rows: 1 + + * - Field + - Required + - Type + - Description + * - ``changelog_mitigations`` + - false + - object + - Three lists for ``shared``, ``new``, and ``dropped`` for Mitigations that are related to a Technique between versions. + * - ``changelog_detections`` + - false + - object + - HTML rendering of a table that displays the differences between descriptions for an ATT&CK object. + * - ``detailed_diff`` + - false + - string + - A python DeepDiff object that has been JSON serialized which represents STIX changes for an ATT&CK object between releases. + * - ``previous_version`` + - false + - string + - If the object existed in the previous release, then it denotes the version the object was in the previous release. + * - ``version_change`` + - false + - string + - If the object existed in the previous release and was changed in the current release, then a descriptive string in the format '``old-version`` → ``new-version``' + + diff --git a/docs/getting_started.rst b/docs/getting_started.rst new file mode 100644 index 00000000..24fe676c --- /dev/null +++ b/docs/getting_started.rst @@ -0,0 +1,87 @@ + +Getting started +============================================== + +This repository houses the ATT&CK catalog's Cyber Threat Intelligence, represented in the STIX 2.0 JSON format. +Additionally, it includes a USAGE document that provides further examples for accessing and parsing our dataset using Python. + +Install +----- + +To use this package, install the mitreattack-python library with `pip `_ : + +.. code:: bash + + pip install mitreattack-python + + +Note: the library requires `python3 `_. + +MitreAttackData Library +------------------------ + +The ``MitreAttackData`` library is used to read in and work with MITRE ATT&CK STIX 2.0 content. This library provides +the ability to query the dataset for objects and their related objects. This is the main content of mitreattack-python; +you can read more about other modules in this library under the "Additional Modules" header in the side pane. + +Additional Modules +------------------------ + +More detailed information and examples about the specific usage of the additional modules in this package can be found in the individual documents for each module linked below. + + +.. list-table:: + :widths: 33 33 34 + :header-rows: 1 + + * - module + - description + - documentation + * - `navlayers `_ + - A collection of utilities for working with `ATT&CK Navigator `_ layers. Provides the ability to import, export, and manipulate layers. Layers can be read in from the filesystem or python dictionaries, combined and edited, and then exported to excel or SVG images. + - Further documentation can be found `here `_ + * - `attackToExcel `_ + - A collection of utilities for converting `ATT&CK STIX data `_ to Excel spreadsheets. It also provides access to `Pandas `_ DataFrames representing the dataset for use in data analysis. + - Further documentation can be found `here `_ + * - `collections `_ + - A set of utilities for working with `ATT&CK Collections and Collection Indexes `_. Provides functionalities for converting and summarizing data in collections and collection indexes, as well as generating a collection from a raw stix bundle input. + - Further documentation can be found `here `_. + * - `diffStix `_ + - Create markdown, HTML, JSON and/or ATT&CK Navigator layers reporting on the changes between two versions of the STIX2 bundles representing the ATT&CK content. Run ``diff_stix -h`` for full usage instructions. + - Further documentation can be found `here `_ + + +Related MITRE Work +------------------------ + +Go to `this link `_ for related MITRE work. + + +Contributing +------------------------ + +To contribute to this project, either through a bug report, feature request, or merge request, +please see the `Contributors Guide `_. + +Notice +------------------------ + +Copyright 2023 The MITRE Corporation + +Approved for Public Release; Distribution Unlimited. Case Number 19-0486. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +This project makes use of ATT&CK® + +`ATT&CK Terms of Use `_ diff --git a/docs/index.rst b/docs/index.rst index a624e96d..6d7acb45 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -15,6 +15,8 @@ other modules in this library under "Additional Modules". related_work contributing notice + getting_started + .. toctree:: :maxdepth: 1 @@ -28,7 +30,14 @@ other modules in this library under "Additional Modules". :maxdepth: 1 :caption: Additional Modules - additional_modules/navlayers - additional_modules/attackToExcel - additional_modules/collections - additional_modules/diffStix \ No newline at end of file + diffinstix + collections + navlayers + navlayercore + attacktoexcel + +.. toctree:: + :maxdepth: 1 + :caption: STIX Primer + + stix_primer/overview \ No newline at end of file diff --git a/docs/navlayercore.rst b/docs/navlayercore.rst new file mode 100644 index 00000000..3632b7dc --- /dev/null +++ b/docs/navlayercore.rst @@ -0,0 +1,271 @@ +Layers Core +=============== + +This subcomponent, as part of the larger navlayers module, is responsible for Layer objects. Please note, this +documentation assumes familiarity with the `ATT&CK Navigator layer format `_. +The main handle for this implementation is the Layer, which stores an individual instance of a LayerObj object, +which further references the various sub-objects that make up a complete Layer. A visual representation of this +object breakdown can be seen here (please note there are other fields, these are just the objects): + +.. code-block:: + + demo (Layer instance) <------------------------------------------------> The container for a layer object + |---> demo.layer (_LayerObj instance)--------------------------------> The raw layer object itself + |---> demo.layer.version (Versions instance)-----------------> A versions object + |---> demo.layer.filters (Filter instance)-------------------> A filter object + |---> demo.layer.layout (Layout instance)--------------------> A layout object + |---> demo.layer.techniques (List of Technique instances)----> A collection of technique objects + |---> demo.layer.gradient (Gradient instance)----------------> A gradient object + |---> demo.layer.legendItems (List of LegendItem instances)--> A collection of legend item objects + |---> demo.layer.metadata (List of Metadata instances)-------> A collection of metadata objects + + +Creating Layers Programmatically +------------------------ + +With knowledge of the objects involved, as well as the additional fields (which have a 1:1 mapping with the +default ATT&CK Navigator spec), it is possible to programmatically generate a layer. Below is an example of +how this might be accomplished, piece by piece. + +.. code-block:: python + + import mitreattack.navlayers as navlayers + + layer_example = navlayers.Layer() + layer_example.from_dict(dict(name="example", domain="enterprise-attack")) # arguments required for every layer + + # configure the versions object + layer_example.layer.versions = dict(layer="4.2", attack="9.1", navigator="4.2") + + # set a description + layer_example.layer.description = "This is a demonstration of how to set up a layer piece by piece" + + # configure the "filters" object + layer_example.layer.filters = dict(platforms=['macOS']) # platforms can be provided during initialization + layer_example.layer.filters.platforms = ['Windows'] # or separately + + # configure the 'sorting' setting + layer_example.layer.sorting = 3 # 0: sort ascending alphabetically by technique name + # 1: sort descending alphabetically by technique name + # 2: sort ascending by technique score + # 3: sort descending by technique score + + # configure the layout object + layer_example.layer.layout = dict(layout="side", + showID=True, + showName=True, + showAggregateScores=True, + countUnscored=True, + aggregateFunction="sum") # average, sum, max, min + + # configure whether or not to hide disabled techniques + layer_example.layer.hideDisabled = True + # configure the gradient object + layer_example.layer.gradient = dict(minValue=-100, maxValue=100, + colors=["#DAF7A6", "#FFC300", "#FF5733", "#C70039", "#900C3F", "#581845"]) + # configure collection of legend items + layer_example.layer.legendItems = [dict(label='A', color='#DAF7A6'), dict(label='B', color='#581845')] + # configure collection of metatdata values + layer_example.layer.metadata = [dict(name='example metadata', value='This is an example')] + # create listing of techniques in this layer + layer_example.layer.techniques = [dict(techniqueID='T1000', tactic='privilege-escalation', score=15, color='#AABBCC'), + dict(techniqueID='T1000.1', tactic='privilege-escalation', score=1, comment='Demo')] + +This first example utilizes the native dictionary form for initializing the layer. This approach is similar to the +method used by the automated import process, but may not be the most intuitive for users. An alternative method, +displayed below, is to create and modify instances of the core objects in the library. Please note, these two examples +produce equivalent internal layers once completed. + +.. code-block:: python + + import mitreattack.navlayers as navlayers + + layer_example = navlayers.Layer(name="example", domain="enterprise-attack") # arguments required for every layer + layer_build = layer_example.layer # short handle to make the rest of this example easier to read + + # configure the versions object + versions_obj = navlayers.Versions() + versions_obj.layer = "4.2" + versions_obj.attack = "9.1" + versions_obj.navigator = "4.2" + layer_build.versions = versions_obj + + # set a description + layer_build.description = "This is a demonstration of how to set up a layer piece by piece" + + # configure the "filters" object + filter_obj = navlayers.core.Filter(domain="enterprise-attack") + filter_obj.platforms = ['Windows'] + layer_build.filters = filter_obj + + # configure the 'sorting' setting + layer_build.sorting = 3 # 0: sort ascending alphabetically by technique name + # 1: sort descending alphabetically by technique name + # 2: sort ascending by technique score + # 3: sort descending by technique score + + # configure the layout object + layout_obj = navlayers.core.Layout() + layout_obj.layout = "side" + layout_obj.showID = True + layout_obj.showName = True + layout_obj.showAggregateScores = True + layout_obj.countUnscored = True + layout_obj.aggregateFunction = "sum" # average, sum, max, min + layer_build.layout = layout_obj + + # configure whether or not to hide disabled techniques + layer_build.hideDisabled = True + + # configure the gradient object + gradient_obj = navlayers.core.Gradient(colors=["#DAF7A6", "#FFC300", "#FF5733", "#C70039", "#900C3F", "#581845"], + minValue=-100, maxValue=100) + layer_build.gradient = gradient_obj + + # configure collection of legend items + legend_item_obj_a = navlayers.core.LegendItem(label='A', color='#DAF7A6') + legend_item_obj_b = navlayers.core.LegendItem(label='B', color='#581845') + list_of_legend_items = [legend_item_obj_a, legend_item_obj_b] + layer_build.legendItems = list_of_legend_items + + # configure collection of metatdata values + metadata_object = navlayers.core.Metadata(name='example metadata', value='This is an example') + layer_build.metadata = [metadata_object] + + # create listing of techniques in this layer + technique_obj_a = navlayers.core.Technique(tID='T1000') + technique_obj_a.tactic = 'privilege-escalation' + technique_obj_a.score = 15 + technique_obj_a.color = '#AABBCC' + technique_obj_b = navlayers.core.Technique(tID='T1000.1') + technique_obj_b.tactic = 'privilege-escalation' + technique_obj_b.score = 1 + technique_obj_b.comment = "Demo" + layer_build.techniques = [technique_obj_a, technique_obj_b] + + + +Object Documentation +------------------------ + +Should it be helpful, the following section provides a breakdown of the available fields and methods for +each of the objects in the Core. This only includes 'public' methods and fields; there may be others used +for processing and other functionality that are not documented here, though documentation does exist for these +in the source code for them. + +**Layer Object** + +.. code-block:: python + + Layer().layer # Stores the raw LayerObj file + Layer().strict # Determines whether or not to be strict about loading files + Layer().from_str() # Initializes data from a string + Layer().from_dict() # Initializes data from a dictionary + Layer().from_file() # Initializes data from a file + Layer().to_file() # Exports the layer data to a file + Layer().to_dict() # Exports the layer data to a dictionary + Layer().to_str() # Exports the layer data to a string + +**LayerObj Object** + +.. code-block:: python + + _LayerObj().versions # Link to a Versions object instance + _LayerObj().name # The Name for the Layer + _LayerObj().description # A description string for the Layer + _LayerObj().domain # The domain for the Layer + _LayerObj().filters # Link to a Filter object instance + _LayerObj().sorting # An integer denoting which sorting form to use + _LayerObj().layout # Link to a Layout object instance + _LayerObj().hideDisabled # Bool determining whether or not to show disabled techniques + _LayerObj().techniques # List of links to Technique objects + _LayerObj().gradient # Link to Gradient object + _LayerObj().legendItems # List of links to LegendItems objects + _LayerObj().showTacticRowBackground # Bool determining whether or not to show a background for tactics + _LayerObj().tacticRowBackground # Color code for tactic background + _LayerObj().selectTechniquesAcrossTactics # Bool determining whether or not to select cross-tactic + _LayerObj().selectSubtechniquesWithParent # Bool determining whether or not to select subtechniques + _LayerObj().metadata # List of links to Metadata items + _LayerObj().get_dict() # Export Layer as a dictionary object + +**Versions Object** + +.. code-block:: python + + Versions().layer # String denoting Layer format version + Versions().__attack # String denoting ATT&CK version + Versions().navigator # String denoting Navigator version + Versions().get_dict() # Export Version data as a dictionary object + +**Filter Object** + +.. code-block:: python + + Filter().domain # String denoting the domain for the Filter + Filter().platforms # String denoting platforms within this filter + Filter().get_dict() # Export Filter data as a dictionary object + +Please note that although not documented here, there is another Filter object variant, Filterv3, which exists +for backwards compatibility reasons. + +**Layout Object** + +.. code-block:: python + + Layout().layout # String denoting which layout form to use + Layout().showID # Bool denoting whether or not to show technique IDs + Layout().showName # Bool denoting whether or not to show technique names + Layout().showAggregateScores # Bool denoting whether or not to utilize Aggregate scores + Layout().countUnscored # Bool denoting whether ot not to count unscored techniques as 0s for Aggregates + Layout().aggregateFunction # A enum integer denoting which aggregate function to utilize + # 1 - Average, 2 - min, 3 - max, 4 - sum + Layout().get_dict() # Export Layout data as a dictionary object + Layout().compute_aggregate() # Compute the aggregate score for a technique and it's subtechniques + +**Technique Object** + +.. code-block:: python + + Technique().techniqueID # String denoting the technique's ID + Technique().tactic # String denoting the technique's tactic + Technique().comment # String denoting any comments + Technique().enabled # Bool denoting if the technique is enabled + Technique().score # Integer denoting technique score + Technique().aggregateScore # Integer denoting pre-configured aggregate score + Technique().color # String denoting manually configured color code + Technique().metadata # List of links to metadata objects + Technique().showSubtechniques # Bool denoting whether or not to show subtechniques + Technique().get_dict() # Export Technique data as a dictionary object + +**Gradient Object** + +.. code-block:: python + Gradient().colors # Array of colors (string codes) over which the gradient is to be calculated + Gradient().minValue # Integer denoting minimum viable value on the gradient + Gradient().maxValue # Integer denoting maximum viable value on the gradient + Gradient().compute_color() # Calculate the appropriate color for a given score on the gradient + Gradient().get_dict() # Export Gradient data as a dictionary object + +**LegendItem Object** + +.. code-block:: python + + LegendItem().label # String denoting the label for this Legend Item' item + LegendItem().color # String denoting the color code for the Legend Item + LegendItem().get_dict() # Export Legend Item data as a dictionary object + +**Metadata/Metadiv Object** + +.. code-block:: python + + Metadata().name # String denoting metadata keypair name + Metadata().value # String denoting metadata keypair value + Metadata().get_dict() # Export metadata data as a dictionary object + +.. code-block:: python + + Metadiv().name # Always set to "DIVIDER" + Metadiv().value # Bool denoting active or not + Metadiv().get_dict() # Export metadiv as a dictionary object + +A ``Metadiv`` object is simply a modified version of a ``Metadata`` object used as a visual divider. \ No newline at end of file diff --git a/docs/navlayers.rst b/docs/navlayers.rst new file mode 100644 index 00000000..921f905c --- /dev/null +++ b/docs/navlayers.rst @@ -0,0 +1,669 @@ +NavLayers +============================================== + +NavLayers contains modules and scripts for working with ATT&CK Navigator layers. +ATT&CK Navigator Layers are a set of annotations overlaid on top of the ATT&CK Matrix. +For more about ATT&CK Navigator layers, visit the ATT&CK Navigator repository. +The core module allows users to load, validate, manipulate, and save ATT&CK layers. +A brief overview of the components can be found below. +All scripts adhere to the MITRE ATT&CK Navigator Layer file format, +`version 4.3 `_, +but will accept legacy `version 3.0 `_ +and version 4.X layers, upgrading them to version 4.3. + +Core Modules +------------------------ + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - script + - description + * - `filter `_ + - Implements a basic `filter object `_ + * - `gradient `_ + - Implements a basic `gradient object `_ + * - `layer `_ + - Provides an interface for interacting with core module's layer representation. A further breakdown can be found in the corresponding :ref:`section` below. + * - `layout `_ + - Implements a basic `layout object `_ + * - `legenditem `_ + - Implements a basic `legenditem object `_ + * - `metadata `_ + - Implements a basic `metadata object `_ + * - `technique `_ + - Implements a basic `technique object `_ + * - `versions `_ + - Implements a basic `versions object `_ + +Manipulator Scripts +------------------------ + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - script + - description + * - `layerops `_ + - Provides a means by which to combine multiple ATT&CK layer objects in customized ways. A further breakdown can be found in the corresponding :ref:`section` below. + +Exporter Scripts +------------------------ + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - script + - description + * - `to_excel `_ + - Provides a means by which to export an ATT&CK Layer to an excel file. A further breakdown can be found in the corresponding `section` below. + * - `to_svg `_ + - Provides a means by which to export an ATT&CK layer to an svg image file. A further breakdown can be found in the corresponding `section`_ + - Provides a means by which to generate an ATT&CK Layer that summarizes, on a per technique basis, all instances of a given ATT&CK object type that reference/utilize each technique. A further explanation can be found in the corresponding `section` below. + * - `usage_generator `_ + - Provides a means by which to generate an ATT&CK Layer that summarizes the techniques associated with a given ATT&CK object. A further explanation can be found in the corresponding `section` below. + * - `sum_generator `_ + - Provides a means by which to generate a collection of ATT&CK Layers, one for each object in a given ATT&CK object class, that summarizes the coverage of that object. A further explanation can be found in the corresponding `section`_ + - Provides a means by which to convert a matrix into a clean excel matrix template. + * - `matrix_gen `_ + - Provides a means by which to generate a matrix from raw data, either from the ATT&CK TAXII server, from a local STIX Bundle, or from an ATT&CK Workbench instance (via url). + * - `svg_templates `_ + - Provides a means by which to convert a layer file into a marked up svg file. + * - `svg_objects `_ + - Provides raw templates and supporting functionality for generating svg objects. + +Command Line Tools +------------------------ + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - script + - description + * - `layerExporter_cli.py `_ + - A commandline utility to export Layer files to excel or svg formats using the exporter tools. Run with `-h` for usage. + * - `layerGenerator_cli.py `_ + - A commandline utility to generate Layer files that correspond to various and collections of various stix objects. Run with `-h` for usage. + +Layer +------------------------ + +The ``Layer`` class provides format validation and read/write capabilities to aid in working with ATT&CK Navigator Layers in python. +It is the primary interface through which other Layer-related classes defined in the core module should be used. +The Layer class API and a usage example are below. +The class currently supports version 3 and 4 of the ATT&CK Layer spec, and will upgrade version 3 layers into compatible version 4 ones whenever possible. + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - method [x = Layer()] + - description + * - ``x.from_str(_input_)`` + - Loads an ATT&CK layer from a string representation of a json layer. + * - ``x.from_dict(_input_)`` + - Loads an ATT&CK layer from a dictionary. + * - ``x.from_file(_filepath_)`` + - Loads an ATT&CK layer from a file location specified by the _filepath_. + * - ``x.to_file(_filepath_)`` + - Saves the current state of the loaded ATT&CK layer to a json file denoted by the _filepath_. + * - ``x.to_dict()`` + - Returns a representation of the current ATT&CK layer object as a dictionary. + * - ``x.to_str()`` + - Returns a representation of the current ATT&CK layer object as a string representation of a dictionary. + +Examples on how to create a layer programmatically, as opposed to loading it from an existing medium, can be found +`here `_. + +Example Usage +------------------------ + +.. code-block:: python + + example_layer3_dict = { + "name": "example layer", + "version": "3.0", + "domain": "mitre-enterprise" + } + + example_layer4_dict = { + "name": "layer v4.3 example", + "versions" : { + "attack": "8", + "layer" : "4.3", + "navigator": "4.4.4" + }, + "domain": "enterprise-attack" + } + + example_layer_location = "/path/to/layer/file.json" + example_layer_out_location = "/path/to/new/layer/file.json" + + from mitreattack.navlayers.core import Layer + + layer1 = Layer(example_layer3_dict) # Create a new layer and load existing data + layer1.to_file(example_layer_out_location) # Write out the loaded layer to the specified file + + layer2 = Layer() # Create a new layer object + layer2.from_dict(example_layer4_dict) # Load layer data into existing layer object + print(layer2.to_dict()) # Retrieve the loaded layer's data as a dictionary, and print it + + layer3 = Layer() # Create a new layer object + layer3.from_file(example_layer_location) # Load layer data from a file into existing layer object + + +**layerops.py** + +``Layerops.py`` provides the ``LayerOps`` class, which is a way to combine layer files in an automated way, using user defined lambda functions. +Each LayerOps instance, when created, ingests the provided lambda functions, and stores them for use. +An existing ``LayerOps`` class can be used to combine layer files according to the initialized lambda using the process method. +The breakdown of this two step process is documented in the table below, while examples of both the list and dictionary modes of operation can be found below. + +**LayerOps()** + +.. code-block:: python + + x = LayerOps(score=score, comment=comment, enabled=enabled, colors=colors, metadata=metadata, name=name, desc=desc, default_values=default_values) + + +Each of the _inputs_ takes a lambda function that will be used to combine technique object fields matching the parameter. +The one exception to this is _default_values_, which is an optional dictionary argument containing default values +to provide the lambda functions if techniques of the combined layers are missing them. + +**.process() Method** + +.. code-block:: python + + x.process(data, default_values=default_values) + + +The process method applies the lambda functions stored during initialization to the layer objects in _data_. +_data_ must be either a list or a dictionary of Layer objects, and is expected to match the format of the lambda equations provided during initialization. +``default_values`` is an optional dictionary argument that overrides the currently stored default values with new ones for this specific processing operation. + +**Example Usage** + +.. code-block:: python + + from mitreattack.navlayers.manipulators.layerops import LayerOps + from mitreattack.navlayers.core.layer import Layer + + demo = Layer() + demo.from_file("C:\Users\attack\Downloads\layer.json") + demo2 = Layer() + demo2.from_file("C:\Users\attack\Downloads\layer2.json") + demo3 = Layer() + demo3.from_file("C:\Users\attack\Downloads\layer3.json") + + # Example 1) Build a LayerOps object that takes a list and averages scores across the layers + lo = LayerOps(score=lambda x: sum(x) / len(x), + name=lambda x: x[1], + desc=lambda x: "This is an list example") # Build LayerOps object + out_layer = lo.process([demo, demo2]) # Trigger processing on a list of demo and demo2 layers + out_layer.to_file("C:\demo_layer1.json") # Save averaged layer to file + out_layer2 = lo.process([demo, demo2, demo3]) # Trigger processing on a list of demo, demo2, demo3 + visual_aid = out_layer2.to_dict() # Retrieve dictionary representation of processed layer + + # Example 2) Build a LayerOps object that takes a dictionary and averages scores across the layers + lo2 = LayerOps(score=lambda x: sum([x[y] for y in x]) / len([x[y] for y in x]), + colors=lambda x: x['b'], + desc=lambda x: "This is a dict example") # Build LayerOps object, with lambda + out_layer3 = lo2.process({'a': demo, 'b': demo2}) # Trigger processing on a dictionary of demo and demo2 + dict_layer = out_layer3.to_dict() # Retrieve dictionary representation of processed layer + print(dict_layer) # Display retrieved dictionary + out_layer4 = lo2.process({'a': demo, 'b': demo2, 'c': demo3})# Trigger processing on a dictionary of demo, demo2, demo3 + out_layer4.to_file("C:\demo_layer4.json") # Save averaged layer to file + + # Example 3) Build a LayerOps object that takes a single element dictionary and inverts the score + lo3 = LayerOps(score=lambda x: 100 - x['a'], + desc= lambda x: "This is a simple example") # Build LayerOps object to invert score (0-100 scale) + out_layer5 = lo3.process({'a': demo}) # Trigger processing on dictionary of demo + print(out_layer5.to_dict()) # Display processed layer in dictionary form + out_layer5.to_file("C:\demo_layer5.json") # Save inverted score layer to file + + # Example 4) Build a LayerOps object that combines the comments from elements in the list, with custom defaults + lo4 = LayerOps(score=lambda x: '; '.join(x), + default_values= { + "comment": "This was an example of new default values" + }, + desc= lambda x: "This is a defaults example") # Build LayerOps object to combine descriptions, defaults + out_layer6 = lo4.process([demo2, demo3]) # Trigger processing on a list of demo2 and demo0 + out_layer6.to_file("C:\demo_layer6.json") # Save combined comment layer to file + + +to_excel.py +------------------------ + +``to_excel.py`` provides the ``ToExcel`` class, which is a way to export an existing layer file as an Excel spreadsheet. +The ``ToExcel`` class has an optional parameter for the initialization function, that tells the exporter what data source to use when building the output matrix. +Valid options include using live data from cti-taxii.mitre.org, using a local STIX bundle, or retrieving data from an ATT&CK Workbench instance. + +**ToExcel()** + +.. code-block:: python + + x = ToExcel(domain='enterprise', source='taxii', resource=None) + + +The ``ToExcel`` constructor takes domain, server, and resource arguments during instantiation. +The domain can be either ``enterprise`` or ``mobile``, and can be pulled directly from a layer file as ``layer.domain``. +The source argument tells the matrix generation tool which data source to use when building the matrix. +``taxii`` indicates that the tool should utilize the official ATT&CK Taxii Server (``cti-taxii``) when building the matrix, +while the ``local`` option indicates that it should use a local bundle, and the ``remote`` option indicates that +it should utilize a remote ATT&CK Workbench instance. +The ``resource`` argument is only required if the source is set to ``local``, in which case it should be a path +to a local stix bundle, or if the source is set to ``remote``, in which case it should be the url of a ATT&CK workbench instance. + +**.to_xlsx() Method** + +.. code-block:: python + + x.to_xlsx(layerInit=layer, filepath="layer.xlsx") + + +The ``to_xlsx`` method exports the layer file referenced as ``layer``, as an excel file to the ``filepath`` specified. + +Example Usage +------------------------ + +.. code-block:: python + + from mitreattack.navlayers import Layer + from mitreattack.navlayers import ToExcel + + lay = Layer() + lay.from_file("path/to/layer/file.json") + # Using taxii server for template + t = ToExcel(domain=lay.layer.domain, source='taxii') + t.to_xlsx(layerInit=lay, filepath="demo.xlsx") + # Using local stix data for template + t2 = ToExcel(domain='mobile', source='local', resource='path/to/local/stix.json') + t2.to_xlsx(layerInit=lay, filepath="demo2.xlsx") + # Using remote ATT&CK Workbench instance for template + workbench_url = 'localhost:3000' + t3 = ToExcel(domain='ics', source='remote', resource=workbench_url) + + +to_svg.py +------------------------ + +``to_svg.py`` provides the ``ToSvg`` class, which is a way to export an existing layer file as an SVG image file. +The ``ToSvg`` class, like the ``ToExcel`` class, has an optional parameter for the initialization function, +that tells the exporter what data source to use when building the output matrix. +Valid options include using live data from cti-taxii.mitre.org, using a local STIX bundle, or utilizing a remote ATT&CK Workbench instance. + +**ToSvg()** + +.. code-block:: python + + x = ToSvg(domain='enterprise', source='taxii', resource=None, config=None) + + +The ``ToSvg`` constructor, just like the ``ToExcel`` constructor, takes domain, server, and resource arguments during instantiation. +The domain can be either ``enterprise`` or ``mobile``, and can be pulled directly from a layer file as ``layer.domain``. +The source argument tells the matrix generation tool which data source to use when building the matrix. +``taxii`` indicates that the tool should utilize the ``cti-taxii`` server when building the matrix, +while the ``local`` option indicates that it should use a local bundle, and the ``remote`` option indicates that it should utilize a remote ATT&CK Workbench instance. +The ``resource`` argument is only required if the source is set to ``local``, in which case it should be a path to a local stix bundle, +or if the source is set to ``remote``, in which case it should be the url of an ATT&CK Workbench instance. +The ``config`` parameter is an optional ``SVGConfig`` object that can be used to configure the export as desired. +If not provided, the configuration for the export will be set to default values. + +**SVGConfig()** + +.. code-block:: python + + y = SVGConfig(width=8.5, height=11, headerHeight=1, unit="in", showSubtechniques="expanded", + font="sans-serif", tableBorderColor="#6B7279", showHeader=True, legendDocked=True, + legendX=0, legendY=0, legendWidth=2, legendHeight=1, showLegend=True, showFilters=True, + showAbout=True, showDomain=True, border=0.104) + + +The ``SVGConfig`` object is used to configure how an SVG export behaves. +The defaults for each of the available values can be found in the declaration above, and a brief explanation for each field is included in the table below. +The config object should be provided to the ``ToSvg`` object during instantiation, but if values need to be updated on the fly, +the currently loaded configuration can be interacted with at ``ToSvg().config``. +The configuration can also be populated from a json file using the ``.load_from_file(filename="path/to/file.json")`` method, +or stored to one using the ``.save_to_file(filename="path/to/file.json)`` method. + +.. list-table:: + :widths: 25 25 25 25 + :header-rows: 1 + + * - attribute + - description + - type + - default value + * - width + - Desired SVG width + - number + - 8.5 + * - height + - Desired SVG height + - number + - 11 + * - headerHeight + - Desired Header Block height + - number + - 1 + * - unit + - SVG measurement units (qualifies width, height, etc.) - "in", "cm", "px", "em", or "pt" + - string + - "in" + * - showSubtechniques + - Display form for subtechniques - "all", "expanded" (decided by layer), or "none" + - string + - "expanded" + * - font + - What font style to use - "serif", "sans-serif", or "monospace" + - string + - "sans-serif" + * - tableBorderColor + - Hex color to use for the technique borders + - string + - "#6B7279" + * - showHeader + - Whether or not to show Header Blocks + - bool + - True + * - legendDocked + - Whether or not the legend should be docked + - bool + - True + * - legendX + - Where to place the legend on the x axis if not docked + - number + - 0 + * - legendY + - Where to place the legend on the y axis if not docked + - number + - 1 + * - legendWidth + - Width of the legend if not docked + - number + - 2 + * - legendHeight + - Height of the legend if not docked + - number + - 1 + * - showLegend + - Whether or not to show the legend + - bool + - True + * - showFilters + - Whether or not to show the Filter Header Block + - bool + - True + * - showDomain + - Whether or not to show the Domain and Version Header Block + - bool + - True + * - showAbout + - Whether or not to show the About Header Block + - bool + - True + * - border + - What default border width to use + - number + - 0.104 + +**.to_svg() Method** + +.. code-block:: python + + x.to_svg(layerInit=layer, filepath="layer.svg") + +The ``to_svg`` method exports the layer file referenced as ``layer``, as an excel file to the ``filepath`` specified. + +Example Usage +-------------- + +.. code-block:: python + + from mitreattack.navlayers import Layer + from mitreattack.navlayers import ToSvg, SVGConfig + + lay = Layer() + lay.from_file("path/to/layer/file.json") + # Using taxii server for template + t = ToSvg(domain=lay.layer.domain, source='taxii') + t.to_svg(layerInit=lay, filepath="demo.svg") + #Using local stix data for template + + conf = SVGConfig() + conf.load_from_file(filename="path/to/poster/config.json") + + t2 = ToSvg(domain='mobile', source='local', resource='path/to/local/stix.json', config=conf) + t2.to_svg(layerInit=lay, filepath="demo2.svg") + + workbench_url = "localhost:3000" + t3 = ToSvg(domain='enterprise', source='remote', resource=workbench_url, config=conf) + t3.to_svg(layerInit=lay, filepath="demo3.svg") + + +overview_generator.py +------------------------ + +``overview_generator.py`` provides the ``OverviewLayerGenerator`` class, which is designed to allow users to +generate an ATT&CK layer that, on a per technique basis, has a score that corresponds to all instances +of the specified ATT&CK object type (group, mitigation, etc.), and a comment that lists all matching instance. + +**OverviewLayerGenerator()** + +.. code-block:: python + + x = OverviewLayerGenerator(source='taxii', domain='enterprise', resource=None) + + +The initialization function for ``OverviewLayerGenerator``, like ``ToSVG`` and ``ToExcel``, requires the specification of where +to retrieve data from (taxii server etc.). +The domain can be either ``enterprise``, ``mobile``, or ``ics``, and can be pulled directly from a layer file as ``layer.domain``. +The source argument tells the matrix generation tool which data source to use when building the matrix. +``taxii`` indicates that the tool should utilize the ``cti-taxii`` server when building the matrix, +while the ``local`` option indicates that it should use a local bundle, and the ``remote`` option indicates that it should utilize a remote ATT&CK Workbench instance. +The ``resource`` argument is only required if the source is set to ``local``, in which case it should be a path to a local stix bundle, +or if the source is set to ``remote``, in which case it should be the url of an ATT&CK Workbench instance. +If not provided, the configuration for the generator will be set to default values. + +**.generate_layer()** + +.. code-block:: python + x.generate_layer(obj_type=object_type_name) + + +The ``generate_layer`` function generates a layer, customized to the input ``object_type_name``. +Valid values include ``group``, ``mitigation``, ``software``, and ``datasource``. + +usage_generator.py +------------------------ + +``usage_ generator.py`` provides the ``UsageLayerGenerator`` class, which is designed to allow users to +generate an ATT&CK layer that scores any relevant techniques that a given input ATT&CK object has. +These objects can be any ``group``, ``software``, ``mitigation``, or ``data component``, +and can be referenced by ID or by any alias when provided to the generator. + +**UsageLayerGenerator()** + +.. code-block:: python + x = UsageLayerGenerator(source='taxii', domain='enterprise', resource=None) + + +The initialization function for ``UsageLayerGenerator``, like ``ToSVG`` and ``ToExcel``, requires the specification of where +to retrieve data from (taxii server etc.). +The domain can be either ``enterprise``, ``mobile``, or ``ics``, and can be pulled directly from a layer file as ``layer.domain``. +The source argument tells the matrix generation tool which data source to use when building the matrix. +``taxii`` indicates that the tool should utilize the ``cti-taxii`` server when building the matrix, +while the ``local`` option indicates that it should use a local bundle, and the ``remote`` option indicates that it should utilize a remote ATT&CK Workbench instance. +The ``resource`` argument is only required if the source is set to ``local``, in which case it should be a path to a local stix bundle, +or if the source is set to ``remote``, in which case it should be the url of an ATT&CK Workbench instance. +If not provided, the configuration for the generator will be set to default values. + +**.generate_layer()** + +.. code-block:: python + + x.generate_layer(match=object_identifier) + + +The ``generate_layer`` function generates a layer, customized to the input ``object_identifier``. +Valid values include ``ATT&CK ID``, ``name``, or any known ``alias`` for ``group``, ``mitigation``, ``software``, and ``data component`` objects within the selected ATT&CK data. + +.. code-block:: python + + from mitreattack.navlayers import UsageLayerGenerator + + handle = UsageLayerGenerator(source='taxii', domain='enterprise') + + layer1 = handle.generate_layer(match='G0018') + layer2 = handle.generate_layer(match='Adups') + + +sum_generator.py +------------------------ + +``sum_generator.py`` provides the ``SumLayerGenerator`` class, which is designed to allow users to +generate a collection of ATT&CK layers that, on a per technique basis, have a score that corresponds to all instances +of the specified ATT&CK object type (group, mitigation, etc.), and a comment that lists all matching instance. +Each one of the generated layers will correspond to a single instance of the specified ATT&CK object type. + +**SumLayerGenerator()** + +.. code-block:: python + + x = SumLayerGenerator(source='taxii', domain='enterprise', resource=None) + + +The initialization function for ``SumGeneratorLayer``, like ``ToSVG`` and ``ToExcel``, requires the specification of where +to retrieve data from (taxii server etc.). +The domain can be either ``enterprise``, ``mobile``, or ``ics``, and can be pulled directly from a layer file as ``layer.domain``. +The source argument tells the matrix generation tool which data source to use when building the matrix. +``taxii`` indicates that the tool should utilize the ``cti-taxii`` server when building the matrix, +while the ``local`` option indicates that it should use a local bundle, and the ``remote`` option indicates that it should utilize a remote ATT&CK Workbench instance. +The ``resource`` argument is only required if the source is set to ``local``, in which case it should be a path to a local stix bundle, +or if the source is set to ``remote``, in which case it should be the url of an ATT&CK Workbench instance. +If not provided, the configuration for the generator will be set to default values. + +**.generate_layer()** + +.. code-block:: python + + x.generate_layer(layers_type=object_type_name) + + +The ``generate_layer`` function generates a collection of layers, each customized to one instance of the input ``object_type_name``. +Valid types include ``group``, ``mitigation``, ``software``, and ``datasource``. + +layerExporter_cli.py +------------------------ + +This command line tool allows users to convert a `navigator `_ +layer file to either an svg image or excel file using the functionality provided by the navlayers module. +Details about the SVG configuration json mentioned below can be found in the +`SVGConfig `_ +entry within the navlayers module documentation. + +.. code:: bash + + C:\Users\attack>layerExporter_cli -h + usage: layerExporter_cli [-h] -m {svg,excel} [-s {taxii,local,remote}] + [--resource RESOURCE] -o OUTPUT [OUTPUT ...] + [-l LOAD_SETTINGS] [-d WIDTH HEIGHT] + input [input ...] + + Export an ATT&CK Navigator layer as a svg image or excel file + + positional arguments: + input Path(s) to the file to export + + optional arguments: + -h, --help show this help message and exit + -m {svg,excel}, --mode {svg,excel} + The form to export the layers in + -s {taxii,local,remote}, --source {taxii,local,remote} + What source to utilize when building the matrix + --resource RESOURCE Path to the local resource if --source=local, or url + of an ATT&CK Workbench instance if --source=remote + -o OUTPUT [OUTPUT ...], --output OUTPUT [OUTPUT ...] + Path(s) to the exported svg/xlsx file + -l LOAD_SETTINGS, --load_settings LOAD_SETTINGS + [SVG Only] Path to a SVG configuration json to use + when rendering + -d WIDTH HEIGHT, --size WIDTH HEIGHT + [SVG Only] X and Y size values (in inches) for SVG + export (use -l for other settings) + + C:\Users\attack>layerExporter_cli -m svg -s taxii -l settings/config.json -o output/svg1.json output/svg2.json files/layer1.json files/layer2.json + + +layerGenerator_cli.py +------------------------ + +This command line tool allows users to generate `ATT&CK Navigator `_ +layer files from either a specific group, software, or mitigation. Alternatively, users can generate a layer file with a +mapping to all associated groups, software, or mitigations across the techniques within ATT&CK. + +.. code:: bash + + C:\Users\attack>layerGenerator_cli -h + usage: layerGenerator_cli [-h] + (--overview-type {group,software,mitigation,datasource} | --mapped-to MAPPED_TO | --batch-type {group,software,mitigation,datasource}) + [-o OUTPUT] [--domain {enterprise,mobile,ics}] + [--source {taxii,local,remote}] + [--resource RESOURCE] + + Generate an ATT&CK Navigator layer + + optional arguments: + -h, --help show this help message and exit + --overview-type {group,software,mitigation,datasource} + Output a layer file where the target type is + summarized across the entire dataset. + --mapped-to MAPPED_TO + Output layer file with techniques mapped to the given + group, software, mitigation, or data component. Argument + can be name, associated group/software, or ATT&CK ID. + --batch-type {group,software,mitigation,datasource} + Output a collection of layer files to the specified + folder, each one representing a different instance of + the target type. + -o OUTPUT, --output OUTPUT + Path to the output layer file/directory + --domain {enterprise,mobile,ics} + Which domain to build off of + --source {taxii,local,remote} + What source to utilize when building the layer files + --resource RESOURCE Path to the local resource if --source=local, or url + of an ATT&CK Workbench instance if --source=remote + + C:\Users\attack>layerGenerator_cli --domain enterprise --source taxii --mapped-to S0065 --output generated_layer.json + C:\Users\attack>layerGenerator_cli --domain mobile --source taxii --overview-type mitigation --output generated_layer2.json + C:\Users\attack>layerGenerator_cli --domain ics --source taxii --batch-type software + C:\Users\attack>layerGenerator_cli --domain enterprise --source taxii --overview-type datasource --output generated_layer3.json diff --git a/docs/related_work.rst b/docs/related_work.rst index 9050f901..c536fabc 100644 --- a/docs/related_work.rst +++ b/docs/related_work.rst @@ -16,7 +16,7 @@ phases of an adversary's lifecycle, and the platforms they are known to target. for understanding security risk against known adversary behavior, for planning security improvements, and verifying defenses work as expected. -https://attack.mitre.org +Visit `the ATT&CK website `_ to browse our curated knowledge base. STIX ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/stix_primer/access-attack.rst b/docs/stix_primer/access-attack.rst new file mode 100644 index 00000000..11aa12f9 --- /dev/null +++ b/docs/stix_primer/access-attack.rst @@ -0,0 +1,218 @@ +Accessing ATT&CK data in python +=============== + +There are several ways to acquire the ATT&CK data in Python. All of them will provide an object +implementing the DataStore API and can be used interchangeably with the recipes provided in the `STIX recipes `_ section. + +This section utilizes the `stix2 python library `_ . Please refer to the `STIX2 Python API Documentation `_ for more information on how to work with STIX programmatically. + +Requirements and imports +---------------------- + +Before installing requirements, we recommend setting up a virtual environment: + +1. Create virtual environment: + - macOS and Linux: ``python3 -m venv env`` + - Windows: ``py -m venv env`` +2. Activate the virtual environment: + - macOS and Linux: ``source env/bin/activate`` + - Windows: ``env/Scripts/activate.bat`` + +stix2 +---------------------- + +stix2 can be installed by following the instructions on their `repository `_ . Imports for the recipes in this repository can be done from the base package, for example: + +.. code-block:: python + + from stix2 import Filter + + +However, if you are aiming to extend the ATT&CK dataset with new objects or implement complex workflows, you may need to use the `v20` specifier for some imports. This ensures that the objects use the STIX 2.0 API instead of the STIX 2.1 API. For example: + +.. code-block:: python + + from stix2.v20 import AttackPattern + + +You can see a full list of the classes which have versioned imports `here `_ . + +taxii2client +------------- + +taxii2-client can be installed by following the instructions on their `repository `_ . The ATT&CK TAXII server implements the 2.0 version of the TAXII specification, but the default import of ``taxii2client`` (version 2.0.0 and above) uses the 2.1 version of the TAXII specification, which can lead to 406 responses when connecting to our TAXII server if not accounted for. + +If the TAXII Client is getting a 406 Response, make sure you are running the latest version (``pip install --upgrade stix2` or `pip install --upgrade taxii2-client``). In addition, make sure you are running the 2.0 version of the client (using the `v20` import) as shown below in order to communicate with the ATT&CK TAXII 2.0 Server. + +.. code-block:: python + + from taxii2client.v20 import Collection + + +Access local content +-------------------- + +Many users may opt to access the ATT&CK content via a local copy of the STIX data on this repo. This can be advantageous for several reasons: + +- Doesn't require internet access after the initial download +- User can modify the ATT&CK content if desired +- Downloaded copy is static, so updates to the ATT&CK catalog won't cause bugs in automated workflows. User can still manually update by cloning a fresh version of the data + +Access via FileSystemSource +--------------------------- + +Each domain in this repo is formatted according to the `STIX2 FileSystem spec `_ . +Therefore you can use a ``FileSystemSource`` to load a domain, for example to load the enterprise-attack domain: + +.. code-block:: python + + from stix2 import FileSystemSource + +src = FileSystemSource('./cti/enterprise-attack') + + +Access via bundle +------------------ + +If you instead prefer to download just the domain bundle, e.g `enterprise-attack.json `_ , you can still load this using a MemoryStore: + +.. code-block:: python + + from stix2 import MemoryStore + + src = MemoryStore() + src.load_from_file("enterprise-attack.json") + + +Access live content +------------------- + +Some users may instead prefer to access "live" ATT&CK content over the internet. This is advantageous for several reasons: + +- Always stays up to date with the evolving ATT&CK catalog +- Doesn't require an initial download of the ATT&CK content, generally requires less setup + +Access from the ATT&CK TAXII server +---------------------------------- + +Users can access the ATT&CK data from the official ATT&CK TAXII server. In TAXII, the ATT&CK domains are represented as collections with static IDs: + +.. list-table:: + :widths: 50 50 + :header-rows: 1 + + * - domain + - collection ID + * - `enterprise-attack` + - `95ecc380-afe9-11e4-9b6c-751b66dd541e` + * - `mobile-attack` + - `2f669986-b40b-4423-b720-4396ca6a462b` + * - `ics-attack` + - `02c3ef24-9cd4-48f3-a99f-b74ce24f1d34` + +You can also get a list of available collection from the server directly: + +.. code-block:: python + + from taxii2client.v20 import Server # only specify v20 if your installed version is >= 2.0.0 + + server = Server("https://cti-taxii.mitre.org/taxii/") + api_root = server.api_roots[0] + # Print name and ID of all ATT&CK domains available as collections + for collection in api_root.collections: + print(collection.title.ljust(20) + collection.id) + + +The following recipe demonstrates how to access the enterprise-attack data from the TAXII server. + +.. code-block:: python + + from stix2 import TAXIICollectionSource + from taxii2client.v20 import Collection # only specify v20 if your installed version is >= 2.0.0 + + collections = { + "enterprise_attack": "95ecc380-afe9-11e4-9b6c-751b66dd541e", + "mobile_attack": "2f669986-b40b-4423-b720-4396ca6a462b", + "ics-attack": "02c3ef24-9cd4-48f3-a99f-b74ce24f1d34" + } + + collection = Collection(f"https://cti-taxii.mitre.org/stix/collections/{collections['enterprise_attack']}/") + src = TAXIICollectionSource(collection) + + +For more about TAXII, please see oasis-open's `Introduction to TAXII `_ . + +Access from Github via requests +---------------------- + +Users can alternatively access the data from MITRE/CTI using HTTP requests, and load the resulting content into a MemoryStore. +While typically the TAXII method is more desirable for "live" access, this method can be useful if you want to +access data on a branch of the MITRE/CTI repo (the TAXII server only holds the master branch) or in the case of a TAXII server outage. + +.. code-block:: python + + import requests + from stix2 import MemoryStore + + def get_data_from_branch(domain, branch="master"): + """get the ATT&CK STIX data from MITRE/CTI. Domain should be 'enterprise-attack', 'mobile-attack' or 'ics-attack'. Branch should typically be master.""" + stix_json = requests.get(f"https://raw.githubusercontent.com/mitre/cti/{branch}/{domain}/{domain}.json").json() + return MemoryStore(stix_data=stix_json["objects"]) + + src = get_data_from_branch("enterprise-attack") + + +Access a specific version of ATT&CK +---------------------- + +ATT&CK versions are tracked on the MITRE/CTI repo using `tags `_ . Tags prefixed with ``ATT&CK-v`` correspond to ATT&CK versions and tags prefixed with ``CAPEC-v`` correspond to CAPEC versions. You can find more information about ATT&CK versions on the `versions of ATT&CK page `_ on the ATT&CK website. + +In addition to checking out the repo under the tag for a given version or downloading the STIX from github using your browser, you can also use a variation on the :ref:`requests method` to access a particular version of ATT&CK: + +.. code-block:: python + + import requests + from stix2 import MemoryStore + + def get_data_from_version(domain, version): + """get the ATT&CK STIX data for the given version from MITRE/CTI. Domain should be 'enterprise-attack', 'mobile-attack' or 'ics-attack'.""" + stix_json = requests.get(f"https://raw.githubusercontent.com/mitre/cti/ATT%26CK-v{version}/{domain}/{domain}.json").json() + return MemoryStore(stix_data=stix_json["objects"]) + + src = get_data_from_version("enterprise-attack", "5.2") + + +You can get a list of ATT&CK versions programmatically using the github API: + +.. code-block:: python + + import requests + import re + + refToTag = re.compile(r"ATT&CK-v(.*)") + tags = requests.get("https://api.github.com/repos/mitre/cti/git/refs/tags").json() + versions = list(map(lambda tag: refToTag.search(tag["ref"]).groups()[0] , filter(lambda tag: "ATT&CK-v" in tag["ref"], tags))) + # versions = ["1.0", "2.0", ...] + + +Access multiple domains simultaneously +---------------------- + +Because ATT&CK is stored in multiple domains (as of this writing, enterprise-attack, mobile-attack and ics-attack), the above methodologies will only allow you to work +with a single domain at a time. While oftentimes the hard separation of domains is advantageous, occasionally it is useful to combine +domains into a single DataStore. Use any of the methods above to acquire the individual datastores, and then use the following approach to combine them into +a single CompositeDataSource: + +.. code-block:: python + + from stix2 import CompositeDataSource + + src = CompositeDataSource() + src.add_data_sources([enterprise_attack_src, mobile_attack_src, ics_attack_src]) + + +You can then use this CompositeDataSource just as you would the DataSource for an individual domain. + + + + diff --git a/docs/stix_primer/attack_id.rst b/docs/stix_primer/attack_id.rst new file mode 100644 index 00000000..eea08d73 --- /dev/null +++ b/docs/stix_primer/attack_id.rst @@ -0,0 +1,24 @@ +By ATT&CK ID +=============== + +The following recipe can be used to retrieve an object according to its ATT&CK ID: + +.. code-block:: python + + from stix2 import Filter + + g0075 = src.query([ Filter("external_references.external_id", "=", "G0075") ])[0] + + +Note: in prior versions of ATT&CK, mitigations had 1:1 relationships with techniques and shared their technique's ID. Therefore the above method does not work properly for techniques because technique ATTT&CK IDs are not truly unique. By specifying the STIX type you're looking for as ``attack-pattern`` you can avoid this issue. + +.. code-block:: python + + from stix2 import Filter + + t1134 = src.query([ + Filter("external_references.external_id", "=", "T1134"), + Filter("type", "=", "attack-pattern") + ])[0] + +The old 1:1 mitigations causing this issue are deprecated, so you can also filter them out that way — see `Removing revoked and deprecated objects `_ . \ No newline at end of file diff --git a/docs/stix_primer/by_alias.rst b/docs/stix_primer/by_alias.rst new file mode 100644 index 00000000..8b01bc4d --- /dev/null +++ b/docs/stix_primer/by_alias.rst @@ -0,0 +1,16 @@ +By Alias +=============== + +The following methodology can be used to find the group corresponding to a given alias: + +.. code-block:: python + + from stix2 import Filter + + def get_group_by_alias(thesrc, alias): + return thesrc.query([ + Filter('type', '=', 'intrusion-set'), + Filter('aliases', '=', alias) + ])[0] + + get_group_by_alias(src, 'Cozy Bear') diff --git a/docs/stix_primer/by_name.rst b/docs/stix_primer/by_name.rst new file mode 100644 index 00000000..63bde04a --- /dev/null +++ b/docs/stix_primer/by_name.rst @@ -0,0 +1,18 @@ +By Name +=============== + + +The following recipe retrieves an object according to its name: + +.. code-block:: python + + from stix2 import Filter + + def get_technique_by_name(thesrc, name): + filt = [ + Filter('type', '=', 'attack-pattern'), + Filter('name', '=', name) + ] + return thesrc.query(filt) + # get the technique titled "System Information Discovery" + get_technique_by_name(src, 'System Information Discovery') diff --git a/docs/stix_primer/deprecated_revoked.rst b/docs/stix_primer/deprecated_revoked.rst new file mode 100644 index 00000000..bbb5bcf4 --- /dev/null +++ b/docs/stix_primer/deprecated_revoked.rst @@ -0,0 +1,6 @@ +Working with deprecated and revoked objects +=============== + +Objects that are deemed no longer beneficial to track as part of the knowledge base are marked as deprecated, and objects which are replaced by a different object are revoked. In both cases, the old object is marked with a field (either ``x_mitre_deprecated`` or ``revoked``) noting their status. In the case of revoked objects, a relationship of type ``revoked-by`` is also created targeting the replacing object. + +Unlike other objects in the dataset, relationships cannot be revoked or deprecated. Relationships are considered deprecated/revoked if one of the objects it is attached to is revoked or deprecated. \ No newline at end of file diff --git a/docs/stix_primer/getting_related_objects.rst b/docs/stix_primer/getting_related_objects.rst new file mode 100644 index 00000000..8ca9edef --- /dev/null +++ b/docs/stix_primer/getting_related_objects.rst @@ -0,0 +1,7 @@ +Getting related objects +=============== + +A large part of working with ATT&CK revolves around parsing relationships between objects. It is useful +to track not only the related object but the relationship itself because a description is often +present to contextualize the nature of the relationship. The following recipes demonstrate +some common uses of relationships. \ No newline at end of file diff --git a/docs/stix_primer/getting_revoked_object.rst b/docs/stix_primer/getting_revoked_object.rst new file mode 100644 index 00000000..333cad8d --- /dev/null +++ b/docs/stix_primer/getting_revoked_object.rst @@ -0,0 +1,21 @@ +Getting a revoking object +=============== + +When an object is replaced by another object, it is marked with the field ``revoked`` and a relationship of type ``revoked-by`` is created where the ``source_ref`` is the revoked object and the ``target_ref`` is the revoking object. This relationship can be followed to find the replacing object: + +.. code-block:: python + + from stix2 import Filter + + def getRevokedBy(stix_id, thesrc): + relations = thesrc.relationships(stix_id, 'revoked-by', source_only=True) + revoked_by = thesrc.query([ + Filter('id', 'in', [r.target_ref for r in relations]), + Filter('revoked', '=', False) + ]) + if revoked_by is not None: + revoked_by = revoked_by[0] + + return revoked_by + + getRevokedBy("attack-pattern--c16e5409-ee53-4d79-afdc-4099dc9292df", src) diff --git a/docs/stix_primer/getting_software.rst b/docs/stix_primer/getting_software.rst new file mode 100644 index 00000000..6308db32 --- /dev/null +++ b/docs/stix_primer/getting_software.rst @@ -0,0 +1,19 @@ +Getting software +=============== + +Because software are the union of two STIX types (`tool` and `malware`), the process for accessing software is slightly more complicated. + +.. code-block:: python + + from itertools import chain + from stix2 import Filter + + def get_software(thesrc): + return list(chain.from_iterable( + thesrc.query(f) for f in [ + Filter("type", "=", "tool"), + Filter("type", "=", "malware") + ] + )) + + get_software(src) diff --git a/docs/stix_primer/multiple_objects.rst b/docs/stix_primer/multiple_objects.rst new file mode 100644 index 00000000..4aface21 --- /dev/null +++ b/docs/stix_primer/multiple_objects.rst @@ -0,0 +1,6 @@ +Getting multiple objects +=============== + +The recipes in this following section address how to query the dataset for multiple objects. + +When working with queries to return objects based on a set of characteristics, it is likely that you'll end up with a few objects which are no longer maintained by ATT&CK. These are objects marked as deprecated or revoked. We keep these outdated objects around so that workflows depending on them don't break, but we recommend you avoid using them when possible. Please see the section [Working with deprecated and revoked objects](#Working-with-deprecated-and-revoked-objects) for more information. \ No newline at end of file diff --git a/docs/stix_primer/objects_by_content.rst b/docs/stix_primer/objects_by_content.rst new file mode 100644 index 00000000..ccbb25a6 --- /dev/null +++ b/docs/stix_primer/objects_by_content.rst @@ -0,0 +1,15 @@ +Objects by content +=============== + +Sometimes it may be useful to query objects by the content of their description: + +.. code-block:: python + + from stix2 import Filter + + def get_techniques_by_content(thesrc, content): + techniques = src.query([ Filter('type', '=', 'attack-pattern') ]) + return list(filter(lambda t: content.lower() in t.description.lower(), techniques)) + + # Get all techniques where the string LSASS appears in the description + get_techniques_by_content(src, 'LSASS') diff --git a/docs/stix_primer/objects_by_type.rst b/docs/stix_primer/objects_by_type.rst new file mode 100644 index 00000000..641ff7c4 --- /dev/null +++ b/docs/stix_primer/objects_by_type.rst @@ -0,0 +1,11 @@ +Objects by type +=============== + +See `The ATT&CK data model `_ for mappings of ATT&CK type to STIX type. + +.. code-block:: python + + from stix2 import Filter + + # use the appropriate STIX type in the query according to the desired ATT&CK type + groups = src.query([ Filter("type", "=", "intrusion-set") ]) diff --git a/docs/stix_primer/objects_since_date.rst b/docs/stix_primer/objects_since_date.rst new file mode 100644 index 00000000..d71cbfef --- /dev/null +++ b/docs/stix_primer/objects_since_date.rst @@ -0,0 +1,28 @@ +Objects created or modified since a given date +=============== + +Sometimes you may want to get a list of objects which have been created or modified after a certain time. + +.. code-block:: python + + from stix2 import Filter + + def get_created_after(thesrc, timestamp): + filt = [ + Filter('created', '>', timestamp) + ] + return thesrc.query(filt) + + get_created_after(src, "2018-10-01T00:14:20.652Z") + + + def get_modified_after(thesrc, timestamp): + filt = [ + Filter('modified', '>', timestamp) + ] + return thesrc.query(filt) + + get_modified_after(src, "2018-10-01T00:14:20.652Z") + + +We don't recommend you use this method to detect a change to the contents of the knowledge base. For detecting an update to the overall knowledge base we recommend using requests to `check the list of released versions of ATT&CK `_ . diff --git a/docs/stix_primer/overview.rst b/docs/stix_primer/overview.rst new file mode 100644 index 00000000..ec1943dc --- /dev/null +++ b/docs/stix_primer/overview.rst @@ -0,0 +1,32 @@ +Overview +=============== + + +This is a short primer on how to manipulate STIX data from ATT&CK. While not comprehensive, it will provide you with a quick starting guide. +You can go to the `Access ATTACK `_ page to find out how to get started. + +.. toctree:: + :maxdepth: 1 + :caption: Table of Contents + + access-attack + stix-recipes + attack_id + by_alias + by_name + deprecated_revoked + getting_related_objects + getting_software + multiple_objects + objects_by_content + objects_by_type + objects_since_date + stix_id + tactics_by_matrix + techniques_by_platform + techniques_by_tactic + techniques_subtechniques + relationships_microlibrary + techniques_by_group_sw + remove_revoked_deprecated + getting_revoked_objects \ No newline at end of file diff --git a/docs/stix_primer/relationships_microlibrary.rst b/docs/stix_primer/relationships_microlibrary.rst new file mode 100644 index 00000000..d3817bf8 --- /dev/null +++ b/docs/stix_primer/relationships_microlibrary.rst @@ -0,0 +1,317 @@ +Relationships microlibrary +=============== + +NOTE: The following code is intended to be used with the ATT&CK v12 release which includes Campaign Objects. +The examples are backwards-compatible for previous versions af ATT&CK that omit those objects. + +This microlibrary can be used to build a lookup table of stixID to related objects and relationships. +The argument to each accessor function is a STIX2 MemoryStore to build the relationship mappings from. + +.. code-block:: python + + from pprint import pprint + from stix2 import MemoryStore, Filter + + # See section below on "Removing revoked and deprecated objects" + def remove_revoked_deprecated(stix_objects): + """Remove any revoked or deprecated objects from queries made to the data source""" + # Note we use .get() because the property may not be present in the JSON data. The default is False + # if the property is not set. + return list( + filter( + lambda x: x.get("x_mitre_deprecated", False) is False and x.get("revoked", False) is False, + stix_objects + ) + ) + + def get_related(thesrc, src_type, rel_type, target_type, reverse=False): + """build relationship mappings + params: + thesrc: MemoryStore to build relationship lookups for + src_type: source type for the relationships, e.g "attack-pattern" + rel_type: relationship type for the relationships, e.g "uses" + target_type: target type for the relationship, e.g "intrusion-set" + reverse: build reverse mapping of target to source + """ + + relationships = thesrc.query([ + Filter('type', '=', 'relationship'), + Filter('relationship_type', '=', rel_type), + Filter('revoked', '=', False), + ]) + + # See section below on "Removing revoked and deprecated objects" + relationships = remove_revoked_deprecated(relationships) + + # stix_id => [ { relationship, related_object_id } for each related object ] + id_to_related = {} + + # build the dict + for relationship in relationships: + if src_type in relationship.source_ref and target_type in relationship.target_ref: + if (relationship.source_ref in id_to_related and not reverse) or (relationship.target_ref in id_to_related and reverse): + # append to existing entry + if not reverse: + id_to_related[relationship.source_ref].append({ + "relationship": relationship, + "id": relationship.target_ref + }) + else: + id_to_related[relationship.target_ref].append({ + "relationship": relationship, + "id": relationship.source_ref + }) + else: + # create a new entry + if not reverse: + id_to_related[relationship.source_ref] = [{ + "relationship": relationship, + "id": relationship.target_ref + }] + else: + id_to_related[relationship.target_ref] = [{ + "relationship": relationship, + "id": relationship.source_ref + }] + # all objects of relevant type + if not reverse: + targets = thesrc.query([ + Filter('type', '=', target_type), + Filter('revoked', '=', False) + ]) + else: + targets = thesrc.query([ + Filter('type', '=', src_type), + Filter('revoked', '=', False) + ]) + + # build lookup of stixID to stix object + id_to_target = {} + for target in targets: + id_to_target[target.id] = target + + # build final output mappings + output = {} + for stix_id in id_to_related: + value = [] + for related in id_to_related[stix_id]: + if not related["id"] in id_to_target: + continue # targeting a revoked object + value.append({ + "object": id_to_target[related["id"]], + "relationship": related["relationship"] + }) + output[stix_id] = value + return output + + # software:group + def software_used_by_groups(thesrc): + """returns group_id => {software, relationship} for each software used by the group and each software used by campaigns attributed to the group.""" + # get all software used by groups + tools_used_by_group = get_related(thesrc, "intrusion-set", "uses", "tool") + malware_used_by_group = get_related(thesrc, "intrusion-set", "uses", "malware") + software_used_by_group = {**tools_used_by_group, **malware_used_by_group} # group_id -> [{software, relationship}] + + # get groups attributing to campaigns and all software used by campaigns + software_used_by_campaign = get_related(thesrc, "campaign", "uses", "tool") + malware_used_by_campaign = get_related(thesrc, "campaign", "uses", "malware") + for id in malware_used_by_campaign: + if id in software_used_by_campaign: + software_used_by_campaign[id].extend(malware_used_by_campaign[id]) + else: + software_used_by_campaign[id] = malware_used_by_campaign[id] + campaigns_attributed_to_group = { + "campaigns": get_related(thesrc, "campaign", "attributed-to", "intrusion-set", reverse=True), # group_id => {campaign, relationship} + "software": software_used_by_campaign # campaign_id => {software, relationship} + } + + for group_id in campaigns_attributed_to_group["campaigns"]: + software_used_by_campaigns = [] + # check if attributed campaign is using software + for campaign in campaigns_attributed_to_group["campaigns"][group_id]: + campaign_id = campaign["object"]["id"] + if campaign_id in campaigns_attributed_to_group["software"]: + software_used_by_campaigns.extend(campaigns_attributed_to_group["software"][campaign_id]) + + # update software used by group to include software used by a groups attributed campaign + if group_id in software_used_by_group: + software_used_by_group[group_id].extend(software_used_by_campaigns) + else: + software_used_by_group[group_id] = software_used_by_campaigns + return software_used_by_group + + def groups_using_software(thesrc): + """returns software_id => {group, relationship} for each group using the software and each software used by attributed campaigns.""" + # get all groups using software + groups_using_tool = get_related(thesrc, "intrusion-set", "uses", "tool", reverse=True) + groups_using_malware = get_related(thesrc, "intrusion-set", "uses", "malware", reverse=True) + groups_using_software = {**groups_using_tool, **groups_using_malware} # software_id => {group, relationship} + + # get campaigns attributed to groups and all campaigns using software + campaigns_using_software = get_related(thesrc, "campaign", "uses", "tool", reverse=True) + campaigns_using_malware = get_related(thesrc, "campaign", "uses", "malware", reverse=True) + for id in campaigns_using_malware: + if id in campaigns_using_software: + campaigns_using_software[id].extend(campaigns_using_malware[id]) + else: + campaigns_using_software[id] = campaigns_using_malware[id] + groups_attributing_to_campaigns = { + "campaigns": campaigns_using_software,# software_id => {campaign, relationship} + "groups": get_related(thesrc, "campaign", "attributed-to", "intrusion-set") # campaign_id => {group, relationship} + } + + for software_id in groups_attributing_to_campaigns["campaigns"]: + groups_attributed_to_campaigns = [] + # check if campaign is attributed to group + for campaign in groups_attributing_to_campaigns["campaigns"][software_id]: + campaign_id = campaign["object"]["id"] + if campaign_id in groups_attributing_to_campaigns["groups"]: + groups_attributed_to_campaigns.extend(groups_attributing_to_campaigns["groups"][campaign_id]) + + # update groups using software to include software used by a groups attributed campaign + if software_id in groups_using_software: + groups_using_software[software_id].extend(groups_attributed_to_campaigns) + else: + groups_using_software[software_id] = groups_attributed_to_campaigns + return groups_using_software + + # software:campaign + def software_used_by_campaigns(thesrc): + """returns campaign_id => {software, relationship} for each software used by the campaign.""" + tools_used_by_campaign = get_related(thesrc, "campaign", "uses", "tool") + malware_used_by_campaign = get_related(thesrc, "campaign", "uses", "malware") + return {**tools_used_by_campaign, **malware_used_by_campaign} + + def campaigns_using_software(thesrc): + """returns software_id => {campaign, relationship} for each campaign using the software.""" + campaigns_using_tool = get_related(thesrc, "campaign", "uses", "tool", reverse=True) + campaigns_using_malware = get_related(thesrc, "campaign", "uses", "malware", reverse=True) + return {**campaigns_using_tool, **campaigns_using_malware} + + # campaign:group + def groups_attributing_to_campaign(thesrc): + """returns campaign_id => {group, relationship} for each group attributing to the campaign.""" + return get_related(thesrc, "campaign", "attributed-to", "intrusion-set") + + def campaigns_attributed_to_group(thesrc): + """returns group_id => {campaign, relationship} for each campaign attributed to the group.""" + return get_related(thesrc, "campaign", "attributed-to", "intrusion-set", reverse=True) + + # technique:group + def techniques_used_by_groups(thesrc): + """returns group_id => {technique, relationship} for each technique used by the group and each + technique used by campaigns attributed to the group.""" + # get all techniques used by groups + techniques_used_by_groups = get_related(thesrc, "intrusion-set", "uses", "attack-pattern") # group_id => {technique, relationship} + + # get groups attributing to campaigns and all techniques used by campaigns + campaigns_attributed_to_group = { + "campaigns": get_related(thesrc, "campaign", "attributed-to", "intrusion-set", reverse=True), # group_id => {campaign, relationship} + "techniques": get_related(thesrc, "campaign", "uses", "attack-pattern") # campaign_id => {technique, relationship} + } + + for group_id in campaigns_attributed_to_group["campaigns"]: + techniques_used_by_campaigns = [] + # check if attributed campaign is using technique + for campaign in campaigns_attributed_to_group["campaigns"][group_id]: + campaign_id = campaign["object"]["id"] + if campaign_id in campaigns_attributed_to_group["techniques"]: + techniques_used_by_campaigns.extend(campaigns_attributed_to_group["techniques"][campaign_id]) + + # update techniques used by groups to include techniques used by a groups attributed campaign + if group_id in techniques_used_by_groups: + techniques_used_by_groups[group_id].extend(techniques_used_by_campaigns) + else: + techniques_used_by_groups[group_id] = techniques_used_by_campaigns + return techniques_used_by_groups + + def groups_using_technique(thesrc): + """returns technique_id => {group, relationship} for each group using the technique and each campaign attributed to groups using the technique.""" + # get all groups using techniques + groups_using_techniques = get_related(thesrc, "intrusion-set", "uses", "attack-pattern", reverse=True) # technique_id => {group, relationship} + + # get campaigns attributed to groups and all campaigns using techniques + groups_attributing_to_campaigns = { + "campaigns": get_related(thesrc, "campaign", "uses", "attack-pattern", reverse=True), # technique_id => {campaign, relationship} + "groups": get_related(thesrc, "campaign", "attributed-to", "intrusion-set") # campaign_id => {group, relationship} + } + + for technique_id in groups_attributing_to_campaigns["campaigns"]: + campaigns_attributed_to_group = [] + # check if campaign is attributed to group + for campaign in groups_attributing_to_campaigns["campaigns"][technique_id]: + campaign_id = campaign["object"]["id"] + if campaign_id in groups_attributing_to_campaigns["groups"]: + campaigns_attributed_to_group.extend(groups_attributing_to_campaigns["groups"][campaign_id]) + + # update groups using techniques to include techniques used by a groups attributed campaign + if technique_id in groups_using_techniques: + groups_using_techniques[technique_id].extend(campaigns_attributed_to_group) + else: + groups_using_techniques[technique_id] = campaigns_attributed_to_group + return groups_using_techniques + + # technique:campaign + def techniques_used_by_campaigns(thesrc): + """returns campaign_id => {technique, relationship} for each technique used by the campaign.""" + return get_related(thesrc, "campaign", "uses", "attack-pattern") + + def campaigns_using_technique(thesrc): + """returns technique_id => {campaign, relationship} for each campaign using the technique.""" + return get_related(thesrc, "campaign", "uses", "attack-pattern", reverse=True) + + # technique:software + def techniques_used_by_software(thesrc): + """return software_id => {technique, relationship} for each technique used by the software.""" + techniques_by_tool = get_related(thesrc, "tool", "uses", "attack-pattern") + techniques_by_malware = get_related(thesrc, "malware", "uses", "attack-pattern") + return {**techniques_by_tool, **techniques_by_malware} + + def software_using_technique(thesrc): + """return technique_id => {software, relationship} for each software using the technique.""" + tools_by_technique_id = get_related(thesrc, "tool", "uses", "attack-pattern", reverse=True) + malware_by_technique_id = get_related(thesrc, "malware", "uses", "attack-pattern", reverse=True) + return {**tools_by_technique_id, **malware_by_technique_id} + + # technique:mitigation + def mitigation_mitigates_techniques(thesrc): + """return mitigation_id => {technique, relationship} for each technique mitigated by the mitigation.""" + return get_related(thesrc, "course-of-action", "mitigates", "attack-pattern", reverse=False) + + def technique_mitigated_by_mitigations(thesrc): + """return technique_id => {mitigation, relationship} for each mitigation of the technique.""" + return get_related(thesrc, "course-of-action", "mitigates", "attack-pattern", reverse=True) + + # technique:sub-technique + def subtechniques_of(thesrc): + """return technique_id => {subtechnique, relationship} for each subtechnique of the technique.""" + return get_related(thesrc, "attack-pattern", "subtechnique-of", "attack-pattern", reverse=True) + + def parent_technique_of(thesrc): + """return subtechnique_id => {technique, relationship} describing the parent technique of the subtechnique""" + return get_related(thesrc, "attack-pattern", "subtechnique-of", "attack-pattern")[0] + + # technique:data-component + def datacomponent_detects_techniques(thesrc): + """return datacomponent_id => {technique, relationship} describing the detections of each data component""" + return get_related(thesrc, "x-mitre-data-component", "detects", "attack-pattern") + + def technique_detected_by_datacomponents(thesrc): + """return technique_id => {datacomponent, relationship} describing the data components that can detect the technique""" + return get_related(thesrc, "x-mitre-data-component", "detects", "attack-pattern", reverse=True) + + # Example usage: + src = MemoryStore() + src.load_from_file("path/to/enterprise-attack.json") + + group_id_to_software = software_used_by_groups(src) + pprint(group_id_to_software["intrusion-set--2a158b0a-7ef8-43cb-9985-bf34d1e12050"]) # G0019 + # [ + # { + # "object": Malware, # S0061 + # "relationship": Relationship # relationship between G0019 and S0061 + # }, + # { + # ... + # } + # ] diff --git a/docs/stix_primer/remove_revoked_deprecated.rst b/docs/stix_primer/remove_revoked_deprecated.rst new file mode 100644 index 00000000..4cd21562 --- /dev/null +++ b/docs/stix_primer/remove_revoked_deprecated.rst @@ -0,0 +1,26 @@ +Removing revoked and deprecated objects +=============== + +Revoked and deprecated objects are kept in the knowledge base so that workflows relying on those objects are not +broken. We recommend you filter out revoked and deprecated objects from your views whenever possible since they are no +longer maintained by ATT&CK. + +We recommend `not` using built-in STIX filters for removing revoked objects (e.g ``Filter('revoked', '=', False)``). This is because the behavior of this specific filter is inconsistent depending on the method of access (using local data or accessing via the TAXII server). We recommend using the following code example to filter revoked objects instead. See `issue #127 `_ for more details. + +.. code-block:: python + + from stix2 import Filter + + def remove_revoked_deprecated(stix_objects): + """Remove any revoked or deprecated objects from queries made to the data source""" + # Note we use .get() because the property may not be present in the JSON data. The default is False + # if the property is not set. + return list( + filter( + lambda x: x.get("x_mitre_deprecated", False) is False and x.get("revoked", False) is False, + stix_objects + ) + ) + + mitigations = src.query([ Filter("type", "=", "course-of-action") ]) + mitigations = remove_revoked_deprecated(mitigations) diff --git a/docs/stix_primer/stix-recipes.rst b/docs/stix_primer/stix-recipes.rst new file mode 100644 index 00000000..aae9db25 --- /dev/null +++ b/docs/stix_primer/stix-recipes.rst @@ -0,0 +1,13 @@ +Stix Recipes +=============== + +**Python recipes** + +Below are example python recipes which can be used to work with ATT&CK data. They assume the existence of an object implementing the DataStore API. Any of the methods outlined in the [Accessing ATT&CK data in python](#accessing-ATTCK-Data-in-Python) section should provide an object implementing this API. + +This section utilizes the `stix2 python library `_ . Please refer to the `STIX2 Python API Documentation `_ for more information on how to work with STIX programmatically. See also the section on [Requirements and imports](#requirements-and-imports). + +**Getting an object** + +The recipes in this section address how to query the dataset for a single object. + diff --git a/docs/stix_primer/stix_id.rst b/docs/stix_primer/stix_id.rst new file mode 100644 index 00000000..60ee9a6e --- /dev/null +++ b/docs/stix_primer/stix_id.rst @@ -0,0 +1,8 @@ +By STIX ID +=============== + +The following recipe can be used to retrieve an object according to its STIX ID. This is typically the preferred way to retrieve objects when working with ATT&CK data because STIX IDs are guaranteed to be unique. + +.. code-block:: python + + g0075 = src.get("intrusion-set--f40eb8ce-2a74-4e56-89a1-227021410142") diff --git a/docs/stix_primer/tactics_by_matrix.rst b/docs/stix_primer/tactics_by_matrix.rst new file mode 100644 index 00000000..c556e180 --- /dev/null +++ b/docs/stix_primer/tactics_by_matrix.rst @@ -0,0 +1,26 @@ +Tactics by matrix +=============== + +The tactics are individual objects (``x-mitre-tactic``), and their order in a matrix a (``x-mitre-matrix``) is +found within the ``tactic_refs`` property in a matrix. The order of the tactics in that list matches +the ordering of the tactics in that matrix. The following recipe returns a structured list of tactics within each matrix of the input DataStore. + +.. code-block:: python + + from stix2 import Filter + + def getTacticsByMatrix(thesrc): + tactics = {} + matrix = thesrc.query([ + Filter('type', '=', 'x-mitre-matrix'), + ]) + + for i in range(len(matrix)): + tactics[matrix[i]['name']] = [] + for tactic_id in matrix[i]['tactic_refs']: + tactics[matrix[i]['name']].append(thesrc.get(tactic_id)) + + return tactics + + # get tactic layout + getTacticsByMatrix(src) diff --git a/docs/stix_primer/techniques_by_group_sw.rst b/docs/stix_primer/techniques_by_group_sw.rst new file mode 100644 index 00000000..42cfd3a7 --- /dev/null +++ b/docs/stix_primer/techniques_by_group_sw.rst @@ -0,0 +1,34 @@ +Getting techniques used by a group's software +=============== + +Because a group uses software, and software uses techniques, groups can be considered indirect users of techniques used by their software. +These techniques are oftentimes distinct from the techniques used directly by a group, although there are occasionally intersections in these two sets of techniques. + +The following recipe can be used to retrieve the techniques used by a group's software: + +.. code-block:: python + + from stix2.utils import get_type_from_id + from stix2 import Filter + + def get_techniques_by_group_software(thesrc, group_stix_id): + # get the malware, tools that the group uses + group_uses = [ + r for r in thesrc.relationships(group_stix_id, 'uses', source_only=True) + if get_type_from_id(r.target_ref) in ['malware', 'tool'] + ] + + # get the technique stix ids that the malware, tools use + software_uses = thesrc.query([ + Filter('type', '=', 'relationship'), + Filter('relationship_type', '=', 'uses'), + Filter('source_ref', 'in', [r.source_ref for r in group_uses]) + ]) + + #get the techniques themselves + return thesrc.query([ + Filter('type', '=', 'attack-pattern'), + Filter('id', 'in', [r.target_ref for r in software_uses]) + ]) + + get_techniques_by_group_software(src, "intrusion-set--f047ee18-7985-4946-8bfb-4ed754d3a0dd") diff --git a/docs/stix_primer/techniques_by_platform.rst b/docs/stix_primer/techniques_by_platform.rst new file mode 100644 index 00000000..2bb48be7 --- /dev/null +++ b/docs/stix_primer/techniques_by_platform.rst @@ -0,0 +1,19 @@ +Techniques by platform +=============== + +Techniques are associated with one or more platforms. You can query the techniques +under a specific platform with the following code: + +.. code-block:: python + + from stix2 import Filter + + def get_techniques_by_platform(thesrc, platform): + return thesrc.query([ + Filter('type', '=', 'attack-pattern'), + Filter('x_mitre_platforms', '=', platform) + ]) + + # get techniques in the windows platform + get_techniques_by_platform(src, 'Windows') + diff --git a/docs/stix_primer/techniques_by_tactic.rst b/docs/stix_primer/techniques_by_tactic.rst new file mode 100644 index 00000000..ee0d0bed --- /dev/null +++ b/docs/stix_primer/techniques_by_tactic.rst @@ -0,0 +1,25 @@ +Techniques by tactic +=============== + +Techniques are related to tactics by their kill_chain_phases property. +The ``phase_name`` of each kill chain phase corresponds to the ``x_mitre_shortname`` of a tactic. + +.. code-block:: python + + from stix2 import Filter + + def get_tactic_techniques(thesrc, tactic): + # double checking the kill chain is MITRE ATT&CK + # note: kill_chain_name is different for other domains: + # - enterprise: "mitre-attack" + # - mobile: "mitre-mobile-attack" + # - ics: "mitre-ics-attack" + return thesrc.query([ + Filter('type', '=', 'attack-pattern'), + Filter('kill_chain_phases.phase_name', '=', tactic), + Filter('kill_chain_phases.kill_chain_name', '=', 'mitre-attack'), + ]) + + + # use the x_mitre_shortname as argument + get_tactic_techniques(src, 'defense-evasion') diff --git a/docs/stix_primer/techniques_subtechniques.rst b/docs/stix_primer/techniques_subtechniques.rst new file mode 100644 index 00000000..d4a7a913 --- /dev/null +++ b/docs/stix_primer/techniques_subtechniques.rst @@ -0,0 +1,35 @@ +Getting techniques or sub-techniques +=============== + +ATT&CK Techniques and sub-techniques are both represented as ``attack-pattern`` objects. Therefore further parsing is necessary to get specifically techniques or sub-techniques. + +.. code-block:: python + + from stix2 import Filter + + def get_techniques_or_subtechniques(thesrc, include="both"): + """Filter Techniques or Sub-Techniques from ATT&CK Enterprise Domain. + include argument has three options: "techniques", "subtechniques", or "both" + depending on the intended behavior.""" + if include == "techniques": + query_results = thesrc.query([ + Filter('type', '=', 'attack-pattern'), + Filter('x_mitre_is_subtechnique', '=', False) + ]) + elif include == "subtechniques": + query_results = thesrc.query([ + Filter('type', '=', 'attack-pattern'), + Filter('x_mitre_is_subtechnique', '=', True) + ]) + elif include == "both": + query_results = thesrc.query([ + Filter('type', '=', 'attack-pattern') + ]) + else: + raise RuntimeError("Unknown option %s!" % include) + + return query_results + + + subtechniques = get_techniques_or_subtechniques(src, "subtechniques") + subtechniques = remove_revoked_deprecated(subtechniques) # see https://github.com/mitre/cti/blob/master/USAGE.md#removing-revoked-and-deprecated-objects diff --git a/mitreattack/attackToExcel/README.md b/mitreattack/attackToExcel/README.md index b3920890..13427ad8 100644 --- a/mitreattack/attackToExcel/README.md +++ b/mitreattack/attackToExcel/README.md @@ -1,108 +1,3 @@ # ATT&CK To Excel -This folder contains a module for converting [ATT&CK STIX data](https://github.com/mitre/cti) to Excel spreadsheets. -It also provides a means to access ATT&CK data as [Pandas](https://pandas.pydata.org/) DataFrames for data analysis. - -## Usage - -### Command Line - -Print full usage instructions: - -```shell -python3 attackToExcel.py -h -``` - -Example execution: - -```shell -python3 attackToExcel.py -``` - -Build a excel files corresponding to a specific domain and version of ATT&CK: - -```shell -python3 attackToExcel -domain mobile-attack -version v5.0 -``` - -### Module - -Example execution targeting a specific domain and version: - -```python -import mitreattack.attackToExcel.attackToExcel as attackToExcel - -attackToExcel.export("mobile-attack", "v5.0", "/path/to/export/folder") -``` - -## Interfaces - -### attackToExcel - -attackToExcel provides the means by which to convert/extract the ATT&CK STIX data to Excel spreadsheets. A brief -overview of the available methods follows. - -| method name | arguments | usage | -|:------------|:----------|:------| -|get_stix_data|`domain`: the domain of ATT&CK to fetch data from
`version`: optional parameter indicating which version to fetch data from (such as "v8.1"). If omitted retrieves the most recent version of ATT&CK.
`remote`: optional parameter that provides a URL of a remote ATT&CK Workbench instance to grab data from.| Retrieves the ATT&CK STIX data for the specified version and returns it as a MemoryStore object| -|build_dataframes| `src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to| Builds a Pandas DataFrame collection as a dictionary, with keys for each type, based on the ATT&CK data provided| -|write_excel| `dataframes`: pandas DataFrame dictionary (generated by build_dataframes)
`domain`: domain of ATT&CK that `dataframes` corresponds to
`version`: optional parameter indicating which version of ATT&CK is in use
`output_dir`: optional parameter specifying output directory| Writes out DataFrame based ATT&CK data to excel files| -|export| `domain`: the domain of ATT&CK to download
`version`: optional parameter specifying which version of ATT&CK to download
`output_dir`: optional parameter specifying output directory| Downloads ATT&CK data from MITRE/CTI and exports it to Excel spreadsheets | - -### stixToDf - -stixToDf provides various methods to process and manipulate the STIX data in order to create [Pandas](https://pandas.pydata.org/) DataFrames for -processing. A brief overview of these methods follows. - -| method name | arguments | usage | -|:------------|:----------|:------| -|techniquesToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX techniques from the provided data and returns corresponding Pandas DataFrames.| -|tacticsToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX tactics from the provided data and returns corresponding Pandas DataFrames.| -|softwareToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX software from the provided data and returns corresponding Pandas DataFrames.| -|groupsToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX groups from the provided data and returns corresponding Pandas DataFrames.| -|mitigationsToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX mitigations from the provided data and returns corresponding Pandas DataFrames.| -|relationshipsToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX relationships from the provided data and returns corresponding Pandas DataFrames.| -|matricesToDf|`src`: MemoryStore or other stix2 DataSource object holding domain data
`domain`: domain of ATT&CK that `src` corresponds to | Parses STIX matrices from the provided data and returns a parsed matrix structure of the form `[{matrix, name, description, merge, border}, ...]`| - -## Spreadsheet format - -The Excel representation of the ATT&CK dataset includes both master spreadsheets, -containing all object types, and individual spreadsheets for each object type. -The individual type spreadsheets break out relationships (e.g procedure examples connecting groups to techniques) -into separate sheets by relationship type, while the master spreadsheet includes all relationship types in a single sheet. -Otherwise, the representation is identical. - -A citations sheet can be used to look up the in-text citations which appear in some fields. -For domains that include multiple matrices, such as Mobile ATT&CK, each matrix gets its own named sheet. -Unlike the STIX dataset, objects that have been revoked or deprecated are not included in the spreadsheets. - -## Accessing the Pandas DataFrames - -Internally, attackToExcel stores the parsed STIX data as [Pandas](https://pandas.pydata.org/) DataFrames. -These can be retrieved for use in data analysis. - -Example of accessing [Pandas](https://pandas.pydata.org/) DataFrames: - -```python -import mitreattack.attackToExcel.attackToExcel as attackToExcel -import mitreattack.attackToExcel.stixToDf as stixToDf - -# download and parse ATT&CK STIX data -attackdata = attackToExcel.get_stix_data("enterprise-attack") -techniques_data = stixToDf.techniquesToDf(attackdata, "enterprise-attack") - -# show T1102 and sub-techniques of T1102 -techniques_df = techniques_data["techniques"] -print(techniques_df[techniques_df["ID"].str.contains("T1102")]["name"]) -# 512 Web Service -# 38 Web Service: Bidirectional Communication -# 121 Web Service: Dead Drop Resolver -# 323 Web Service: One-Way Communication -# Name: name, dtype: object - -# show citation data for LOLBAS Wmic reference -citations_df = techniques_data["citations"] -print(citations_df[citations_df["reference"].str.contains("LOLBAS Wmic")]) -# reference citation url -# 1010 LOLBAS Wmic LOLBAS. (n.d.). Wmic.exe. Retrieved July 31, 2... https://lolbas-project.github.io/lolbas/Binari... -``` +If you wish to read more about the ATT&CK to Excel module, please click [here](https://mitreattack-python.readthedocs.io/en/126-docs-add-section-to-docs-for-accessing-stix/attacktoexcel.html) for more information. diff --git a/mitreattack/collections/README.md b/mitreattack/collections/README.md index 412aa48d..542ef720 100644 --- a/mitreattack/collections/README.md +++ b/mitreattack/collections/README.md @@ -1,94 +1,3 @@ # collections -This folder contains modules and scripts for working with ATT&CK collections. -Collections are sets of ATT&CK STIX objects, grouped for user convienence. -For more information about ATT&CK collections, see the corresponding -[ATT&CK documentation](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collections). - -## Collections Scripts - -| script | description | -|:-------|:------------| -|[index_to_markdown](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/collections/index_to_markdown.py)| Provides a means by which to convert a [collection index](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collection-indexes) into a human-readable markdown file. More information can be found in the corresponding [section](#index_to_markdown.py) below.| -|[collection_to_index](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/collections/collection_to_index.py)| Provides a means by which to convert a [collection](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collections) into a easy-to-share [index file](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collection-indexes). More information can be found in the corresponding [section](#collection_to_index.py) below.| -|[stix_to_collection](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/collections/stix_to_collection.py)| Provides a means by which to convert raw stix (in the form of [bundles](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_gms872kuzdmg)) into a [collection](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collections). More information can be found in the corresponding [section](#stix_to_collection.py) below.| - -### index_to_markdown.py - -`index_to_markdown.py` provides the `IndexToMarkdown` class, which provides a way to transform an existing -[collection index file](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collection-indexes) -into a markdown file for easy of use and reference. -The `IndexToMarkdown` class is very simple, and provides a single method, `index_to_markdown`, -which in turn only requires a single parameter - a dictionary representation of the desired index file to convert to markdown. -An example of how to use the class, and method, can be found below. - -#### Example Usage - -```python -import json -from mitreattack.collections import IndexToMarkdown - -with open('collection_index.json', 'r') as input_file: - with open('collection_index.md', 'w') as output_file: - input_index = json.load(input_file) - generated_md = IndexToMarkdown.index_to_markdown(input_index) # Convert index to markdown - output_file.write(generated_md) -print(generated_md) -``` - -### collection_to_index.py - -`collection_to_index.py` provides the `CollectionToIndex` class, which proves a means by which to summarize existing -[collections](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collections) -into a single [collection index](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collection-indexes) file. -The `CollectionToIndex` class contains the `generate_index` function, which when provided with a name, description, root url (pointing to where the raw collections are stored), -and a list of either files, folders, or already loaded bundles in the form of dictionaries, will create a summarizing index. - -#### Example Usage - -```python -import json -from mitreattack.collections import CollectionToIndex - -output_indexA = CollectionToIndex.generate_index(name='example', description='example index', - root_url='www.example.com', - files=['/path/to/collection1.json', '/path/to/collection2.json'], - folders=None, sets=None) -output_indexB = CollectionToIndex.generate_index(name='example2', description='demonstration index', - root_url='www.example.com', - files=None, folders=['/path/to/folder/with/collections'], sets=None) -with open('path/to/bundle/bundleC.json', 'r') as f: - data = json.load(f) -output_indexC = CollectionToIndex.generate_index(name='example3', description='exhibit index', - root_url='www.example.com', - files=None, folders=None, sets=[data]) -print(output_indexA) -print(output_indexB) -print(output_indexC) -``` - -### stix_to_collection.py - -`stix_to_collection.py` provides the `STIXToCollection` class, which proves a means by which to convert -existing stix bundles into ones containing a -[collection](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md#collections) object. -The `STIXToCollection` class contains the `stix_to_collection` function, which when provided with a starter bundle, -a name, a version, and an optional description, will output a modified bundle that contains a summary collection object. - -#### Example Usage - -```python -import json -from mitreattack.collections import STIXToCollection - -with open('path/to/bundle/bundle2_0.json', 'r') as f: - data = json.load(f) -output_bundleA = STIXToCollection.stix_to_collection(bundle=data, name='collectionA', version='9.1', description="demo bundle (2.0)") - -with open('path/to/bundle/bundle2_1.json', 'r') as f: - data = json.load(f) -output_bundleB = STIXToCollection.stix_to_collection(bundle=data, name='collectionB', version='9.0', description="demo bundle (2.1)") - -print(output_bundleA) -print(output_bundleB) -``` +If you wish to read more about the collections module, please click [here](https://mitreattack-python.readthedocs.io/en/126-docs-add-section-to-docs-for-accessing-stix/collections.html) for more information. diff --git a/mitreattack/diffStix/README.md b/mitreattack/diffStix/README.md index 80453812..b33d06b2 100644 --- a/mitreattack/diffStix/README.md +++ b/mitreattack/diffStix/README.md @@ -1,119 +1,3 @@ # Diff Stix -This folder contains a module for creating markdown, HTML, JSON and/or ATT&CK Navigator layers -reporting on the changes between two versions of the STIX2 bundles representing the ATT&CK content. -Run `diff_stix -h` for full usage instructions. - -## Usage - -### Command Line - -Print full usage instructions: - -```shell -# You must run `pip install mitreattack-python` in order to access the diff_stix command -diff_stix --help -usage: diff_stix [-h] [--old OLD] [--new NEW] [--domains {enterprise-attack,mobile-attack,ics-attack} [{enterprise-attack,mobile-attack,ics-attack} ...]] [--markdown-file MARKDOWN_FILE] [--html-file HTML_FILE] [--html-file-detailed HTML_FILE_DETAILED] - [--json-file JSON_FILE] [--layers [LAYERS ...]] [--site_prefix SITE_PREFIX] [--unchanged] [--use-mitre-cti] [--show-key] [--contributors] [--no-contributors] [-v] - -Create changelog reports on the differences between two versions of the ATT&CK content. Takes STIX bundles as input. For default operation, put enterprise-attack.json, mobile-attack.json, and ics-attack.json bundles in 'old' and 'new' folders for the script to compare. - -options: - -h, --help show this help message and exit - --old OLD Directory to load old STIX data from. - --new NEW Directory to load new STIX data from. - --domains {enterprise-attack,mobile-attack,ics-attack} [{enterprise-attack,mobile-attack,ics-attack} ...] - Which domains to report on. Choices (and defaults) are enterprise-attack, mobile-attack, ics-attack - --markdown-file MARKDOWN_FILE - Create a markdown file reporting changes. - --html-file HTML_FILE - Create HTML page from markdown content. - --html-file-detailed HTML_FILE_DETAILED - Create an HTML file reporting detailed changes. - --json-file JSON_FILE - Create a JSON file reporting changes. - --layers [LAYERS ...] - Create layer files showing changes in each domain expected order of filenames is 'enterprise', 'mobile', 'ics', 'pre attack'. If values are unspecified, defaults to output/January_2023_Updates_Enterprise.json, - output/January_2023_Updates_Mobile.json, output/January_2023_Updates_ICS.json, output/January_2023_Updates_Pre.json - --site_prefix SITE_PREFIX - Prefix links in markdown output, e.g. [prefix]/techniques/T1484 - --unchanged Show objects without changes in the markdown output - --use-mitre-cti Use content from the MITRE CTI repo for the -old data - --show-key Add a key explaining the change types to the markdown - --contributors Show new contributors between releases - --no-contributors Do not show new contributors between releases - -v, --verbose Print status messages -``` - -Example execution: - -```shell -diff_stix -v --show-key --html-file output/changelog.html --html-file-detailed output/changelog-detailed.html --markdown-file output/changelog.md --json-file output/changelog.json --layers output/layer-enterprise.json output/layer-mobile.json output/layer-ics.json --old path/to/old/stix/ --new path/to/new/stix/ -``` - -## Changelog JSON format - -The changelog helper script has the option to output a JSON file with detailed differences between ATT&CK releases. -This is the overall structure you can expect to find in the file. -A brief explanation of key pieces can be found below. - -```JSON -{ - "enterprise-attack": { - "techniques": { - "additions": [], - "major_version_changes": [], - "minor_version_changes": [], - "other_version_changes": [], - "patches": [], - "revocations": [], - "deprecations": [], - "deletions": [], - }, - "software": {}, - "groups": {}, - "campaigns": {}, - "mitigations": {}, - "datasources": {}, - "datacomponents": {} - }, - "mobile-attack": {}, - "ics-attack": {}, - "new-contributors": [ - "Contributor A", - "Contributor B", - "Contributor C" - ] -} -``` - -* The top-level objects include information about specific domains as well as `new-contributors`, which are only found in the newer ATT&CK release. -* For domain changes, they are broken down by object type, e.g. `techniques` or `mitigations`. -* The following table helps break down the change types that are currently tracked. - -| Field | Type | Description | -|-------------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------| -| `additions` | array[object] | ATT&CK objects which are only present in the new STIX data. | -| `major_version_changes` | array[object] | ATT&CK objects that have a major version change. (e.g. 1.0 → 2.0). | -| `minor_version_changes` | array[object] | ATT&CK objects that have a minor version change. (e.g. 1.0 → 1.1). | -| `other_version_changes` | array[object] | ATT&CK objects that have a version change of any other kind. (e.g. 1.0 → 1.3). These are unintended, but can be found in previous releases. | -| `patches` | array[object] | ATT&CK objects that have been patched while keeping the version the same. | -| `revocations` | array[object] | ATT&CK objects which are revoked by a different object. | -| `deprecations` | array[object] | ATT&CK objects which are deprecated and no longer in use, and not replaced. | -| `deletions` | array[object] | ATT&CK objects which are no longer found in the STIX data. This should almost never happen. | - -### Changed Objects - -The bulk of the changelog file consists of lists of JSON objects. -If you are familiar with reading the STIX format, they may look famliar, yet a little "off". -That is because there are a few fields that have been added in some cases depending on what section they appear in. -For example, objects that are brand new do not have `previous_version` available to them. -The following table lists the extra fields that can be found in objects in the changelog. - -| Field | Required | Type | Description | -|----------------------------|----------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `changelog_mitigations` | false | object | Three lists for `shared`, `new`, and `dropped` for Mitigations that are related to a Technique between versions. | -| `changelog_detections` | false | object | Three lists for `shared`, `new`, and `dropped` for Detections that are related to a Technique between versions. || `description_change_table` | false | string | HTML rendering of a table that displays the differences between descriptions for an ATT&CK object. | -| `detailed_diff` | false | string | A python DeepDiff object that has been JSON serialized which represents STIX changes for an ATT&CK object between releases. | -| `previous_version` | false | string | If the object existed in the previous release, then it denotes the version the object was in the previous release. | -| `version_change` | false | string | If the object existed in the previous release and was changed in the current release, then a descriptive string in the format '`old-version` → `new-version`' | +If you wish to read more about the collections module, please click [here](https://mitreattack-python.readthedocs.io/en/126-docs-add-section-to-docs-for-accessing-stix/diffinstix.html) for more information. \ No newline at end of file diff --git a/mitreattack/navlayers/README.md b/mitreattack/navlayers/README.md index ae507a80..d02dd82e 100644 --- a/mitreattack/navlayers/README.md +++ b/mitreattack/navlayers/README.md @@ -1,521 +1,3 @@ # navlayers -This folder contains modules and scripts for working with ATT&CK Navigator layers. -ATT&CK Navigator Layers are a set of annotations overlaid on top of the ATT&CK Matrix. -For more about ATT&CK Navigator layers, visit the ATT&CK Navigator repository. -The core module allows users to load, validate, manipulate, and save ATT&CK layers. -A brief overview of the components can be found below. -All scripts adhere to the MITRE ATT&CK Navigator Layer file format, -[version 4.3](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_3.md), -but will accept legacy [version 3.0](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv3.md) -and version 4.X layers, upgrading them to version 4.3. - -## Core Modules - -| script | description | -|:-------|:------------| -| [filter](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/filter.py) | Implements a basic [filter object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#filter-object-properties). | -| [gradient](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/gradient.py) | Implements a basic [gradient object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#gradient-object-properties). | -| [layer](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/layer.py) | Provides an interface for interacting with core module's layer representation. A further breakdown can be found in the corresponding [section](#Layer) below. | -| [layout](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/layout.py) | Implements a basic [layout object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#layout-object-properties). | -| [legenditem](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/legenditem.py) | Implements a basic [legenditem object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#legenditem-object-properties). | -| [metadata](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/metadata.py) | Implements a basic [metadata object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#metadata-object-properties). | -| [technique](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/technique.py) | Implements a basic [technique object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#technique-object-properties). | -| [versions](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/versions.py) | Implements a basic [versions object](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md#versions-object-properties).| - -### Manipulator Scripts - -| script | description | -|:-------|:------------| -| [layerops](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/manipulators/layerops.py) | Provides a means by which to combine multiple ATT&CK layer objects in customized ways. A further breakdown can be found in the corresponding [section](#layerops.py) below. | - -### Exporter Scripts - -| script | description | -|:-------|:------------| -| [to_excel](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/to_excel.py) | Provides a means by which to export an ATT&CK Layer to an excel file. A further breakdown can be found in the corresponding [section](#to_excel.py) below. | -| [to_svg](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/to_svg.py) | Provides a means by which to export an ATT&CK layer to an svg image file. A further breakdown can be found in the corresponding [section](#to_svg.py) below. This file also contains the `SVGConfig` object that can be used to configure the SVG export.| - -### Generator Scripts - -| script | description | -|:-------|:------------| -| [overview_generator](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/generators/overview_generator.py)| Provides a means by which to generate an ATT&CK Layer that summarizes, on a per technique basis, all instances of a given ATT&CK object type that reference/utilize each technique. A further explanation can be found in the corresponding [section](#overview_generator.py) below. | -| [usage_generator](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/generators/usage_generator.py)| Provides a means by which to generate an ATT&CK Layer that summarizes the techniques associated with a given ATT&CK object. A further explanation can be found in the corresponding [section](#usage_generator.py) below. | -| [sum_generator](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/generators/sum_generator.py)| Provides a means by which to generate a collection of ATT&CK Layers, one for each object in a given ATT&CK object class, that summarizes the coverage of that object. A further explanation can be found in the corresponding [section](#sum_generator.py) below. | - -### Utility Modules - -| script | description | -|:-------|:------------| -| [excel_templates](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/excel_templates.py) | Provides a means by which to convert a matrix into a clean excel matrix template. | -| [matrix_gen](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/matrix_gen.py) | Provides a means by which to generate a matrix from raw data, either from the ATT&CK TAXII server, from a local STIX Bundle, or from an ATT&CK Workbench instance (via url). | -| [svg_templates](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/svg_templates.py) | Provides a means by which to convert a layer file into a marked up svg file. | -| [svg_objects](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/exporters/svg_objects.py) | Provides raw templates and supporting functionality for generating svg objects. | - -### Command Line Tools - -| script | description | -|:-------|:------------| -| [layerExporter_cli.py](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/layerExporter_cli.py) | A commandline utility to export Layer files to excel or svg formats using the exporter tools. Run with `-h` for usage. | -| [layerGenerator_cli.py](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/layerGenerator_cli.py) | A commandline utility to generate Layer files that correspond to various and collections of various stix objects. Run with `-h` for usage. | - -## Layer - -The `Layer` class provides format validation and read/write capabilities to aid in working with ATT&CK Navigator Layers in python. -It is the primary interface through which other Layer-related classes defined in the core module should be used. -The Layer class API and a usage example are below. -The class currently supports version 3 and 4 of the ATT&CK Layer spec, and will upgrade version 3 layers into compatible version 4 ones whenever possible. - -| method [x = Layer()]| description | -|:-------|:------------| -| `x.from_str(_input_)` | Loads an ATT&CK layer from a string representation of a json layer. | -| `x.from_dict(_input_)` | Loads an ATT&CK layer from a dictionary. | -| `x.from_file(_filepath_)` | Loads an ATT&CK layer from a file location specified by the _filepath_. | -| `x.to_file(_filepath_)` | Saves the current state of the loaded ATT&CK layer to a json file denoted by the _filepath_. | -| `x.to_dict()` | Returns a representation of the current ATT&CK layer object as a dictionary. | -| `x.to_str()` | Returns a representation of the current ATT&CK layer object as a string representation of a dictionary. | - -Examples on how to create a layer programmatically, as opposed to loading it from an existing medium, can be found -[here](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/core/README.md). - -### Example Usage - -```python -example_layer3_dict = { - "name": "example layer", - "version": "3.0", - "domain": "mitre-enterprise" -} - -example_layer4_dict = { - "name": "layer v4.3 example", - "versions" : { - "attack": "8", - "layer" : "4.3", - "navigator": "4.4.4" - }, - "domain": "enterprise-attack" -} - -example_layer_location = "/path/to/layer/file.json" -example_layer_out_location = "/path/to/new/layer/file.json" - -from mitreattack.navlayers.core import Layer - -layer1 = Layer(example_layer3_dict) # Create a new layer and load existing data -layer1.to_file(example_layer_out_location) # Write out the loaded layer to the specified file - -layer2 = Layer() # Create a new layer object -layer2.from_dict(example_layer4_dict) # Load layer data into existing layer object -print(layer2.to_dict()) # Retrieve the loaded layer's data as a dictionary, and print it - -layer3 = Layer() # Create a new layer object -layer3.from_file(example_layer_location) # Load layer data from a file into existing layer object -``` - -### layerops.py - -`Layerops.py` provides the `LayerOps` class, which is a way to combine layer files in an automated way, using user defined lambda functions. -Each LayerOps instance, when created, ingests the provided lambda functions, and stores them for use. -An existing `LayerOps` class can be used to combine layer files according to the initialized lambda using the process method. -The breakdown of this two step process is documented in the table below, while examples of both the list and dictionary modes of operation can be found below. - -#### LayerOps() - -```python -x = LayerOps(score=score, comment=comment, enabled=enabled, colors=colors, metadata=metadata, name=name, desc=desc, default_values=default_values) -``` - -Each of the _inputs_ takes a lambda function that will be used to combine technique object fields matching the parameter. -The one exception to this is _default_values_, which is an optional dictionary argument containing default values -to provide the lambda functions if techniques of the combined layers are missing them. - -##### .process() Method - -```python -x.process(data, default_values=default_values) -``` - -The process method applies the lambda functions stored during initialization to the layer objects in _data_. -_data_ must be either a list or a dictionary of Layer objects, and is expected to match the format of the lambda equations provided during initialization. -`default_values` is an optional dictionary argument that overrides the currently stored default values with new ones for this specific processing operation. - -#### Example Usage - -```python -from mitreattack.navlayers.manipulators.layerops import LayerOps -from mitreattack.navlayers.core.layer import Layer - -demo = Layer() -demo.from_file("C:\Users\attack\Downloads\layer.json") -demo2 = Layer() -demo2.from_file("C:\Users\attack\Downloads\layer2.json") -demo3 = Layer() -demo3.from_file("C:\Users\attack\Downloads\layer3.json") - -# Example 1) Build a LayerOps object that takes a list and averages scores across the layers -lo = LayerOps(score=lambda x: sum(x) / len(x), - name=lambda x: x[1], - desc=lambda x: "This is an list example") # Build LayerOps object -out_layer = lo.process([demo, demo2]) # Trigger processing on a list of demo and demo2 layers -out_layer.to_file("C:\demo_layer1.json") # Save averaged layer to file -out_layer2 = lo.process([demo, demo2, demo3]) # Trigger processing on a list of demo, demo2, demo3 -visual_aid = out_layer2.to_dict() # Retrieve dictionary representation of processed layer - -# Example 2) Build a LayerOps object that takes a dictionary and averages scores across the layers -lo2 = LayerOps(score=lambda x: sum([x[y] for y in x]) / len([x[y] for y in x]), - colors=lambda x: x['b'], - desc=lambda x: "This is a dict example") # Build LayerOps object, with lambda -out_layer3 = lo2.process({'a': demo, 'b': demo2}) # Trigger processing on a dictionary of demo and demo2 -dict_layer = out_layer3.to_dict() # Retrieve dictionary representation of processed layer -print(dict_layer) # Display retrieved dictionary -out_layer4 = lo2.process({'a': demo, 'b': demo2, 'c': demo3})# Trigger processing on a dictionary of demo, demo2, demo3 -out_layer4.to_file("C:\demo_layer4.json") # Save averaged layer to file - -# Example 3) Build a LayerOps object that takes a single element dictionary and inverts the score -lo3 = LayerOps(score=lambda x: 100 - x['a'], - desc= lambda x: "This is a simple example") # Build LayerOps object to invert score (0-100 scale) -out_layer5 = lo3.process({'a': demo}) # Trigger processing on dictionary of demo -print(out_layer5.to_dict()) # Display processed layer in dictionary form -out_layer5.to_file("C:\demo_layer5.json") # Save inverted score layer to file - -# Example 4) Build a LayerOps object that combines the comments from elements in the list, with custom defaults -lo4 = LayerOps(score=lambda x: '; '.join(x), - default_values= { - "comment": "This was an example of new default values" - }, - desc= lambda x: "This is a defaults example") # Build LayerOps object to combine descriptions, defaults -out_layer6 = lo4.process([demo2, demo3]) # Trigger processing on a list of demo2 and demo0 -out_layer6.to_file("C:\demo_layer6.json") # Save combined comment layer to file -``` - -## to_excel.py - -`to_excel.py` provides the `ToExcel` class, which is a way to export an existing layer file as an Excel spreadsheet. -The `ToExcel` class has an optional parameter for the initialization function, that tells the exporter what data source to use when building the output matrix. -Valid options include using live data from cti-taxii.mitre.org, using a local STIX bundle, or retrieving data from an ATT&CK Workbench instance. - -### ToExcel() - -```python -x = ToExcel(domain='enterprise', source='taxii', resource=None) -``` - -The `ToExcel` constructor takes domain, server, and resource arguments during instantiation. -The domain can be either `enterprise` or `mobile`, and can be pulled directly from a layer file as `layer.domain`. -The source argument tells the matrix generation tool which data source to use when building the matrix. -`taxii` indicates that the tool should utilize the official ATT&CK Taxii Server (`cti-taxii`) when building the matrix, -while the `local` option indicates that it should use a local bundle, and the `remote` option indicates that -it should utilize a remote ATT&CK Workbench instance. -The `resource` argument is only required if the source is set to `local`, in which case it should be a path -to a local stix bundle, or if the source is set to `remote`, in which case it should be the url of a ATT&CK workbench instance. - -### .to_xlsx() Method - -```python -x.to_xlsx(layerInit=layer, filepath="layer.xlsx") -``` - -The `to_xlsx` method exports the layer file referenced as `layer`, as an excel file to the `filepath` specified. - -#### Example Usage - -```python -from mitreattack.navlayers import Layer -from mitreattack.navlayers import ToExcel - -lay = Layer() -lay.from_file("path/to/layer/file.json") -# Using taxii server for template -t = ToExcel(domain=lay.layer.domain, source='taxii') -t.to_xlsx(layerInit=lay, filepath="demo.xlsx") -# Using local stix data for template -t2 = ToExcel(domain='mobile', source='local', resource='path/to/local/stix.json') -t2.to_xlsx(layerInit=lay, filepath="demo2.xlsx") -# Using remote ATT&CK Workbench instance for template -workbench_url = 'localhost:3000' -t3 = ToExcel(domain='ics', source='remote', resource=workbench_url) -``` - -## to_svg.py - -`to_svg.py` provides the `ToSvg` class, which is a way to export an existing layer file as an SVG image file. -The `ToSvg` class, like the `ToExcel` class, has an optional parameter for the initialization function, -that tells the exporter what data source to use when building the output matrix. -Valid options include using live data from cti-taxii.mitre.org, using a local STIX bundle, or utilizing a remote ATT&CK Workbench instance. - -### ToSvg() - -```python -x = ToSvg(domain='enterprise', source='taxii', resource=None, config=None) -``` - -The `ToSvg` constructor, just like the `ToExcel` constructor, takes domain, server, and resource arguments during instantiation. -The domain can be either `enterprise` or `mobile`, and can be pulled directly from a layer file as `layer.domain`. -The source argument tells the matrix generation tool which data source to use when building the matrix. -`taxii` indicates that the tool should utilize the `cti-taxii` server when building the matrix, -while the `local` option indicates that it should use a local bundle, and the `remote` option indicates that it should utilize a remote ATT&CK Workbench instance. -The `resource` argument is only required if the source is set to `local`, in which case it should be a path to a local stix bundle, -or if the source is set to `remote`, in which case it should be the url of an ATT&CK Workbench instance. -The `config` parameter is an optional `SVGConfig` object that can be used to configure the export as desired. -If not provided, the configuration for the export will be set to default values. - -### SVGConfig() - -```python -y = SVGConfig(width=8.5, height=11, headerHeight=1, unit="in", showSubtechniques="expanded", - font="sans-serif", tableBorderColor="#6B7279", showHeader=True, legendDocked=True, - legendX=0, legendY=0, legendWidth=2, legendHeight=1, showLegend=True, showFilters=True, - showAbout=True, showDomain=True, border=0.104) -``` - -The `SVGConfig` object is used to configure how an SVG export behaves. -The defaults for each of the available values can be found in the declaration above, and a brief explanation for each field is included in the table below. -The config object should be provided to the `ToSvg` object during instantiation, but if values need to be updated on the fly, -the currently loaded configuration can be interacted with at `ToSvg().config`. -The configuration can also be populated from a json file using the `.load_from_file(filename="path/to/file.json")` method, -or stored to one using the `.save_to_file(filename="path/to/file.json)` method. - -| attribute| description | type | default value | -|:-------|:------------|:------------|:------------| -| width | Desired SVG width | number | 8.5 | -| height | Desired SVG height | number | 11 | -| headerHeight | Desired Header Block height | number | 1 | -| unit | SVG measurement units (qualifies width, height, etc.) - "in", "cm", "px", "em", or "pt"| string | "in" | -| showSubtechniques | Display form for subtechniques - "all", "expanded" (decided by layer), or "none" | string | "expanded" | -| font | What font style to use - "serif", "sans-serif", or "monospace" | string | "sans-serif" | -| tableBorderColor | Hex color to use for the technique borders | string | "#6B7279" | -| showHeader | Whether or not to show Header Blocks | bool | True | -| legendDocked | Whether or not the legend should be docked | bool | True | -| legendX | Where to place the legend on the x axis if not docked | number | 0 | -| legendY | Where to place the legend on the y axis if not docked | number | 1 | -| legendWidth | Width of the legend if not docked | number | 2 | -| legendHeight | Height of the legend if not docked | number | 1 | -| showLegend | Whether or not to show the legend | bool | True | -| showFilters | Whether or not to show the Filter Header Block | bool | True | -| showDomain | Whether or not to show the Domain and Version Header Block | bool | True | -| showAbout | Whether or not to show the About Header Block | bool | True | -| border | What default border width to use | number | 0.104 | - -### .to_svg() Method - -```python -x.to_svg(layerInit=layer, filepath="layer.svg") -``` - -The `to_svg` method exports the layer file referenced as `layer`, as an excel file to the `filepath` specified. - -#### Example Usage - -```python -from mitreattack.navlayers import Layer -from mitreattack.navlayers import ToSvg, SVGConfig - -lay = Layer() -lay.from_file("path/to/layer/file.json") -# Using taxii server for template -t = ToSvg(domain=lay.layer.domain, source='taxii') -t.to_svg(layerInit=lay, filepath="demo.svg") -#Using local stix data for template - -conf = SVGConfig() -conf.load_from_file(filename="path/to/poster/config.json") - -t2 = ToSvg(domain='mobile', source='local', resource='path/to/local/stix.json', config=conf) -t2.to_svg(layerInit=lay, filepath="demo2.svg") - -workbench_url = "localhost:3000" -t3 = ToSvg(domain='enterprise', source='remote', resource=workbench_url, config=conf) -t3.to_svg(layerInit=lay, filepath="demo3.svg") -``` - -## overview_generator.py - -`overview_generator.py` provides the `OverviewLayerGenerator` class, which is designed to allow users to -generate an ATT&CK layer that, on a per technique basis, has a score that corresponds to all instances -of the specified ATT&CK object type (group, mitigation, etc.), and a comment that lists all matching instance. - -### OverviewLayerGenerator() - -```python -x = OverviewLayerGenerator(source='taxii', domain='enterprise', resource=None) -``` - -The initialization function for `OverviewLayerGenerator`, like `ToSVG` and `ToExcel`, requires the specification of where -to retrieve data from (taxii server etc.). -The domain can be either `enterprise`, `mobile`, or `ics`, and can be pulled directly from a layer file as `layer.domain`. -The source argument tells the matrix generation tool which data source to use when building the matrix. -`taxii` indicates that the tool should utilize the `cti-taxii` server when building the matrix, -while the `local` option indicates that it should use a local bundle, and the `remote` option indicates that it should utilize a remote ATT&CK Workbench instance. -The `resource` argument is only required if the source is set to `local`, in which case it should be a path to a local stix bundle, -or if the source is set to `remote`, in which case it should be the url of an ATT&CK Workbench instance. -If not provided, the configuration for the generator will be set to default values. - -### .generate_layer() - -```python -x.generate_layer(obj_type=object_type_name) -``` - -The `generate_layer` function generates a layer, customized to the input `object_type_name`. -Valid values include `group`, `mitigation`, `software`, and `datasource`. - -## usage_generator.py - -`usage_ generator.py` provides the `UsageLayerGenerator` class, which is designed to allow users to -generate an ATT&CK layer that scores any relevant techniques that a given input ATT&CK object has. -These objects can be any `group`, `software`, `mitigation`, or `data component`, -and can be referenced by ID or by any alias when provided to the generator. - -### UsageLayerGenerator() - -```python -x = UsageLayerGenerator(source='taxii', domain='enterprise', resource=None) -``` - -The initialization function for `UsageLayerGenerator`, like `ToSVG` and `ToExcel`, requires the specification of where -to retrieve data from (taxii server etc.). -The domain can be either `enterprise`, `mobile`, or `ics`, and can be pulled directly from a layer file as `layer.domain`. -The source argument tells the matrix generation tool which data source to use when building the matrix. -`taxii` indicates that the tool should utilize the `cti-taxii` server when building the matrix, -while the `local` option indicates that it should use a local bundle, and the `remote` option indicates that it should utilize a remote ATT&CK Workbench instance. -The `resource` argument is only required if the source is set to `local`, in which case it should be a path to a local stix bundle, -or if the source is set to `remote`, in which case it should be the url of an ATT&CK Workbench instance. -If not provided, the configuration for the generator will be set to default values. - -### .generate_layer() - -```python -x.generate_layer(match=object_identifier) -``` - -The `generate_layer` function generates a layer, customized to the input `object_identifier`. -Valid values include `ATT&CK ID`, `name`, or any known `alias` for `group`, `mitigation`, `software`, and `data component` objects within the selected ATT&CK data. - -#### Example Usage - -```python -from mitreattack.navlayers import UsageLayerGenerator - -handle = UsageLayerGenerator(source='taxii', domain='enterprise') - -layer1 = handle.generate_layer(match='G0018') -layer2 = handle.generate_layer(match='Adups') -``` - -## sum_generator.py - -`sum_generator.py` provides the `SumLayerGenerator` class, which is designed to allow users to -generate a collection of ATT&CK layers that, on a per technique basis, have a score that corresponds to all instances -of the specified ATT&CK object type (group, mitigation, etc.), and a comment that lists all matching instance. -Each one of the generated layers will correspond to a single instance of the specified ATT&CK object type. - -### SumLayerGenerator() - -```python -x = SumLayerGenerator(source='taxii', domain='enterprise', resource=None) -``` - -The initialization function for `SumGeneratorLayer`, like `ToSVG` and `ToExcel`, requires the specification of where -to retrieve data from (taxii server etc.). -The domain can be either `enterprise`, `mobile`, or `ics`, and can be pulled directly from a layer file as `layer.domain`. -The source argument tells the matrix generation tool which data source to use when building the matrix. -`taxii` indicates that the tool should utilize the `cti-taxii` server when building the matrix, -while the `local` option indicates that it should use a local bundle, and the `remote` option indicates that it should utilize a remote ATT&CK Workbench instance. -The `resource` argument is only required if the source is set to `local`, in which case it should be a path to a local stix bundle, -or if the source is set to `remote`, in which case it should be the url of an ATT&CK Workbench instance. -If not provided, the configuration for the generator will be set to default values. - -### .generate_layer() - -```python -x.generate_layer(layers_type=object_type_name) -``` - -The `generate_layer` function generates a collection of layers, each customized to one instance of the input `object_type_name`. -Valid types include `group`, `mitigation`, `software`, and `datasource`. - -## layerExporter_cli.py - -This command line tool allows users to convert a [navigator](https://github.com/mitre-attack/attack-navigator) -layer file to either an svg image or excel file using the functionality provided by the navlayers module. -Details about the SVG configuration json mentioned below can be found in the -[SVGConfig](https://github.com/mitre-attack/mitreattack-python/blob/master/mitreattack/navlayers/README.md#svgconfig) -entry within the navlayers module documentation. - -```commandline -C:\Users\attack>layerExporter_cli -h -usage: layerExporter_cli [-h] -m {svg,excel} [-s {taxii,local,remote}] - [--resource RESOURCE] -o OUTPUT [OUTPUT ...] - [-l LOAD_SETTINGS] [-d WIDTH HEIGHT] - input [input ...] - -Export an ATT&CK Navigator layer as a svg image or excel file - -positional arguments: - input Path(s) to the file to export - -optional arguments: - -h, --help show this help message and exit - -m {svg,excel}, --mode {svg,excel} - The form to export the layers in - -s {taxii,local,remote}, --source {taxii,local,remote} - What source to utilize when building the matrix - --resource RESOURCE Path to the local resource if --source=local, or url - of an ATT&CK Workbench instance if --source=remote - -o OUTPUT [OUTPUT ...], --output OUTPUT [OUTPUT ...] - Path(s) to the exported svg/xlsx file - -l LOAD_SETTINGS, --load_settings LOAD_SETTINGS - [SVG Only] Path to a SVG configuration json to use - when rendering - -d WIDTH HEIGHT, --size WIDTH HEIGHT - [SVG Only] X and Y size values (in inches) for SVG - export (use -l for other settings) - -C:\Users\attack>layerExporter_cli -m svg -s taxii -l settings/config.json -o output/svg1.json output/svg2.json files/layer1.json files/layer2.json -``` - -## layerGenerator_cli.py - -This command line tool allows users to generate [ATT&CK Navigator](https://github.com/mitre-attack/attack-navigator) -layer files from either a specific group, software, or mitigation. Alternatively, users can generate a layer file with a -mapping to all associated groups, software, or mitigations across the techniques within ATT&CK. - -```commandline -C:\Users\attack>layerGenerator_cli -h -usage: layerGenerator_cli [-h] - (--overview-type {group,software,mitigation,datasource} | --mapped-to MAPPED_TO | --batch-type {group,software,mitigation,datasource}) - [-o OUTPUT] [--domain {enterprise,mobile,ics}] - [--source {taxii,local,remote}] - [--resource RESOURCE] - -Generate an ATT&CK Navigator layer - -optional arguments: - -h, --help show this help message and exit - --overview-type {group,software,mitigation,datasource} - Output a layer file where the target type is - summarized across the entire dataset. - --mapped-to MAPPED_TO - Output layer file with techniques mapped to the given - group, software, mitigation, or data component. Argument - can be name, associated group/software, or ATT&CK ID. - --batch-type {group,software,mitigation,datasource} - Output a collection of layer files to the specified - folder, each one representing a different instance of - the target type. - -o OUTPUT, --output OUTPUT - Path to the output layer file/directory - --domain {enterprise,mobile,ics} - Which domain to build off of - --source {taxii,local,remote} - What source to utilize when building the layer files - --resource RESOURCE Path to the local resource if --source=local, or url - of an ATT&CK Workbench instance if --source=remote - -C:\Users\attack>layerGenerator_cli --domain enterprise --source taxii --mapped-to S0065 --output generated_layer.json -C:\Users\attack>layerGenerator_cli --domain mobile --source taxii --overview-type mitigation --output generated_layer2.json -C:\Users\attack>layerGenerator_cli --domain ics --source taxii --batch-type software -C:\Users\attack>layerGenerator_cli --domain enterprise --source taxii --overview-type datasource --output generated_layer3.json -``` \ No newline at end of file +If you wish to read more about the Navlayers module, please click [here](https://mitreattack-python.readthedocs.io/en/126-docs-add-section-to-docs-for-accessing-stix/navlayers.html) for more information. \ No newline at end of file diff --git a/mitreattack/navlayers/core/README.md b/mitreattack/navlayers/core/README.md index 98e101b2..a73068c3 100644 --- a/mitreattack/navlayers/core/README.md +++ b/mitreattack/navlayers/core/README.md @@ -1,242 +1,3 @@ # Layers Core -This subcomponent, as part of the larger navlayers module, is responsible for Layer objects. Please note, this -documentation assumes familiarity with the [ATT&CK Navigator layer format](https://github.com/mitre-attack/attack-navigator/blob/develop/layers/LAYERFORMATv4_1.md). -The main handle for this implementation is the Layer, which stores an individual instance of a LayerObj object, -which further references the various sub-objects that make up a complete Layer. A visual representation of this -object breakdown can be seen here (please note there are other fields, these are just the objects): -``` -demo (Layer instance) <------------------------------------------------> The container for a layer object - |---> demo.layer (_LayerObj instance)--------------------------------> The raw layer object itself - |---> demo.layer.version (Versions instance)-----------------> A versions object - |---> demo.layer.filters (Filter instance)-------------------> A filter object - |---> demo.layer.layout (Layout instance)--------------------> A layout object - |---> demo.layer.techniques (List of Technique instances)----> A collection of technique objects - |---> demo.layer.gradient (Gradient instance)----------------> A gradient object - |---> demo.layer.legendItems (List of LegendItem instances)--> A collection of legend item objects - |---> demo.layer.metadata (List of Metadata instances)-------> A collection of metadata objects -``` -## Creating Layers Programmatically -With knowledge of the objects involved, as well as the additional fields (which have a 1:1 mapping with the -default ATT&CK Navigator spec), it is possible to programmatically generate a layer. Below is an example of -how this might be accomplished, piece by piece. - -```python -import mitreattack.navlayers as navlayers - -layer_example = navlayers.Layer() -layer_example.from_dict(dict(name="example", domain="enterprise-attack")) # arguments required for every layer - -# configure the versions object -layer_example.layer.versions = dict(layer="4.2", attack="9.1", navigator="4.2") - -# set a description -layer_example.layer.description = "This is a demonstration of how to set up a layer piece by piece" - -# configure the "filters" object -layer_example.layer.filters = dict(platforms=['macOS']) # platforms can be provided during initialization -layer_example.layer.filters.platforms = ['Windows'] # or separately - -# configure the 'sorting' setting -layer_example.layer.sorting = 3 # 0: sort ascending alphabetically by technique name -# 1: sort descending alphabetically by technique name -# 2: sort ascending by technique score -# 3: sort descending by technique score - -# configure the layout object -layer_example.layer.layout = dict(layout="side", - showID=True, - showName=True, - showAggregateScores=True, - countUnscored=True, - aggregateFunction="sum") # average, sum, max, min - -# configure whether or not to hide disabled techniques -layer_example.layer.hideDisabled = True -# configure the gradient object -layer_example.layer.gradient = dict(minValue=-100, maxValue=100, - colors=["#DAF7A6", "#FFC300", "#FF5733", "#C70039", "#900C3F", "#581845"]) -# configure collection of legend items -layer_example.layer.legendItems = [dict(label='A', color='#DAF7A6'), dict(label='B', color='#581845')] -# configure collection of metatdata values -layer_example.layer.metadata = [dict(name='example metadata', value='This is an example')] -# create listing of techniques in this layer -layer_example.layer.techniques = [dict(techniqueID='T1000', tactic='privilege-escalation', score=15, color='#AABBCC'), - dict(techniqueID='T1000.1', tactic='privilege-escalation', score=1, comment='Demo')] -``` -This first example utilizes the native dictionary form for initializing the layer. This approach is similar to the -method used by the automated import process, but may not be the most intuitive for users. An alternative method, -displayed below, is to create and modify instances of the core objects in the library. Please note, these two examples -produce equivalent internal layers once completed. - -```python -import mitreattack.navlayers as navlayers - -layer_example = navlayers.Layer(name="example", domain="enterprise-attack") # arguments required for every layer -layer_build = layer_example.layer # short handle to make the rest of this example easier to read - -# configure the versions object -versions_obj = navlayers.Versions() -versions_obj.layer = "4.2" -versions_obj.attack = "9.1" -versions_obj.navigator = "4.2" -layer_build.versions = versions_obj - -# set a description -layer_build.description = "This is a demonstration of how to set up a layer piece by piece" - -# configure the "filters" object -filter_obj = navlayers.core.Filter(domain="enterprise-attack") -filter_obj.platforms = ['Windows'] -layer_build.filters = filter_obj - -# configure the 'sorting' setting -layer_build.sorting = 3 # 0: sort ascending alphabetically by technique name -# 1: sort descending alphabetically by technique name -# 2: sort ascending by technique score -# 3: sort descending by technique score - -# configure the layout object -layout_obj = navlayers.core.Layout() -layout_obj.layout = "side" -layout_obj.showID = True -layout_obj.showName = True -layout_obj.showAggregateScores = True -layout_obj.countUnscored = True -layout_obj.aggregateFunction = "sum" # average, sum, max, min -layer_build.layout = layout_obj - -# configure whether or not to hide disabled techniques -layer_build.hideDisabled = True - -# configure the gradient object -gradient_obj = navlayers.core.Gradient(colors=["#DAF7A6", "#FFC300", "#FF5733", "#C70039", "#900C3F", "#581845"], - minValue=-100, maxValue=100) -layer_build.gradient = gradient_obj - -# configure collection of legend items -legend_item_obj_a = navlayers.core.LegendItem(label='A', color='#DAF7A6') -legend_item_obj_b = navlayers.core.LegendItem(label='B', color='#581845') -list_of_legend_items = [legend_item_obj_a, legend_item_obj_b] -layer_build.legendItems = list_of_legend_items - -# configure collection of metatdata values -metadata_object = navlayers.core.Metadata(name='example metadata', value='This is an example') -layer_build.metadata = [metadata_object] - -# create listing of techniques in this layer -technique_obj_a = navlayers.core.Technique(tID='T1000') -technique_obj_a.tactic = 'privilege-escalation' -technique_obj_a.score = 15 -technique_obj_a.color = '#AABBCC' -technique_obj_b = navlayers.core.Technique(tID='T1000.1') -technique_obj_b.tactic = 'privilege-escalation' -technique_obj_b.score = 1 -technique_obj_b.comment = "Demo" -layer_build.techniques = [technique_obj_a, technique_obj_b] - -``` - -### Object Documentation -Should it be helpful, the following section provides a breakdown of the available fields and methods for -each of the objects in the Core. This only includes 'public' methods and fields; there may be others used -for processing and other functionality that are not documented here, though documentation does exist for these -in the source code for them. - -#### Layer Object -```python - Layer().layer # Stores the raw LayerObj file - Layer().strict # Determines whether or not to be strict about loading files - Layer().from_str() # Initializes data from a string - Layer().from_dict() # Initializes data from a dictionary - Layer().from_file() # Initializes data from a file - Layer().to_file() # Exports the layer data to a file - Layer().to_dict() # Exports the layer data to a dictionary - Layer().to_str() # Exports the layer data to a string - ``` -#### LayerObj Object -```python - _LayerObj().versions # Link to a Versions object instance - _LayerObj().name # The Name for the Layer - _LayerObj().description # A description string for the Layer - _LayerObj().domain # The domain for the Layer - _LayerObj().filters # Link to a Filter object instance - _LayerObj().sorting # An integer denoting which sorting form to use - _LayerObj().layout # Link to a Layout object instance - _LayerObj().hideDisabled # Bool determining whether or not to show disabled techniques - _LayerObj().techniques # List of links to Technique objects - _LayerObj().gradient # Link to Gradient object - _LayerObj().legendItems # List of links to LegendItems objects - _LayerObj().showTacticRowBackground # Bool determining whether or not to show a background for tactics - _LayerObj().tacticRowBackground # Color code for tactic background - _LayerObj().selectTechniquesAcrossTactics # Bool determining whether or not to select cross-tactic - _LayerObj().selectSubtechniquesWithParent # Bool determining whether or not to select subtechniques - _LayerObj().metadata # List of links to Metadata items - _LayerObj().get_dict() # Export Layer as a dictionary object -``` -#### Versions Object -```python - Versions().layer # String denoting Layer format version - Versions().__attack # String denoting ATT&CK version - Versions().navigator # String denoting Navigator version - Versions().get_dict() # Export Version data as a dictionary object -``` -#### Filter Object -```python - Filter().domain # String denoting the domain for the Filter - Filter().platforms # String denoting platforms within this filter - Filter().get_dict() # Export Filter data as a dictionary object -``` -Please note that although not documented here, there is another Filter object variant, Filterv3, which exists -for backwards compatibility reasons. -#### Layout Object -```python - Layout().layout # String denoting which layout form to use - Layout().showID # Bool denoting whether or not to show technique IDs - Layout().showName # Bool denoting whether or not to show technique names - Layout().showAggregateScores # Bool denoting whether or not to utilize Aggregate scores - Layout().countUnscored # Bool denoting whether ot not to count unscored techniques as 0s for Aggregates - Layout().aggregateFunction # A enum integer denoting which aggregate function to utilize - # 1 - Average, 2 - min, 3 - max, 4 - sum - Layout().get_dict() # Export Layout data as a dictionary object - Layout().compute_aggregate() # Compute the aggregate score for a technique and it's subtechniques -``` -#### Technique Object -```python - Technique().techniqueID # String denoting the technique's ID - Technique().tactic # String denoting the technique's tactic - Technique().comment # String denoting any comments - Technique().enabled # Bool denoting if the technique is enabled - Technique().score # Integer denoting technique score - Technique().aggregateScore # Integer denoting pre-configured aggregate score - Technique().color # String denoting manually configured color code - Technique().metadata # List of links to metadata objects - Technique().showSubtechniques # Bool denoting whether or not to show subtechniques - Technique().get_dict() # Export Technique data as a dictionary object -``` -#### Gradient Object -```python - Gradient().colors # Array of colors (string codes) over which the gradient is to be calculated - Gradient().minValue # Integer denoting minimum viable value on the gradient - Gradient().maxValue # Integer denoting maximum viable value on the gradient - Gradient().compute_color() # Calculate the appropriate color for a given score on the gradient - Gradient().get_dict() # Export Gradient data as a dictionary object -``` -#### LegendItem Object -```python - LegendItem().label # String denoting the label for this Legend Item' item - LegendItem().color # String denoting the color code for the Legend Item - LegendItem().get_dict() # Export Legend Item data as a dictionary object -``` -#### Metadata/Metadiv Object -```python - Metadata().name # String denoting metadata keypair name - Metadata().value # String denoting metadata keypair value - Metadata().get_dict() # Export metadata data as a dictionary object -``` -```python - Metadiv().name # Always set to "DIVIDER" - Metadiv().value # Bool denoting active or not - Metadiv().get_dict() # Export metadiv as a dictionary object -``` -A `Metadiv` object is simply a modified version of a `Metadata` object used as a visual divider. \ No newline at end of file +If you wish to read more about the Nav Layers Core submodule, please click [here](https://mitreattack-python.readthedocs.io/en/126-docs-add-section-to-docs-for-accessing-stix/navlayercore.html) for more information. \ No newline at end of file