Housekeeping and Testing

[plasticuproject]

• Updated example/default feed to use OTX AlienVault/AT&T Cybersecurity, as EclecticIQ OpenTAXII is no longer operational
• Updated dependencies
• Added type annotations passing strict static mypy type checking for local code
• Minor formatting improvements including reordering imports and changing string formatting to use modern fstrings.
• Added basic pytest tests (coverage is low, still need to mock more data/tests for external api calls, this is just a start), flake8 linting checks, and mypy strict type checks in a github workflow

NOTE:
I was not able to test the following type consistency change as I do not have access to an ExtraHop server. If it fails just revert back to the commented out variable and call and ignore the type error:

    # file_body = {'file': (file_name, open(file_path, 'rb')), 'name': threatcollection_name}
    file_handle: BinaryIO
    with open(file_path, 'rb') as file_handle:
        files = {'file': (file_name, file_handle)}
        data = {'name': threatcollection_name}

    # log InsecureRequestWarning if making an unverified https request
    if not eh_verify_cert:
        logging.warning("InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning")

    try:
        # send PUT request to create or update
        # r = requests.put(url, headers=headers, files=file_body, verify=eh_verify_cert)
        r = requests.put(url, headers=headers, files=files, data=data, verify=eh_verify_cert)

[Dan-at-Extrahop]

Thanks for taking the time to contribute to the project @plasticuproject! Much appreciated! I'll have the team look over the changes and test the updated file upload bits that you called out.

The reason the project has been a bit dormant lately is that since ExtraHop version 9.6 we added the ability to configure a TAXII feed from within the ExtraHop UI. Check out the following blog and docs if you're interested:

• https://www.extrahop.com/blog/extrahop-demonstrates-aiml-prowess-in-latest-revealx-updates
• https://docs.extrahop.com/9.7/threat-upload/#add-a-taxii-feed

Regardless, this script/toolkit is still useful for backwards compatibility and also specific local filtering or refinement use cases.