Parallelize DNS lookups

[krmaxwell]

Not everybody has DNSDB type availability. Fail gracefully if they don't, preferably with some other source.

[krmaxwell]

Also: speedups by making requests in parallel or using sessions / pipelining?

[krmaxwell]

The original part of this is complete, actually. But we do need to do the speedup.

[juju4]

2 comments:

• free alternative: Add support for CIRCL PassiveDNS #144 and existing python module based on rfc draft (common with farsight) https://github.com/CIRCL/PyPDNS
• implement requests-cache in grequests (basic, need 2 lines of extra code: https://pypi.python.org/pypi/requests-cache)

[juju4]

On DNS resolution, I think
1- it should be optional (be it for performance vs passivedns info, be it for opsec or other reason)
2- can switch to adns in async mode but should be bulk call not one to one
http://www.catonmat.net/blog/asynchronous-dns-resolution/