Skip to content

Create gsoc25_refactor_analyzer_tests.md #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions Blogs/gsoc25_refactor_analyzer_tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
title: Migrating IntelOwl Analyzer Tests
date: 2025-08-25
cover: /images/gsoclogo.png
author: Pranjal Gupta

# Migrating IntelOwl Analyzer Tests: From Monkeypatching to a Structured Unit Testing Framework

For IntelOwl, which integrates dozens of analyzers to process a wide range of observables, maintaining robust and reliable tests is crucial. This summer, during my GSoC project, I worked on **migrating IntelOwl’s analyzer tests** from a **monkeypatch-heavy approach** to a **structured, maintainable unit testing framework** using Python’s built-in `unittest` module.

The goal was to improve reliability, reduce code duplication, and provide contributors with a simpler way to write and extend tests.

---

## 🌱 Pre-GSoC Contributions

Before being selected for GSoC, I was already actively contributing to IntelOwl and its ecosystem. These contributions not only helped me understand the codebase deeply but also demonstrated my commitment to the project.

Here are some of my notable pre-GSoC deliverables:

- **IntelOwl analyzers**:
- [UltraDNS Analyzer (#2620)](https://github.com/intelowlproject/IntelOwl/pull/2620)
- [Nuclei Analyzer (#2697)](https://github.com/intelowlproject/IntelOwl/pull/2697)
- [Alter DNS Malicious Detector (#2753)](https://github.com/intelowlproject/IntelOwl/pull/2753)
- [Spamhaus IPv6 and ASN Support (#2761)](https://github.com/intelowlproject/IntelOwl/pull/2761)
- [ShellCheck Analyzer (#2775)](https://github.com/intelowlproject/IntelOwl/pull/2775)
- [Malshare Ingestor (#2804)](https://github.com/intelowlproject/IntelOwl/pull/2804)

- **IntelOwl Integrations & SDKs**:
- [GreedyBear Ingestor (#2709)](https://github.com/intelowlproject/IntelOwl/pull/2709)
- [Playbook Support (#70)](https://github.com/intelowlproject/go-intelowl/pull/70)
- [Pre-commit hooks for SDK (#71)](https://github.com/intelowlproject/go-intelowl/pull/71)
- [Job List Error Fix (#263)](https://github.com/intelowlproject/pyintelowl/pull/263)
- [Pages Flag for Listing Jobs (#265)](https://github.com/intelowlproject/pyintelowl/pull/265)
- [HealthCheck URL Correction (#266)](https://github.com/intelowlproject/pyintelowl/pull/266)

These early contributions gave me strong familiarity with IntelOwl’s analyzers, ingestors, SDKs, and testing setup. Naturally, this became the foundation for my GSoC work on **refactoring analyzer tests**.

---

## 🔎 The Problem

Previously, IntelOwl relied heavily on **monkeypatching** for testing analyzers. While it worked, it came with drawbacks:

- Limited flexibility for adding **custom tests** for complex analyzers.
- Harder for new contributors to extend or debug.

We needed a **systematic, scalable approach**—one that could:
1. Cover all analyzers with all supported observable types.
2. Reduce duplication by leveraging reusable test classes.
3. Support both automated and custom tests.
4. Make future contributions easier.

---

## 🛠 My GSoC Work

I approached the migration in clear phases:

- **Framework Design:** Built a reusable superclass for analyzer tests, eliminating duplication and replacing monkeypatch-heavy logic.
- **Expanding Coverage:** Extended the framework to cover all analyzers, ensuring they worked with all supported observable types.
- **Documentation:** Wrote contributor-friendly guidelines to help future maintainers and contributors easily extend tests.
- **Polishing:** Final refinements, debugging, and optimization to make the framework stable and efficient.

By the end of GSoC, IntelOwl’s analyzers are now backed by a **clean, extensible, and fully documented unit testing framework**.

👉 Related PRs from my GSoC work:
- [IntelOwl: Refactor Analyzer Tests (#2886)](https://github.com/intelowlproject/IntelOwl/pull/2886)
- [Docs: Updated Testing Documentation (#38)](https://github.com/intelowlproject/docs/pull/38)
---

## 🚀 The Outcome

Key achievements:
-**100% analyzer coverage** with unit tests.
-**Monkeypatching eliminated** (where unnecessary).
-**Custom analyzer tests** applicable for complex cases.
-**Contributor-friendly documentation** for future maintainers and contributors.

---

## 💭 Reflections

Being part of GSoC with IntelOwl has been a truly rewarding journey. From my first pre-GSoC contributions to completing a full-scale migration of the testing framework, I’ve grown both technically and personally. It was an incredible learning experience—improving my skills in Python testing, open-source collaboration, and large-scale project design.

More than anything, I feel proud to have contributed to making IntelOwl more reliable and sustainable for the community. This journey has reinforced my passion for open source.

---

## 🙌 Acknowledgements

A huge thanks to the IntelOwl maintainers for guidance and feedback throughout the migration process.

---