XTB dividend analysis

A production-grade Python pipeline, engineered as a senior-level testing showcase — 530+ tests across 8 independent tiers: unit, integration, property-based, fuzz, metamorphic, contract, security, and mutation.

A Python data pipeline that parses XTB broker statements (.xlsx), converts foreign dividends to PLN using NBP D-1 exchange rates, calculates Polish Belka tax (19%) with WHT deduction, and exports a tab-separated CSV ready for Google Sheets. The pipeline is a real-world problem; the repository is a showcase of risk-based, multi-layer test engineering.

---

Testing strategy

Each tier was chosen deliberately — it catches a class of defect the others cannot. This is defense-in-depth applied to test design: example-based tests pin the happy path, property-based tests generalise it, metamorphic tests replace the missing oracle, fuzz tests probe hostile input, contract tests guard the data boundary, and mutation tests verify the tests themselves are actually asserting something.

Tier	Files	Framework	What it catches that nothing else does
Unit	17	pytest + unittest.mock	Logic defects in every specialist class in isolation; all I/O mocked
Integration	6	pytest + real DataFrames	Wiring defects — end-to-end pipeline against real XLSX fixtures and NBP CSV files
Property-based	2	hypothesis (100 examples/run)	Violated mathematical invariants in tax, FX, and date logic that hand-picked cases miss
Metamorphic	3	hypothesis + relation asserts	Incorrect behaviour when there is no ground-truth oracle — permutation, duplication, additivity, linear scaling relations
Fuzz	2	hypothesis (binary + text)	Crashes, hangs, and silent corruption on hostile XLSX bytes and malformed strings; whitelisted exceptions only
Contract	2	pandera	Silent broker format drift — schema tripwires on raw XTB XLSX and the exported Google-Sheets CSV
Security	2	bandit + SARIF parser	Insecure code patterns; SARIF output structure + severity mapping for GitHub Security tab integration
Mutation	—	mutmut	Test-suite weakness — whether the assertions would actually fail if the production code were broken

All tests follow the AAA pattern (Arrange / Act / Assert, blank line between sections) and are named test_<unit>_<scenario>_<expected_outcome>. Run the full suite or a single tier:

poetry run pytest                        # full suite
poetry run pytest -m property_based      # one tier at a time
poetry run pytest -m metamorphic
poetry run pytest -m fuzz
poetry run pytest -m contract
poetry run tox                           # Python 3.9–3.13 locally

---

Quality engineering toolkit

Skills and practices applied throughout — each one is visible in the repo, not just listed.

• Risk-based test design — 8 tiers, each targeting a distinct failure mode, not overlapping coverage
• Property-based testing — generative invariants over tax, FX, and date logic with hypothesis; shrinks to minimal counter-examples automatically
• Metamorphic testing — tolerates the absence of an oracle by asserting relations between runs (permutation, duplication, additivity, scaling)
• Fuzz testing — hostile binary and text input strategies; whitelisted exception set prevents silent regressions
• Schema contracts — pandera tripwires on both input and output data, failing loudly on upstream format drift
• Mutation testing — mutmut validates that the suite actually catches broken code; skip-rules pragma-marked inline for auditability
• Static analysis stack — mypy strict mode, ruff, bandit with SARIF upload to GitHub Security, safety dependency CVE scanning
• Deterministic CI — dependency cache keyed on lockfile hash; dorny/paths-filter skips test job when no Python changed; JUnit XML + Codecov artifacts
• Test hygiene — AAA pattern enforced; pytest-randomly catches order-dependent tests; strict markers; deprecation warnings escalated to errors for first-party modules only
• Structured logging — loguru throughout; zero print() calls in source
• Type-safe configuration — pydantic-settings; no hardcoded paths, rates, or URLs in source

---

CI/CD pipeline

ci.yml — quality gates

Runs on ubuntu-latest with Python 3.13. Twelve jobs gated by a fail-fast quality stage: lint → type-check → unit → integration → property-based → metamorphic → fuzz → contract → security → coverage → mutation-smoke. Artifacts: JUnit XML (dorny/test-reporter), Codecov upload with PR comment, Bandit SARIF report to the GitHub Security tab. Lint and security jobs use continue-on-error: true so a style warning never blocks a merge; functional test jobs are strict.

fast-tests.yml — smoke feedback loop

Ubuntu + Python 3.12, sub-60-second feedback before the full pipeline completes. Used as a pre-push sanity check.

Local parity

poetry run tox runs the suite against Python 3.9–3.13 locally so drift between developer machine and CI is caught pre-commit.

---

Architecture — facade orchestrator + delegate-then-assign specialists

DataFrameProcessor is a facade that owns self.df and delegates every transformation to a stateless specialist class. Each step follows delegate-then-assign:

specialist = SpecialistClass(self.df)
self.df = specialist.method()

Step	Specialist class	Responsibility
1	ColumnNormalizer	Maps bilingual (PL/EN) column names to canonical English names via ColumnName enum
2	DividendFilter	Filters dividend and WHT rows; groups by ticker, date, and comment
3	DataAggregator	Merges split rows, moves negative WHT values to a dedicated column, reorders columns
4	CurrencyConverter	Detects account currency from XLSX cell F6; looks up NBP D-1 mid-rate for each payment date
5	TaxExtractor	Parses WHT percentage from free-text comment strings using MultiConditionExtractor
6	TaxCalculator	Computes Belka tax: gross × 0.19 − WHT_paid in PLN; handles both USD and PLN accounts
7	ColumnFormatter	Applies ANSI ticker colorization, formats display columns, appends currency labels

All domain constants (ColumnName, Currency, TickerSuffix) are enums in data_processing/constants.py. No raw string literals for column names or currency codes appear anywhere in source.

Getting started — install, run, export to Google Sheets

Prerequisites

• Python 3.13 or later
• Poetry for dependency management
• Chromium (installed automatically by Playwright during setup)

Installation

git clone https://github.com/darekwojciechowski/xtb-dividend-analysis.git
cd xtb-dividend-analysis
poetry install
poetry run playwright install chromium

Usage

Step 1 — Download NBP exchange rate archives. The files are saved to data/ as archiwum_tab_a_<YYYY>.csv.

poetry run python data_acquisition/playwright_download_currency_archive.py

Step 2 — Process the broker statement. Place your XTB .xlsx in data/, then run:

poetry run python main.py

Output is written to output/for_google_spreadsheet.csv.

Step 3 — Import to Google Sheets. Paste the contents of output/for_google_spreadsheet.csv directly into a sheet. Tab separators are recognized automatically.

Step 4 — Visualize (optional). Feed the exported CSV into the Streamlit Dividend Dashboard.

---

Stack

Core: Python 3.13, pandas, numpy, openpyxl, pydantic-settings, loguru, playwright

Testing: pytest, hypothesis, pandera, mutmut, bandit, safety, tox, pytest-randomly

Tooling: Poetry, ruff, mypy (strict), pre-commit, Codecov, GitHub Actions

License

MIT