Migale was born out of a need to extract quickly and with a very low development cost. This package is not intended to replace complete and structured libraries like DotnetSpider.
- Multi-Threaded
- Fail & retries handling
- Event-Driven
- Extensible
- Document parsing (HTML, JSON, XML) (Work in progress !)
You can find samples with the crawlers implementation here !
| Package | Description | Nuget |
|---|---|---|
| Migale.Core | The core of the project |  |
| Migale.Crawlers.HttpClient | The HttpClient crawler implementation with RequestMessages |  |
| Migale.Crawlers.Playwright | The Playwright crawler implementation with browser automation |  |
We would love community contributions here.
There is actually no contributation guide or convention as i'm actually the only maintainer of the project.
This project is licensed with the MIT license.
You should take a look at these related projects: