Should we add an anti-porn igset?

[TheTechRobo]

I don't watch, nor do I archive, porn.

Should we add some porn sites to the global igset?

Alternatively should we at least make an "antiporn" igset?

[brandongalbraith]

This sounds antithetical to the idea of digital archival and preservation and something that should probably be left to individuals electing to use grab-site to manually define in their igset config.

[TheTechRobo]

That makes sense. In that case, do you think adding a specialised igset for it would be good?

[brandongalbraith]

@TheTechRobo Hard to say; I've never had the need to ignore adult content in my web archival crawls. If you think there's a need, a course of action might be to spike an initial adult site igset and see what consumption of it by grab-site users (product market fit sort of thing).

[TheTechRobo]

I was archivinng a huuge number of a Reddit user's bookmarks for them; I'm not sure if they watch it themselves or if their bookmarks were just sketchy websites but I saw a LOT of itt in my queue.

Doesn't hurt to add one ig, I'll see if I have the time (and motivation :P)

edit: sorry for the double letters, my keyboard seems to bne running out of battery...