Deep search gatherer

Disclaimer

Since i did not have time to update this project with all the major changes on the libraries used for twitter/twitch/etc, it is dead for now. If you try it, it will probably simply don't work.

Prerequisite

• Docker Compose version v2.6.0 or higher

Introduction

This project is a full featured RSS discord bot using 2 services over a docker compose. It is used to track RSS feeds / youtube channels / twitter accounts/ twitch channels / subreddit or common websites and post it over discord channels

• The discord information gatherer bot who track RSS feeds, youtube channel, twitter accounts and subreddit and publish it on discord
• The scrape2RSS who generate an RSS feed based on a simple url

Features

example url that can be tracked (you can put this in your config file to try it)

• A Twitter account @MyTwitterAccount
• A Twitch channel tw/TwitchChannel
• A Subreddit r/MySubreddit
• A very common RSS feed url https://www.cert.ssi.gouv.fr/alerte/feed/
• A youtube channel https://www.youtube.com/c/LiveOverflow
• A static webpage https://www.synacktiv.com/publications

Disclaimer

This project need docker to be installed and this is not intended as an introduction to docker.

If you are unfamiliar with Docker, check out the Introduction to Docker webinar, or consult your favorite search engine.

Installation

Copy the config-example.yaml file, change all the option you need and put the file into config_file_goes_in_here/ directory

Then you can simply start docker compose

docker compose up -d

Create the config file

The configuration file is compatible with either json and yaml (or yml) format. View the documentation here for json usage.
Put your configuration file in the config_file_goes_in_here/ directory

Bot Commands

To interact with the bot, simply tag the bot at the beggining of the message(@Information Gatherer by default)

Commands	Explanations
add	No Aliases Parameters: - just give the desired url after specifying the "add" command - channel: the channel ID where you want to post news on (you need to enable dev mode in your discord settings) - name: (optional) Chose a name for the registered feed
delete	Aliases: del / dl Take the name of the feed to delete as parameter
help	Aliases: -h / `` (no params) Display the help menu
list	List all the registered feeds in your server

Examples

Add Youtube channel or an RSS feed

@Information Gatherer add https://www.youtube.com/c/LiveOverflow channel 1009496824605843607

@Information Gatherer add https://www.youtube.com/c/LiveOverflow channel 1009496824605843607 name LiveOverflow

Add Twitter

@Information Gatherer add @LiveOverflow channel 1009496824605843607

Add Reddit

@Information Gatherer add r/netsec channel 1009496824605843607

Delete a feed

@Information Gatherer del LiveOverflow

List feeds

@Information Gatherer ls

Troubleshooting

If the name of you channel/account is parsed by the discord markdown (for exemple: _my_channel_) you can use backticks arround the name to escape it

@Information Gatherer add `@LiveOverflow` channel 1009496824605843607

Minimal configuration needed

token: <TOKEN>

Configuration parameters

Parameters	Explanation	Default value
token	Your bot token, it's mandatory variable.	""
refresh-time	Time between refreshes of a feed, in second	900
published_since_default	Maximum age of news before it's discarded, in second. Used only when published_since of a feed is not set. If published_since_default or published_since are equal to 0, only posts published after the initialization of this bot will be sent (usefull in case you use Scrape2RSS feature)	0
gameplayed	Change the game displayed in bot profile	"Eating some RSS feeds"
twitter	enabled (default: False) - Enable the feature bearer_token (default: "") Needed to auth the Twitter API	[]
reddit	enabled (default: False) - Enable the feature client_id (default: "") Needed to auth the Reddit API client_secret (default: "") Needed to auth the Reddit API password (default: "") Needed to auth the Reddit account for accessing reddit data username (default: "") Needed to auth the Reddit account for accessing reddit data	[]
twitch	enabled (default: False) - Enable the feature client_id (default: "") Needed to auth the Twitch API client_secret (default: "") Needed to auth the Twitch API	[]

Scrape2RSS feature

If you want to follow a website that doesn't have an RSS feed, submit the URL of the page in the url parameter like a normal feed. To be used, you need to set the full bot project

Youtube Feature

To follow a youtube channel just put the youtube url in the url field.

Format tested:

• https://www.youtube.com/c/<Channel-Name>
• https://www.youtube.com/user/<Channel-Name>