Use when you need to scrape something very fast.
python3 -m venv env
source dev.sh
pip install -r requirements.txt
Typing d in the terminal will run the script in debug mode and dump everything into python interactive shell, where you can inspect the variables and objects.
Press q to quit the interactive shell.
Typing r in the terminal will run the script.
write(data, filename='test.html') - writes response data to a file with the given filename. The default filename is test.html. You can change it to whatever you want.
dfs(data, match_fn) - performs a depth-first search on a nested dictionary or list. It takes a function as an argument that returns True if the data matches the condition you are looking for. Returns a tuple of matching data and the path to the matching data.
- Add nested attributes class for converting dicts to objects