-
Notifications
You must be signed in to change notification settings - Fork 48
Open
Description
i just ran into this issue when crawling https://docs.xendit.co/
.
command:
markdown-crawler --max-depth 10 --num-threads 5 --base-dir ./xendit-docs --domain-match --base-path-match https://docs.xendit.co/
issue:
Exception in thread Thread-1 (worker):
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 989, in run
self._target(*self._args, **self._kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 255, in worker
child_urls = crawl(
^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 120, in crawl
content = get_target_content(soup, target_content=target_content)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 184, in get_target_content
content = str(main_content)
^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'main_content' where it is not associated with a value
Tiberriver256 and jkarttunen
Metadata
Metadata
Assignees
Labels
No labels