Skip to content

cannot access local variable 'main_content' where it is not associated with a value #11

@assumednormal

Description

@assumednormal

i just ran into this issue when crawling https://docs.xendit.co/.

command:

markdown-crawler --max-depth 10 --num-threads 5 --base-dir ./xendit-docs --domain-match --base-path-match https://docs.xendit.co/

issue:

Exception in thread Thread-1 (worker):
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 255, in worker
    child_urls = crawl(
                 ^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 120, in crawl
    content = get_target_content(soup, target_content=target_content)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/markdown_crawler/__init__.py", line 184, in get_target_content
    content = str(main_content)
                  ^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'main_content' where it is not associated with a value

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions