Pre/beta - Unit Tests

[codebeaver-ai]

CodeBeaver Report

I started working from Pre/beta

🔄 2 test files added.
🐛 Found 1 bug
🛠️ 91/137 tests passed

🔄 Test Updates

I've added 2 tests. They all pass ☑️
New Tests:

• tests/test_chromium.py
• tests/test_cleanup_html.py

No existing tests required updates.

🐛 Bug Detection

Potential issues:

• scrapegraphai/utils/proxy_rotation.py
  After analyzing the source code, tests, and error log, it appears that the errors are caused by bugs in the code being tested, specifically in the parse_or_search_proxy function. Let's break down the issues:

1. In test_parse_or_search_proxy_success, the function is raising a ValueError for a valid proxy server format. The function is not correctly parsing the IP address and port combination.
2. In test_parse_or_search_proxy_exception, the error message has changed from "missing server in the proxy configuration" to "Missing 'server' field in the proxy configuration." This indicates that the assertion message in the code has been updated, but the test hasn't been updated to match.
3. In test_parse_or_search_proxy_unknown_server, the function is raising a ValueError instead of the expected AssertionError. The function is not correctly handling unknown server types as intended.
   These issues point to problems in the implementation of parse_or_search_proxy:

• It's not correctly handling IP:port combinations in the server field.
• The error message for a missing server field has been changed without updating the corresponding test.
• It's not correctly asserting for unknown server types as the test expects.
  To fix these issues, the parse_or_search_proxy function needs to be revised to correctly handle different server formats and raise the appropriate exceptions as expected by the tests.

Test Error Log

tests.utils.test_proxy_rotation#test_parse_or_search_proxy_success: def test_parse_or_search_proxy_success():
        proxy = {
            "server": "192.168.1.1:8080",
            "username": "username",
            "password": "password",
        }
    
>       parsed_proxy = parse_or_search_proxy(proxy)
tests/utils/test_proxy_rotation.py:82: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
proxy = {'password': 'password', 'server': '192.168.1.1:8080', 'username': 'username'}
    def parse_or_search_proxy(proxy: Proxy) -> ProxySettings:
        """
        Parses a proxy configuration or searches for a matching one via broker.
        """
        assert "server" in proxy, "Missing 'server' field in the proxy configuration."
    
        parsed_url = urlparse(proxy["server"])
        server_address = parsed_url.hostname
    
        if server_address is None:
>           raise ValueError(f"Invalid proxy server format: {proxy['server']}")
E           ValueError: Invalid proxy server format: 192.168.1.1:8080
scrapegraphai/utils/proxy_rotation.py:200: ValueError
tests.utils.test_proxy_rotation#test_parse_or_search_proxy_exception: def test_parse_or_search_proxy_exception():
        proxy = {
            "username": "username",
            "password": "password",
        }
    
        with pytest.raises(AssertionError) as error_info:
            parse_or_search_proxy(proxy)
    
>       assert "missing server in the proxy configuration" in str(error_info.value)
E       assert 'missing server in the proxy configuration' in "Missing 'server' field in the proxy configuration."
E        +  where "Missing 'server' field in the proxy configuration." = str(AssertionError("Missing 'server' field in the proxy configuration."))
E        +    where AssertionError("Missing 'server' field in the proxy configuration.") = <ExceptionInfo AssertionError("Missing 'server' field in the proxy configuration.") tblen=2>.value
tests/utils/test_proxy_rotation.py:110: AssertionError
tests.utils.test_proxy_rotation#test_parse_or_search_proxy_unknown_server: def test_parse_or_search_proxy_unknown_server():
        proxy = {
            "server": "unknown",
        }
    
        with pytest.raises(AssertionError) as error_info:
>           parse_or_search_proxy(proxy)
tests/utils/test_proxy_rotation.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
proxy = {'server': 'unknown'}
    def parse_or_search_proxy(proxy: Proxy) -> ProxySettings:
        """
        Parses a proxy configuration or searches for a matching one via broker.
        """
        assert "server" in proxy, "Missing 'server' field in the proxy configuration."
    
        parsed_url = urlparse(proxy["server"])
        server_address = parsed_url.hostname
    
        if server_address is None:
>           raise ValueError(f"Invalid proxy server format: {proxy['server']}")
E           ValueError: Invalid proxy server format: unknown
scrapegraphai/utils/proxy_rotation.py:200: ValueError

☂️ Coverage Improvements

Coverage improvements by file:

• tests/test_chromium.py

  New coverage: 17.24%
  Improvement: +17.24%

• tests/test_cleanup_html.py

  New coverage: 0.00%
  Improvement: +5.00%

🎨 Final Touches

• I ran the hooks included in the pre-commit config.

---

_{Settings | Logs | CodeBeaver}

[github-actions]

🎉 This PR is included in version 1.47.0-beta.1 🎉

The release is available on:

• v1.47.0-beta.1
• GitHub release

Your semantic-release bot 📦🚀

[github-actions]

🎉 This PR is included in version 1.47.0 🎉

The release is available on:

• v1.47.0
• GitHub release

Your semantic-release bot 📦🚀