[Bug]: KeyError: 'commit' in GithubRepositoryReader

[Jmarks199]

Bug Description

I've tried every example combination I can find for the GithubRepositoryReader and multiple repositories as targets, but I always get the exact same stack trace below. Using the commit SHA did not make a difference, as reported as a fix in a previous bug report. I'm using the latest version of all packages, and my GH token has all repo privileges. There seems to be erroneous config in readers.llama-index-readers-github. Latest commits reference 0.12.0 but the pyproject.toml lists 0.5.0. I retried installation to make sure I have the latest version. In a previous bug report it was stated that this error message was improved, but I do not see evidence of that.

llama-index-readers-github>=0.5.0

import os

from llama_index.readers.github import GithubRepositoryReader, GithubClient

client = GithubClient(github_token=os.environ["GITHUB_TOKEN"], verbose=True)

loader = GithubRepositoryReader(
    github_client=client,
    owner="jerryjliu",
    repo="llama_index",
    use_parser=False,
    verbose=True,
    filter_directories=["examples"],
)

docs = loader.load_data(branch='main')

for doc in docs:
    print(doc.extra_info)

Version

0.12.19

Steps to Reproduce

Run the example code.

Relevant Logs/Tracbacks

Traceback (most recent call last):
  File "/Users/jaredmarks/PycharmProjects/playground/try_example2.py", line 32, in <module>
    documents = reader1.load_data(branch="main")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/llama_index/readers/github/repository/base.py", line 308, in load_data
    return self._load_data_from_branch(branch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/llama_index/readers/github/repository/base.py", line 268, in _load_data_from_branch
    branch_data: GitBranchResponseModel = self._loop.run_until_complete(
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/llama_index/readers/github/repository/github_client.py", line 366, in get_branch
    return GitBranchResponseModel.from_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/dataclasses_json/api.py", line 63, in from_json
    return cls.from_dict(kvs, infer_missing=infer_missing)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/dataclasses_json/api.py", line 70, in from_dict
    return _decode_dataclass(cls, kvs, infer_missing)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaredmarks/PycharmProjects/playground/.venv/lib/python3.12/site-packages/dataclasses_json/core.py", line 185, in _decode_dataclass
    field_value = kvs[field.name]
                  ~~~^^^^^^^^^^^^
KeyError: 'commit'

[dosubot]

Hey there, @Jmarks199! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve this issue!

A similar issue was previously reported and closed. The suggested solution was to ensure that your OAuth token has the proper authorization. In that case, creating a new Personal Access Token (PAT) with the correct permissions resolved the issue ^[1].

Additionally, the KeyError: 'commit' might occur if the commit attribute is not present in the response from the GitHub API. Ensure that the GitCommitResponseModel and GitBranchResponseModel are correctly structured and that the commit attribute is being accessed properly. Verify that the GitHub API is returning the expected data and that the commit_sha provided is valid ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}