Add Support for Complex Type Annotations in Python Parser #40
Conversation
ShenaoW
requested review from
AntJiuFo,
Arielwyy and
alipaydeshui
as code owners
Summary of ChangesHello @ShenaoW, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the YASA-UAST Python parser by introducing robust support for complex type annotations. It centralizes the parsing logic for various type hints, ensuring that the Universal Abstract Syntax Tree accurately reflects the rich type information present in modern Python code. This enhancement resolves a long-standing issue and improves the parser's fidelity for static analysis and code understanding. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new, comprehensive test suite for Python type annotations in test_type_annotations.py, covering basic types, container types, special types, generics, and newer Python syntax (3.9+, 3.10+). Concurrently, uast/visitor.py was updated to include a new private method, _parse_type_annotation, which centralizes the parsing logic for various AST type annotation nodes into UAST type nodes. This new method is then integrated into visit_FunctionDef, visit_arguments, and visit_AnnAssign to uniformly handle return types, parameter types, and variable annotations, replacing previous ad-hoc parsing. Review comments identified an issue with ArrayType construction in the new parsing logic, suggested adding a test case for Callable with no arguments, and recommended a stylistic improvement for type name checks.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR is being reviewed by Cursor Bugbot
Details
Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
parser-Python/uast/visitor.py
Outdated
| # Callable[..., bool] → Params 包装器, typeArguments=None（None，表示任意参数） | ||
| # Callable[[int, str], bool] → Params 包装器, typeArguments=[int, str]（具体参数类型） | ||
| param_types_list = self.packPos(param_list_node, UNode.ArrayType(UNode.SourceLocation(), | ||
| UNode.Meta(), element=param_types if param_types else [])) |
There was a problem hiding this comment.
ArrayType element receives list instead of single type
High Severity
When handling Callable parameter types, the code creates an ArrayType with element=param_types, but param_types is a list of types (accumulated via append in the loop). According to ArrayType in asttype.py, the element field expects a single Type, not a list. This semantic mismatch will cause the UAST to have an incorrect structure for Callable parameter representations, potentially breaking downstream consumers that expect ArrayType.element to be a single type value.
parser-Python/uast/visitor.py
Outdated
| elif isinstance(param_list_node, ast.Constant) and param_list_node.value == ...: | ||
| # Callable[..., ReturnType] 表示任意参数 | ||
| param_types_list = self.packPos(param_list_node, UNode.DynamicType(UNode.SourceLocation(), | ||
| UNode.Meta(), id="Params", typeArguments="Ellipsis")) # Any 表示任意参数 |
There was a problem hiding this comment.
DynamicType typeArguments receives string instead of list
High Severity
When handling Callable[..., ReturnType] (ellipsis parameters), the code passes typeArguments="Ellipsis" as a string literal to DynamicType. However, the typeArguments field in DynamicType is defined as Optional[List['Type']], not a string. This type mismatch will produce invalid UAST nodes that don't conform to the expected schema and may cause errors in downstream processing.
2 participants
Description
This PR adds comprehensive support for complex type annotations in the YASA-UAST Python parser, addressing the issue described in #39.
Changes
1. Extended Type Annotation Parsing
Added a comprehensive
_parse_type_annotationmethod that supports:int,float,str,bool,NoneList[T]→ArrayType(element=T)Dict[K, V]→MapType(keyType=K, valueType=V)Tuple[T1, T2, ...]→DynamicType(id="Tuple", typeArguments=[T1, T2, ...])Set[T]→DynamicType(id="Set", typeArguments=[T])Optional[T]→DynamicType(id="Optional", typeArguments=[T])Union[T1, T2, ...]→DynamicType(id="Union", typeArguments=[T1, T2, ...])Literal[value, ...]→DynamicType(id="Literal", typeArguments=[...])Any→DynamicType()Callable[[Args], ReturnType]with proper separation of parameter types and return typelist[T],dict[K, V], etc.)2. Updated Function and Variable Annotation Handling
visit_FunctionDefto use_parse_type_annotationfor return typesvisit_AnnAssignto use_parse_type_annotationfor variable annotationsast.Constant(value=None)for None type annotations (Python 3.8+)3. Test Cases
Added comprehensive test cases in
parser-Python/test/case.pycovering:Implementation Details
Type Mapping
int,floatPrimitiveType(kind="number")strPrimitiveType(kind="string")boolPrimitiveType(kind="boolean")NonePrimitiveType(kind="null")List[T]ArrayType(element=T)Dict[K, V]MapType(keyType=K, valueType=V)Tuple[T1, T2, ...]DynamicType(id="Tuple", typeArguments=[T1, T2, ...])Optional[T]DynamicType(id="Optional", typeArguments=[T])Union[T1, T2, ...]DynamicType(id="Union", typeArguments=[T1, T2, ...])Literal[value, ...]DynamicType(id="Literal", typeArguments=[Literal(...)])Callable[[Args], Ret]DynamicType(id="Callable", typeArguments=[Params, Ret])MyClass[T]DynamicType(id="MyClass", typeArguments=[T])Testing
int,str,bool,None)List[int],Dict[str, int])Optional[str],Union[int, str],Literal["a", "b"])List[Dict[str, int]])Optional[List[Dict[str, Union[int, str]]]])Callable[[int, str], bool])MyClass[str])Examples
Before (Issue)
Result:
return_type=None❌After (This PR)
Result:
return_type=ArrayType(element=PrimitiveType(kind="number"))✅Note
Adds full-featured type-annotation parsing to the Python UAST transformer and applies it across return/parameter/variable annotations.
_parse_type_annotationsupportingint/float/str/bool/None,list/dict/tuple/set,Optional,Union(incl.|),Literal,Any,Callable, and generic/user-defined types; maps to UASTPrimitiveType,ArrayType,MapType, andDynamicTypewithtypeArgumentsvisit_FunctionDef,visit_arguments, andvisit_AnnAssignto use unified parsing for return types and parameter/variable annotationsparser-Python/test/test_type_annotations.pycovering basic, container, special, callable, nested, and generic scenariosMapTypeinuast/asttype.Nodeunion; update.gitignorewith.vscode/Written by Cursor Bugbot for commit c219c8b. This will update automatically on new commits. Configure here.