Skip to content

[Rust] Add core single-file syntax lowering to UAST#58

Closed
KinomotoMio wants to merge 36 commits into
antgroup:mainfrom
KinomotoMio:feat/rust-parser-s3
Closed

[Rust] Add core single-file syntax lowering to UAST#58
KinomotoMio wants to merge 36 commits into
antgroup:mainfrom
KinomotoMio:feat/rust-parser-s3

Conversation

@KinomotoMio

@KinomotoMio KinomotoMio commented Feb 27, 2026

Copy link
Copy Markdown

Summary

Implement core single-file Rust syntax lowering in parser-Rust and add focused regression tests.

Scope in this PR

  • Lower core single-file syntax to UAST for:
    • fn
    • struct
    • let
    • assignment (including compound assignment operators)
    • call
    • member access
    • return
  • Remove placeholder fallback identifiers for core syntax paths and keep lowering deterministic
  • Add single-file synthetic fixtures and golden/regression tests for core lowering

Out of scope

  • Control-flow lowering (if/match/for/while/loop)
  • Engine integration / taint wiring

Validation

  • cargo fmt
  • cargo test (all pass in parser-Rust)

Related issues

Stacking note

This branch is stacked on top of #57.
Mirror stacked review PR (base = feat/rust-parser-s2) is available at:

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces the foundational Rust parser (uast4rust) for YASA-UAST, enabling the conversion of single-file Rust code into a Universal Abstract Syntax Tree. The primary focus is on lowering core language constructs such as function and struct definitions, variable declarations, assignments, function calls, and member access. It also establishes a robust testing framework with synthetic fixtures and golden files to ensure the accuracy and stability of the parsing process.

Highlights

  • Core Single-File Syntax Lowering: Implemented the lowering of core single-file Rust syntax elements to UAST, including functions, structs, variable declarations (let), assignments (including compound operators), function calls, member access, and return statements.
  • Deterministic Lowering: Removed placeholder fallback identifiers for core syntax paths, ensuring that the UAST lowering process is deterministic and consistent.
  • Comprehensive Testing: Added single-file synthetic fixtures and golden/regression tests to thoroughly validate the newly implemented core lowering functionality.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • parser-Rust/Cargo.lock
    • Added a new dependency lock file for the Rust parser.
  • parser-Rust/Cargo.toml
    • Added the manifest for the uast4rust Rust parser, defining its dependencies.
  • parser-Rust/README.md
    • Added documentation for building and using the uast4rust parser in single-file and project modes.
  • parser-Rust/src/lib.rs
    • Implemented the core logic for parsing Rust files into UAST, including CLI argument handling, file I/O, and AST lowering for functions, structs, variables, expressions, and statements.
  • parser-Rust/src/main.rs
    • Added the main executable entry point for the uast4rust parser, handling command-line arguments and error reporting.
  • parser-Rust/src/model.rs
    • Defined the data structures for the Universal Abstract Syntax Tree (UAST) representation specific to Rust.
  • parser-Rust/testdata/project/basic/Cargo.toml
    • Added a basic Cargo.toml file for project mode testing.
  • parser-Rust/testdata/project/basic/src/main.rs
    • Added a simple main.rs file for basic project mode testing.
  • parser-Rust/testdata/project/multi/Cargo.toml
    • Added a workspace Cargo.toml for multi-crate project testing.
  • parser-Rust/testdata/project/multi/crates/a/Cargo.toml
    • Added a Cargo.toml for crate 'a' within the multi-crate project test setup.
  • parser-Rust/testdata/project/multi/crates/a/src/main.rs
    • Added a main.rs for crate 'a' within the multi-crate project test setup.
  • parser-Rust/testdata/project/multi/crates/b/Cargo.toml
    • Added a Cargo.toml for crate 'b' within the multi-crate project test setup.
  • parser-Rust/testdata/project/multi/crates/b/src/lib.rs
    • Added a lib.rs for crate 'b' within the multi-crate project test setup.
  • parser-Rust/testdata/project/multi/expected.project.json
    • Added the golden UAST output for the multi-crate project test.
  • parser-Rust/testdata/single/basic.rs
    • Added a basic Rust source file for single-file parsing tests.
  • parser-Rust/testdata/single/core_syntax.rs
    • Added a Rust source file demonstrating core syntax elements for single-file parsing tests.
  • parser-Rust/testdata/single/expected.core.json
    • Added the golden UAST output for the core syntax single-file test.
  • parser-Rust/tests/cli_smoke.rs
    • Added integration tests to verify the command-line interface, single-file and project parsing, and output correctness against golden files.
Activity
  • The author validated changes using cargo fmt and cargo test, with all tests passing in the parser-Rust module.
  • This branch is built on top of #57, indicating a sequential development approach.
  • Reviewers are requested to focus on commits specific to this PR branch, excluding those from the base branch.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Rust parser for generating UAST. The implementation covers core Rust syntax for single-file analysis, including functions, structs, let bindings, assignments, and expressions. It also adds project-level analysis by discovering and parsing Cargo.toml files. The code is well-structured and includes a comprehensive set of tests, including golden file testing for stability and correctness.

My review focuses on improving error handling and ensuring correctness in the syntax lowering logic. I've identified a case where let bindings with side effects could be incorrectly dropped, and an opportunity to improve error reporting for malformed Cargo.toml files. Overall, this is a solid foundation for Rust UAST extraction.

Comment thread parser-Rust/src/lib.rs
Comment on lines +407 to +422
fn lower_single_pattern_local(local: &Local) -> Option<Value> {
let (name, explicit_type) = extract_binding_name_and_type(&local.pat)?;
let init = local
.init
.as_ref()
.and_then(|local_init| lower_expr(&local_init.expr));
let var_type = explicit_type.map(lower_type).unwrap_or_else(dynamic_type);
let cloned = init.is_some();
Some(variable_declaration(
identifier(name),
init,
var_type,
cloned,
false,
))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This function currently discards let statements where no variable is bound, such as let Foo { .. } = my_func();. The use of ? on line 408 causes an early return, and the initializer expression (e.g., my_func()) is dropped from the UAST. This is incorrect if the initializer has side effects.

The initializer expression should be preserved in the UAST even if no variables are bound. You can adjust the logic to handle this case, similar to how lower_tuple_pattern_local handles empty declarations.

fn lower_single_pattern_local(local: &Local) -> Option<Value> {
    if let Some((name, explicit_type)) = extract_binding_name_and_type(&local.pat) {
        let init = local
            .init
            .as_ref()
            .and_then(|local_init| lower_expr(&local_init.expr));
        let var_type = explicit_type.map(lower_type).unwrap_or_else(dynamic_type);
        let cloned = init.is_some();
        Some(variable_declaration(
            identifier(name),
            init,
            var_type,
            cloned,
            false,
        ))
    } else {
        // No bindings, but preserve initializer for side effects.
        local
            .init
            .as_ref()
            .and_then(|local_init| lower_expr(&local_init.expr))
    }
}

Comment thread parser-Rust/src/lib.rs

let value = match toml::from_str::<toml::Value>(&content) {
Ok(v) => v,
Err(_) => return Ok(None),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Errors from parsing Cargo.toml files are silently ignored here. This can hide issues with malformed manifests, making it harder for users to debug their project setup. It would be better to propagate a RunError when TOML parsing fails. You could introduce a new variant to the RunError enum for this purpose, similar to how ParseSource is handled.

@KinomotoMio

Copy link
Copy Markdown
Author

For incremental review, I’m currently managing this branch as a stacked PR in my fork (same head branch):
KinomotoMio#2

I will keep this upstream PR aligned as changes are validated.

@KinomotoMio

Copy link
Copy Markdown
Author

Closing in favor of #56.

Rust parser progress is now consolidated on branch feat/rust-parser and tracked in #56.
Planning/tracking is also consolidated in #50.

Note: this is currently exploratory work and does not guarantee mergeability yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Rust] Add core single-file syntax lowering to UAST

1 participant