-
Notifications
You must be signed in to change notification settings - Fork 15
[Rust] Scaffold uast4rust CLI with minimal valid output #56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
KinomotoMio
wants to merge
41
commits into
antgroup:main
Choose a base branch
from
KinomotoMio:feat/rust-parser
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 7 commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
a321524
feat(rust-parser): scaffold uast4rust cli and minimal output model
KinomotoMio 7c0f0e8
test(rust-parser): add smoke tests with synthetic fixtures
KinomotoMio 57ebf9b
docs(rust-parser): add uast4rust usage README
KinomotoMio d42f103
fix(rust-parser): include flag name in missing-value arg errors
KinomotoMio bc17bb8
refactor(rust-parser): replace stringly errors with typed RunError
KinomotoMio 90ebe0a
perf(rust-parser): avoid full file read when checking readability
KinomotoMio 34143e3
refactor(rust-parser): simplify main flow and use code 0 for help
KinomotoMio f0956bb
refactor(rust-parser): switch cli arg parsing to clap with legacy fla…
KinomotoMio 90900af
Revert "refactor(rust-parser): switch cli arg parsing to clap with le…
KinomotoMio 8eccdf0
fix(rust-parser): prevent source overwrite when output path equals input
KinomotoMio 51282bc
fix(rust-parser): reject known flags as option values
KinomotoMio 365bc5c
feat(rust-parser): discover Cargo manifests and build project metadat…
KinomotoMio 1d13600
test(rust-parser): add project discovery golden and stability checks
KinomotoMio d01405b
fix(rust-parser): skip hidden dirs and parse quoted manifest names sa…
KinomotoMio 07bc1af
refactor(rust-parser): parse Cargo.toml package name via toml crate
KinomotoMio 8879b61
refactor(rust-parser): make manifest path assumption explicit with ex…
KinomotoMio 851df41
refactor(rust-parser): simplify package name extraction chain
KinomotoMio 2adf42d
feat(rust-parser): lower core single-file rust syntax into uast
KinomotoMio 4f42be8
test(rust-parser): add core syntax single-file golden coverage
KinomotoMio 844895c
fix(rust-parser): avoid placeholder fallback for core syntax lowering
KinomotoMio 87fed63
fix(rust-parser): return Option for lowered function args
KinomotoMio de30fc3
fix(rust-parser): filter out unsupported function parameters
KinomotoMio 3c64718
refactor(rust-parser): assert named struct field identifiers
KinomotoMio 11d3fb9
feat(rust-parser): lower common control-flow constructs
KinomotoMio cc92922
test(rust-parser): add control-flow golden coverage
KinomotoMio 58b7120
Fix rust range/match lowering and add regressions
KinomotoMio 9fb9f20
Fix guarded match lowering to if/else chain
KinomotoMio 003b781
Add PR checks workflow for Rust parser
KinomotoMio a737903
Add golden regression fixtures for Rust parser
KinomotoMio 81da544
Document Rust parser usage and scope
KinomotoMio 8ee93e8
Merge pull request #5 from KinomotoMio/feat/rust-parser-s5
KinomotoMio 97187e5
Ensure guarded match if-branches use scoped statements
KinomotoMio 11f711d
Remove dead empty-check in match case lowering
KinomotoMio 603f9fb
Refresh regression golden for scoped guarded-match output
KinomotoMio 6895d82
Merge pull request #4 from KinomotoMio/feat/rust-parser-s4
KinomotoMio 948172a
Skip wildcard names in tuple destructuring locals
KinomotoMio 98552ee
Merge pull request #2 from KinomotoMio/feat/rust-parser-s3
KinomotoMio 2e07791
Skip wildcard single-pattern locals in lowering
KinomotoMio e019400
Lower tuple destructuring locals from non-literal RHS
KinomotoMio 1af8b65
Format Rust parser updates with rustfmt
KinomotoMio 1d9eb41
Merge pull request #1 from KinomotoMio/feat/rust-parser-s2
KinomotoMio File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| [package] | ||
| name = "uast4rust" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
|
|
||
| [dependencies] | ||
| serde = { version = "1.0", features = ["derive"] } | ||
| serde_json = "1.0" | ||
|
|
||
| [dev-dependencies] | ||
| tempfile = "3.10" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # uast4rust | ||
|
|
||
| `uast4rust` is the Rust parser binary for YASA-UAST. | ||
|
|
||
| ## Build | ||
|
|
||
| ```bash | ||
| cargo build --release | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| Single-file mode: | ||
|
|
||
| ```bash | ||
| ./target/release/uast4rust -rootDir /path/to/file.rs -output /path/to/output.json -single | ||
| ``` | ||
|
|
||
| Project mode: | ||
|
|
||
| ```bash | ||
| ./target/release/uast4rust -rootDir /path/to/project -output /path/to/output.json | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,234 @@ | ||
| mod model; | ||
|
|
||
| pub use model::{CompileUnit, NodeInfo, Output, PackagePathInfo, LANGUAGE}; | ||
| use std::collections::BTreeMap; | ||
| use std::env; | ||
| use std::fmt; | ||
| use std::fs; | ||
| use std::path::{Path, PathBuf}; | ||
|
|
||
| const SINGLE_FILE_PACKAGE_NAME: &str = "__single__"; | ||
| const SINGLE_FILE_MODULE_NAME: &str = "__single_module__"; | ||
| const UNKNOWN_MODULE_NAME: &str = "__unknown_module__"; | ||
|
|
||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| pub struct CliArgs { | ||
| pub root_dir: PathBuf, | ||
| pub output: PathBuf, | ||
| pub single: bool, | ||
| } | ||
|
|
||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| pub enum ParseArgsError { | ||
| HelpRequested, | ||
| Message(String), | ||
| } | ||
|
|
||
| impl fmt::Display for ParseArgsError { | ||
| fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { | ||
| match self { | ||
| Self::HelpRequested => write!(f, "{}", usage()), | ||
| Self::Message(msg) => write!(f, "{msg}"), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| #[derive(Debug)] | ||
| pub enum RunError { | ||
| InvalidInputPath { mode: &'static str, path: PathBuf }, | ||
| Io { | ||
| context: String, | ||
| source: std::io::Error, | ||
| }, | ||
| Serialize { source: serde_json::Error }, | ||
| } | ||
|
|
||
| impl fmt::Display for RunError { | ||
| fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { | ||
| match self { | ||
| Self::InvalidInputPath { mode, path } => { | ||
| write!(f, "{} mode expects a valid path: {}", mode, path.display()) | ||
| } | ||
| Self::Io { context, source } => write!(f, "{context}: {source}"), | ||
| Self::Serialize { source } => write!(f, "failed to serialize output json: {source}"), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl std::error::Error for RunError { | ||
| fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { | ||
| match self { | ||
| Self::InvalidInputPath { .. } => None, | ||
| Self::Io { source, .. } => Some(source), | ||
| Self::Serialize { source } => Some(source), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| pub fn usage() -> &'static str { | ||
| "Usage: uast4rust -rootDir <path> -output <path> [-single]" | ||
| } | ||
|
|
||
| pub fn parse_args_from<I, S>(args: I) -> Result<CliArgs, ParseArgsError> | ||
| where | ||
| I: IntoIterator<Item = S>, | ||
| S: Into<String>, | ||
| { | ||
| let mut root_dir: Option<PathBuf> = None; | ||
| let mut output: Option<PathBuf> = None; | ||
| let mut single = false; | ||
| let mut iter = args.into_iter().map(Into::into); | ||
|
|
||
| while let Some(arg) = iter.next() { | ||
| match arg.as_str() { | ||
| "-rootDir" | "--rootDir" => { | ||
| let value = iter.next().ok_or_else(|| { | ||
| ParseArgsError::Message(format!("Missing value for '{}'", arg)) | ||
| })?; | ||
| root_dir = Some(PathBuf::from(value)); | ||
| } | ||
| "-output" | "--output" => { | ||
| let value = iter.next().ok_or_else(|| { | ||
| ParseArgsError::Message(format!("Missing value for '{}'", arg)) | ||
| })?; | ||
| output = Some(PathBuf::from(value)); | ||
| } | ||
| "-single" | "--single" => { | ||
| single = true; | ||
| } | ||
| "-h" | "--help" => { | ||
| return Err(ParseArgsError::HelpRequested); | ||
| } | ||
| unknown => { | ||
| return Err(ParseArgsError::Message(format!( | ||
| "Unknown argument: {unknown}" | ||
| ))); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| let root_dir = root_dir.ok_or_else(|| { | ||
| ParseArgsError::Message(format!("Missing required argument: -rootDir\n{}", usage())) | ||
| })?; | ||
| let output = output.ok_or_else(|| { | ||
| ParseArgsError::Message(format!("Missing required argument: -output\n{}", usage())) | ||
| })?; | ||
|
|
||
| Ok(CliArgs { | ||
| root_dir, | ||
| output, | ||
| single, | ||
| }) | ||
| } | ||
|
KinomotoMio marked this conversation as resolved.
|
||
|
|
||
| pub fn parse_args() -> Result<CliArgs, ParseArgsError> { | ||
| parse_args_from(env::args().skip(1)) | ||
| } | ||
|
|
||
| pub fn run(cli: &CliArgs) -> Result<(), RunError> { | ||
| if cli.single { | ||
| parse_single_file(&cli.root_dir, &cli.output) | ||
| } else { | ||
| parse_project(&cli.root_dir, &cli.output) | ||
| } | ||
| } | ||
|
|
||
| fn parse_single_file(file: &Path, output: &Path) -> Result<(), RunError> { | ||
| if !file.is_file() { | ||
| return Err(RunError::InvalidInputPath { | ||
| mode: "single", | ||
| path: file.to_path_buf(), | ||
| }); | ||
| } | ||
|
|
||
| fs::File::open(file).map_err(|source| RunError::Io { | ||
| context: format!("failed to read file {}", file.display()), | ||
| source, | ||
| })?; | ||
|
|
||
| let file_key = file.to_string_lossy().to_string(); | ||
| let mut files = BTreeMap::new(); | ||
| files.insert( | ||
| file_key.clone(), | ||
| NodeInfo { | ||
| node: CompileUnit::empty(file_key), | ||
| package_name: SINGLE_FILE_PACKAGE_NAME.to_string(), | ||
| }, | ||
| ); | ||
|
|
||
| let output_model = Output { | ||
| package_info: PackagePathInfo { | ||
| path_name: "/".to_string(), | ||
| files, | ||
| subs: BTreeMap::new(), | ||
| }, | ||
| module_name: SINGLE_FILE_MODULE_NAME.to_string(), | ||
| cargo_toml_path: String::new(), | ||
| num_of_cargo_toml: 0, | ||
| }; | ||
|
|
||
| write_output(output, &output_model) | ||
| } | ||
|
cursor[bot] marked this conversation as resolved.
|
||
|
|
||
| fn parse_project(root_dir: &Path, output: &Path) -> Result<(), RunError> { | ||
| if !root_dir.is_dir() { | ||
| return Err(RunError::InvalidInputPath { | ||
| mode: "project", | ||
| path: root_dir.to_path_buf(), | ||
| }); | ||
| } | ||
|
|
||
| let output_model = Output { | ||
| package_info: PackagePathInfo::empty_root(), | ||
| module_name: UNKNOWN_MODULE_NAME.to_string(), | ||
| cargo_toml_path: String::new(), | ||
| num_of_cargo_toml: 0, | ||
| }; | ||
|
|
||
| write_output(output, &output_model) | ||
| } | ||
|
|
||
| fn write_output(output_path: &Path, output: &Output) -> Result<(), RunError> { | ||
| if let Some(parent) = output_path.parent() { | ||
| if !parent.as_os_str().is_empty() { | ||
| fs::create_dir_all(parent).map_err(|source| RunError::Io { | ||
| context: format!("failed to create output directory {}", parent.display()), | ||
| source, | ||
| })?; | ||
| } | ||
| } | ||
|
|
||
| let json = serde_json::to_vec(output).map_err(|source| RunError::Serialize { source })?; | ||
| fs::write(output_path, json).map_err(|source| RunError::Io { | ||
| context: format!("failed to write output {}", output_path.display()), | ||
| source, | ||
| }) | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| #[test] | ||
| fn parse_args_accepts_expected_flags() { | ||
| let args = vec![ | ||
| "-rootDir", | ||
| "/tmp/example.rs", | ||
| "-output", | ||
| "/tmp/output.json", | ||
| "-single", | ||
| ]; | ||
| let cli = parse_args_from(args).expect("parse args"); | ||
| assert_eq!(cli.root_dir, PathBuf::from("/tmp/example.rs")); | ||
| assert_eq!(cli.output, PathBuf::from("/tmp/output.json")); | ||
| assert!(cli.single); | ||
| } | ||
|
|
||
| #[test] | ||
| fn parse_args_rejects_missing_required_flags() { | ||
| let args = vec!["-rootDir", "/tmp/example.rs"]; | ||
| let err = parse_args_from(args).expect_err("missing output should fail"); | ||
| assert!(matches!(err, ParseArgsError::Message(_))); | ||
| assert!(err.to_string().contains("-output")); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| use std::process; | ||
|
|
||
| fn main() { | ||
| let cli = match uast4rust::parse_args() { | ||
| Ok(cli) => cli, | ||
| Err(uast4rust::ParseArgsError::HelpRequested) => { | ||
| println!("{}", uast4rust::usage()); | ||
| process::exit(0); | ||
| } | ||
| Err(err) => { | ||
| eprintln!("{err}"); | ||
| process::exit(2); | ||
| } | ||
| }; | ||
|
|
||
| if let Err(err) = uast4rust::run(&cli) { | ||
| eprintln!("{err}"); | ||
| process::exit(1); | ||
| } | ||
| } | ||
|
KinomotoMio marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| use serde::Serialize; | ||
| use serde_json::Value; | ||
| use std::collections::BTreeMap; | ||
|
|
||
| pub const LANGUAGE: &str = "rust"; | ||
| pub const LANGUAGE_VERSION: &str = ""; | ||
|
|
||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct Output { | ||
| pub package_info: PackagePathInfo, | ||
| pub module_name: String, | ||
| pub cargo_toml_path: String, | ||
| pub num_of_cargo_toml: usize, | ||
| } | ||
|
|
||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct PackagePathInfo { | ||
| pub path_name: String, | ||
| pub files: BTreeMap<String, NodeInfo>, | ||
| pub subs: BTreeMap<String, PackagePathInfo>, | ||
| } | ||
|
|
||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct NodeInfo { | ||
| pub node: CompileUnit, | ||
| pub package_name: String, | ||
| } | ||
|
|
||
| #[derive(Debug, Serialize)] | ||
| #[serde(rename_all = "camelCase")] | ||
| pub struct CompileUnit { | ||
| #[serde(rename = "type")] | ||
| pub node_type: String, | ||
| pub body: Vec<Value>, | ||
| pub language: String, | ||
| pub language_version: String, | ||
| pub uri: String, | ||
| pub version: String, | ||
| } | ||
|
|
||
| impl CompileUnit { | ||
| pub fn empty(uri: String) -> Self { | ||
| Self { | ||
| node_type: "CompileUnit".to_string(), | ||
| body: Vec::new(), | ||
| language: LANGUAGE.to_string(), | ||
| language_version: LANGUAGE_VERSION.to_string(), | ||
| uri, | ||
| version: String::new(), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl PackagePathInfo { | ||
| pub fn empty_root() -> Self { | ||
| Self { | ||
| path_name: "/".to_string(), | ||
| files: BTreeMap::new(), | ||
| subs: BTreeMap::new(), | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| [package] | ||
| name = "demo" | ||
| version = "0.1.0" | ||
| edition = "2021" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| fn main() { | ||
| println!("project fixture"); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| fn main() { | ||
| println!("hello, yasa"); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.