feat(analyze): add --save-analysis flag to save computed results as JSON

[yoavkatz]

Summary

This PR bundles two independent sets of changes:

1. --save-analysis flag — write the computed analysis (per-trace records + group summaries) to a JSON file alongside the existing text report.
2. Minor fixes — Prometheus port-forward readiness check in evaluate-benchmark.sh and a temp-file cleanup fix in libsh/keycloak-direct-access.sh.

---

Changes

analyze_traces.py

• New functions: record_to_dict, build_group_summaries, build_analysis_json
  • build_group_summaries computes all aggregated stats per group (timing avg/p50/p95/min/max, token counts, infra metrics, per-call latency averages, % time breakdowns) using the existing avg/percentile/std helpers.
  • build_analysis_json returns {"traces": [...], "groups": [...]} — every parsed TraceRecord plus all group summaries.
• New CLI flag: --json PATH — writes the analysis JSON to PATH; text report still prints to stdout unchanged.
• Refactored print_report: now reads from build_group_summaries instead of recomputing stats from raw records. The text output and JSON output share a single computation, so numbers are guaranteed to match.

analyze-run.sh

• Added --save-analysis FILE flag and SAVE_ANALYSIS_FILE env var.
• PYTHON_ARGS converted from a string to a bash array (safe handling of paths with spaces).
• --json FILE forwarded to analyze_traces.py in both pipeline paths (with and without --save-traces).
• Banner and ✓ Saved analysis to FILE confirmation added.

run-ibac-comparison.sh

• Added --save-analysis FILE flag, forwarded to the analyze-run.sh comparison call at the end of the run.

evaluate-benchmark.sh

• Added a readiness check for the Prometheus port-forward: wait 3 s, confirm the process is still alive, print a clear error (with kubectl log tail) if it died, and emit a ✓ confirmation on success — matching the existing pattern for the OTEL collector port-forward.

libsh/keycloak-direct-access.sh

• Replace the hardcoded /tmp/kc_put_response.txt scratch file with a mktemp-generated temp file and a RETURN trap that removes it. Avoids collisions under concurrent invocations and ensures cleanup on error paths.

---

Usage

# save analysis from a live cluster run
./analyze-run.sh --kind --window 3h --save-analysis ./analysis.json

# save analysis from a full ibac comparison
./run-ibac-comparison.sh --save-analysis ./analysis.json

# use analyze_traces.py directly on a saved traces file
python3 analyze_traces.py --json ./analysis.json traces.json

Test plan

• echo '{"traces":[]}' | python3 analyze_traces.py --json /tmp/out.json → file contains {"traces": [], "groups": []}, text prints "No Agent.Session traces found."
• Run against a saved raw traces file: python3 analyze_traces.py --json /tmp/out.json traces-*.json → groups and traces populated, numbers match the printed tables.
• ./analyze-run.sh --kind --window 3h --save-analysis ./analysis.json → text report prints, ✓ Saved analysis to ./analysis.json appears, jq . analysis.json is valid.
• ./run-ibac-comparison.sh --dry --save-analysis ./analysis.json → dry-run output shows --save-analysis forwarded to analyze-run.sh.
• Prometheus port-forward failure in evaluate-benchmark.sh produces a clear error rather than silently continuing.
• Concurrent enable_direct_access_grants calls no longer collide on /tmp/kc_put_response.txt.

🤖 Generated with Claude Code