Skip to content

ui diff: Chunk frame diffs and group in report with thumbnails - #37328

Open
TheSecurityDev wants to merge 156 commits into
commaai:masterfrom
TheSecurityDev:replay-chunks-diff-thumbnails
Open

ui diff: Chunk frame diffs and group in report with thumbnails#37328
TheSecurityDev wants to merge 156 commits into
commaai:masterfrom
TheSecurityDev:replay-chunks-diff-thumbnails

Conversation

@TheSecurityDev

@TheSecurityDev TheSecurityDev commented Feb 23, 2026

Copy link
Copy Markdown
Contributor

Description:

Does a proper diff on the replay video frame hashes using difflib to create groups of changes, and updates the HTML report to support showing these groups. Shows frame changes, additions, and deletions, and allows viewing the exact diff for each one.

It's still possible to see the full clips and diff like before, although we can consider whether the full diff will be needed in the future when we have these chunk diffs now.

Other changes:

The HTML report now supports video placeholders for no video slots (e.g. when frames added/removed) or as a loading indicator.

Also adds multithreading for creating the full diff video in the background while everything else is happening, hashing both video frames at once, and extracting the diff chunk frames in parallel.

Finally, cleans up the script and improves the logging, etc.

Example output:
============================================================
UI VIDEO DIFF REPORT
============================================================
Video 1:      /home/user/openpilot/selfdrive/ui/tests/diff/report/tizi_ui_replay.mp4
Video 2:      /home/user/openpilot/selfdrive/ui/tests/diff/report/tizi_ui_replay_2.mp4
HTML output:  diff.html
Diff video:   diff.mp4
Chunks dir:   diff-chunks

[1/5] Starting full video diff generation in background thread...
[2/5] Hashing frames...
Generating frame hashes for both videos...
  Found 870 frames in video 1 and 814 frames in video 2.
[3/5] Computing diff chunks...
  Found 9 diff chunk(s) with a total of 381 different frames.
[4/5] Extracting 9 diff chunk(s)...
  Getting video fps... 60.00 fps
  Processing 9 chunks with 8 threads...
Processing chunks: 100%|██████████████████████████████████████████████████████████████████| 9/9 [00:11<00:00,  1.30s/it]
[5/5] Generating HTML report...
  Results: ❌ Found 381 different frames out of 870 total (43.8%). Video 1 is longer by 56 frames.
  Report generated at: /home/user/openpilot/selfdrive/ui/tests/diff/report/diff.html
Waiting for diff video generation to finish...

Example video:

Note: This was generated by taking a replay I had lying around from a previous version, and for the second video, modifying the toggle color to blue, adding 2 seconds to the firehose page, and removing the experimental mode description scenario. So the diff is a lot messier than you might get in most realistic situations.

Recording.2026-02-22.190317.mp4

Here's a simpler example with just adding 2 more steps to the mici replay:

image

Identical example:

image

More comments:

  • The diff chunks are stored by default in the diff-chunks folder under report, where diff matches the stem of the output name (e.g. mici-diff.html -> mici-diff-chunks). Thumbnails of the diff/new/removed clips are also extracted and used in the report.
  • -nostdin is used for ffmpeg calls that run in threads because without it my terminal glitches out where I can't see the output as I type.
  • Chunk padding support is left in the code, but I found I like it much better without it since it makes shorter diffs hard to see.

@github-actions

github-actions Bot commented Apr 5, 2026

Copy link
Copy Markdown
Contributor

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

@github-actions github-actions Bot added the stale label Apr 5, 2026
@TheSecurityDev

Copy link
Copy Markdown
Contributor Author

Bump

@github-actions github-actions Bot removed the stale label Apr 6, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

@github-actions github-actions Bot added the stale label Apr 30, 2026
@github-actions

github-actions Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Process replay diff report

Replays driving segments through this PR and compares the behavior to master.
Please review any changes carefully to ensure they are expected.

✅ 0 changed, 66 passed, 0 errors

@github-actions github-actions Bot removed the stale label May 1, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

@github-actions github-actions Bot added the stale label Jun 12, 2026
adeebshihadeh added a commit that referenced this pull request Jun 17, 2026
…37474)

* split out and refactor

* simplify

* rm extra line

* use threadpoolexecutor to handle exceptions

* simplify

* improve

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
@TheSecurityDev

Copy link
Copy Markdown
Contributor Author

I'll fix the merge conflicts

@github-actions github-actions Bot removed the stale label Jun 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has had no activity for 24 days. It will be automatically closed in 7 days if there is no activity.

@github-actions github-actions Bot added the stale label Jul 21, 2026
@TheSecurityDev

TheSecurityDev commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Looks like we need #38514 to fix the errors

@github-actions github-actions Bot removed the stale label Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants