Loss of steering control (Bolt EV 2022)

[simontheflutist]

Describe the bug

"TAKE CONTROL IMMEDIATELY" while engaged, changing lanes on a freeway. Was able to re-engage right after.

Provide a route where the issue occurs

https://connect.comma.ai/a8150bd87c6ab253/00000005--d2b993a118/1058/1065

openpilot version

nightly-dev

Additional info

No response

[simontheflutist]

panda

clanker analysis

Timeline of Events

All timestamps are in log monotonic time (nanoseconds). I'll use relative seconds from the first event.

1. ~17656630s — preLaneChangeLeft event fires. Openpilot begins planning a left lane change. Alert: "Steer Left / Confirm Lane Change".

2. ~17657630s — laneChange event fires. The lane change is now active. Alert changes to "Changing Lanes". State remains enabled.

3. ~17660940s — steerFaultTemporary becomes true in carState. The vehicle's Power Steering Control Module (PSCM) reports a temporary steering fault. This is the root cause.

4. ~17660950s — Openpilot detects the temporary steer fault and triggers a steerTempUnavailable OnroadEvent with softDisable = true. State transitions from enabled → softDisabling. Alert: "TAKE CONTROL IMMEDIATELY / Steering Assist Temporarily Unavailable". Lateral torque controller output drops to 0 (active = false).

5. ~17660950s – 17661139s (~190 seconds) — System remains in softDisabling state. The steering fault persists. The torque controller is inactive (error=0, output=0). Vehicle speed is ~27 m/s (~60 mph) with near-zero steering angle (~0.3°), suggesting the car was traveling nearly straight.

6. ~17661147s — steerFaultTemporary clears (becomes false). State transitions back to enabled. Lateral controller reactivates.

7. ~17662311s — pedalPressed event with userDisable = true. Driver presses the pedal, causing a user-initiated disengagement. State → disabled.

8. ~17662401s – 17665411s — Multiple pcmDisable and steerOverride events. The driver is manually controlling the vehicle (steering overrides and gas pedal overrides). System stays disabled.

9. ~17666021s — pcmEnable event. Driver re-engages openpilot. State → enabled.

10. ~17667711s — userBookmark event. Driver saves a bookmark (likely marking the incident).

Root Cause

The panda reported a persistent interruptRateCan2 fault and faultStatus = faultTemp throughout the entire log. This indicates CAN bus 2 was experiencing interrupt rate issues (high CAN bus error counts: totalErrorCnt = 54434 on bus 0, canCoreResetCnt = 1 on bus 0, canCoreResetCnt = 13 on bus 1).

During the lane change maneuver, this CAN bus issue caused the GM PSCM (Power Steering Control Module) to report a temporary steering fault (steerFaultTemporary = true). Openpilot correctly responded by issuing a soft disable with a "TAKE CONTROL IMMEDIATELY" alert, relinquishing lateral control for safety.

The steering fault persisted for approximately 190 seconds before clearing, at which point openpilot automatically resumed. The driver then manually disengaged shortly after (pedal press), drove manually for a while, and later re-engaged.

Summary

• What happened: During an active left lane change at ~60 mph, the vehicle's power steering module reported a temporary fault, likely caused by underlying CAN bus 2 interrupt rate issues on the panda. Openpilot soft-disabled lateral control and alerted the driver to take over.
• Root cause: CAN bus 2 interrupt rate fault (interruptRateCan2) on the panda, leading to steerFaultTemporary from the GM PSCM.
• Safety response: Openpilot correctly issued a soft disable with "TAKE CONTROL IMMEDIATELY" alert. No immediate disable or crash occurred. The driver maintained control throughout.

[adeebshihadeh]

#38464 should fix this; your commit was slightly before this.

Also please do not paste a giant LLM wall of text into GitHub issues again.