fix(api): handle gRPC unavailable and deadline errors in volume orchestrator retry loop

[chill-czar]

Closes #3577

Summary

• Update isRetryableError in packages/api/internal/handlers/volume_util.go to extract status.FromError(err) and return true for codes.Unavailable, codes.DeadlineExceeded, and codes.Canceled.
• Ensure executeOnOrchestratorByClusterID properly retries subsequent candidate nodes in the cluster when an orchestrator node is down, unreachable, or timing out.
• Add comprehensive table-driven unit tests in packages/api/internal/handlers/volume_util_test.go covering standard errors and gRPC status codes.

Why

In packages/api/internal/handlers/volume_util.go, when a volume creation or deletion RPC was attempted on a node that was rebooting or unavailable, gRPC returned status.Error(codes.Unavailable, ...). Because isRetryableError only checked errors.Is(err, net.ErrClosed), gRPC status errors evaluated to false, causing the loop to abort on the first node rather than failing over to other healthy nodes in the cluster pool.

Diff Overview

 func isRetryableError(err error) bool {
-	if errors.Is(err, net.ErrClosed) || errors.Is(err, context.DeadlineExceeded) {
+	if errors.Is(err, net.ErrClosed) || errors.Is(err, context.DeadlineExceeded) || errors.Is(err, context.Canceled) {
 		return true
 	}
 
+	st, ok := status.FromError(err)
+	if ok {
+		switch st.Code() {
+		case codes.Unavailable, codes.DeadlineExceeded, codes.Canceled:
+			return true
+		}
+	}
+
 	return false
 }

Test Plan

• Unit test: go test -v ./packages/api/internal/handlers -run TestIsRetryableError (10 test cases verified)
• Package test suite passes: go test -v ./packages/api/internal/handlers/...
• Linter & Formatter pass: make fmt and make lint
• Verified codes.Unavailable, codes.DeadlineExceeded, and codes.Canceled return true while terminal errors return false

/cc @jakubno @dobrac @ValentaTomas @arkamar @tvi