-
Notifications
You must be signed in to change notification settings - Fork 2
adapter: Fence Redis reads before snapshots #1167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
cbc99dd
0ae2c34
a171f46
a9b26b9
86cd4f0
257d4b5
09a0e49
cf73c96
8496aa7
b36da7a
94fb695
d2951fb
a5a5520
4ed6cae
ae80151
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,6 +38,33 @@ var txnApplyHandlers = map[string]txnCommandHandler{ | |
| cmdPExpire: (*txnContext).applyExpireMilliseconds, | ||
| } | ||
|
|
||
| func (r *RedisServer) verifyQueuedCommandLeaders(ctx context.Context, queue []redcon.Command) error { | ||
| if r.coordinator == nil { | ||
| return nil | ||
| } | ||
| seen := make(map[string]struct{}, len(queue)) | ||
| for _, cmd := range queue { | ||
| if len(cmd.Args) == 0 { | ||
| continue | ||
| } | ||
| meta, ok := redisCommandTable[strings.ToUpper(string(cmd.Args[0]))] | ||
| if !ok { | ||
| continue | ||
| } | ||
| for _, key := range redisCommandGetKeys(meta, cmd.Args) { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a valid transaction contains many distinct keys, such as one variadic Useful? React with 👍 / 👎. |
||
| keyID := string(key) | ||
| if _, ok := seen[keyID]; ok { | ||
| continue | ||
| } | ||
| seen[keyID] = struct{}{} | ||
| if err := r.coordinator.VerifyLeaderForKey(ctx, key); err != nil { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For a legal Redis user key that resembles another adapter's namespace, such as Useful? React with 👍 / 👎. |
||
| return errors.WithStack(err) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In a multi-group deployment with leadership balanced across nodes, Useful? React with 👍 / 👎. |
||
| } | ||
| } | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // MULTI/EXEC/DISCARD handling | ||
| func (r *RedisServer) multi(conn redcon.Conn, _ redcon.Command) { | ||
| state := getConnState(conn) | ||
|
|
@@ -2379,6 +2406,10 @@ func (r *RedisServer) runTransactionDirect(queue []redcon.Command) ([]redisResul | |
|
|
||
| var results []redisResult | ||
| err := r.retryRedisWrite(dispatchCtx, func() error { | ||
| if err := r.verifyQueuedCommandLeaders(dispatchCtx, queue); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| startTS := r.txnStartTS() | ||
| readPin := r.pinReadTS(startTS) | ||
| defer readPin.Release() | ||
|
|
@@ -2593,6 +2624,10 @@ func (r *RedisServer) runTransactionWithDedup(queue []redcon.Command) ([]redisRe | |
| // from runTransactionWithDedup to keep that loop under the cyclop | ||
| // budget; the dedup rationale lives there. | ||
| func (r *RedisServer) firstExecAttempt(dispatchCtx context.Context, queue []redcon.Command) ([]redisResult, *reusableExecTxn, error) { | ||
| if err := r.verifyQueuedCommandLeaders(dispatchCtx, queue); err != nil { | ||
| return nil, nil, err | ||
| } | ||
|
|
||
| startTS := r.txnStartTS() | ||
| readPin := r.pinReadTS(startTS) | ||
| defer readPin.Release() | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the local leader has committed but not yet applied entries,
VerifyLeaderForKeyis not an applied-state fence: it reachesEngine.VerifyLeader, which callssubmitRead(ctx, false), andhandleReadStatescompletes such requests immediately without waiting fore.applied. Consequently this call can succeed while the MVCC store and itsLastCommitTSremain stale, so the immediately selected LRANGE snapshot can still omit a write that completed before the read; the new EXEC pre-pass has the same problem. Use the linearizable/lease-read path that waits for the read index to be applied before selecting the snapshot.Useful? React with 👍 / 👎.