[KYUUBI #7738][AUTHZ] Keep PermanentViewMarker.newInstance() canonically equal to the original - #7739
Open
j1wonpark wants to merge 1 commit into
Open
[KYUUBI #7738][AUTHZ] Keep PermanentViewMarker.newInstance() canonically equal to the original#7739j1wonpark wants to merge 1 commit into
j1wonpark wants to merge 1 commit into
Conversation
…nonically equal to the original newInstance() wraps the child in a Project that survives canonicalization, so a renewed marker is not sameResult with the original and a cached permanent view is read again. Drop the Cast that blocks Spark's rename-only Project stripping, and match on the tag alone so the guard covers any depth and any child. Assisted-by: Claude Opus 5 Signed-off-by: Jiwon Park <jiwonpark@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Fixes #7738.
newInstance()wraps the child in aProject, and three things kept thatProjectin thecanonical form, so a renewed marker was not
sameResultwith the original andCacheManagercould not find the cache.
Dropping the
Castlets Spark's own rename-onlyProjectstripping apply. That strippingarrived in SPARK-51109, so it is absent on 3.5; on 4.0 the backported condition requires
alias.explicitMetadata.isEmptyand PVM passesSome(attr.metadata), so it does not applythere either. The guard is the only mechanism on those two versions, hence both changes.
Matching on the tag alone drops two conditions the guard does not need — that the child is a
View, and that there is a single layer. Both are violated in practice: markers over subqueryplans have a non-
Viewchild, and a renewed marker reused in another query gains a secondProject.How was this patch tested?
Three unit assertions in a new
PermanentViewMarkerSuite, one per cause, and a plan levelcheck next to
[KYUUBI #3411] skip checking cache tableinRangerSparkExtensionSuitethatalso covers the nested shape.
Castremoved only70 tests pass on 3.5 and on 4.1 (
PermanentViewMarkerSuiteplus the Hive and in-memory catalogRangerSparkExtensionSuite), and the masking and row filtering suites are unchanged.Was this patch assisted by generative AI tooling?
Assisted-by: Claude Opus 5