Add code-coverage-setup skill - #28
Draft
roydahan wants to merge 1 commit into
Draft
Conversation
Distills a hands-on session that added code coverage measurement (unit + integration tests, combined into one report) to four ScyllaDB driver repositories -- Python, Go, Rust, and Java -- across two rounds of maintainer code review. Not a ScyllaDB-product skill like the others in this repo: it's a general software-engineering skill (how to add/fix coverage tooling in any repository) that happened to come out of ScyllaDB driver work. Opening as a draft to discuss whether/how it fits this repo's scope before merging. The content leads with empirically-verified gotchas rather than generic tool docs, since that's what actually made the difference in practice: a stale compiled extension shadowing instrumented Python source, a test runner's default fail-fast behavior silently discarding coverage data for every test after the first failure (three different disguises, in Go/nextest/Maven), a Maven argLine override silently clobbering JaCoCo's injected agent flag, and a coverage counter-mode clash when merging runs recorded under different modes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a new skill,
code-coverage-setup, distilling a hands-on session that added code coverage measurement (unit + integration tests, combined into one report) to four ScyllaDB driver repositories -- Python (python-driver#967), Go (gocql#989), Rust (scylla-rust-driver#1844), and Java (java-driver#1005) -- across two rounds of real maintainer code review.Opening as draft -- scope question
This isn't a ScyllaDB-product skill like the others here (Cloud setup, data modeling, Vector Search, Alternator, Kubernetes) -- it's a general software-engineering skill (how to add or fix coverage tooling in any repository, in any language) that happened to come out of ScyllaDB driver work. The README currently describes this repo as ScyllaDB-specific ("deep knowledge of ScyllaDB Cloud, ScyllaDB CQL data modeling, and ScyllaDB Vector Search"), so I want to flag that before this merges rather than assume it fits. Options as I see them:
Why this content, and why it leads with bugs instead of tool docs
Every high-value finding in the four driver PRs came from actually running the coverage tool and checking the output artifact had real, non-zero content -- not from reading a log line and assuming success.
SKILL.mdleads with that principle, and each language reference documents the specific bugs found and fixed, since a generic "how coverage.py/JaCoCo/cargo-llvm-cov works" writeup wouldn't have caught any of them:.so) silently shadowing the instrumented pure-Python source on import, producing a 0% report with no error.go test's flag parser silently defaulting the package pattern to.the moment it hits a custom flag it doesn't recognize -- meaning coverage flags placed afterward (or a package meant to be tested) are silently ignored, not errored.jacoco:prepare-agentsetting theargLineMaven property correctly, only for a module's own Surefire/Failsafe config to silently overwrite it with just its own JVM flags -- coverage was never actually being collected for that module, despite a log line saying it was.Validation
passes (
skill-validator check --strict), after fixing three "potentially unreferenced file" warnings by linking each reference file explicitly from the table inSKILL.md.