Skip to content

Add code-coverage-setup skill - #28

Draft
roydahan wants to merge 1 commit into
scylladb:mainfrom
roydahan:claude/add-code-coverage-setup-skill
Draft

Add code-coverage-setup skill#28
roydahan wants to merge 1 commit into
scylladb:mainfrom
roydahan:claude/add-code-coverage-setup-skill

Conversation

@roydahan

Copy link
Copy Markdown
Contributor

What

Adds a new skill, code-coverage-setup, distilling a hands-on session that added code coverage measurement (unit + integration tests, combined into one report) to four ScyllaDB driver repositories -- Python (python-driver#967), Go (gocql#989), Rust (scylla-rust-driver#1844), and Java (java-driver#1005) -- across two rounds of real maintainer code review.

skills/code-coverage-setup/
├── SKILL.md                  # universal workflow + empirical-verification principles
└── references/
    ├── python.md              # coverage.py
    ├── go.md                  # go test -cover + covdata
    ├── rust.md                 # cargo-llvm-cov
    └── java.md                 # JaCoCo / jacoco-maven-plugin

Opening as draft -- scope question

This isn't a ScyllaDB-product skill like the others here (Cloud setup, data modeling, Vector Search, Alternator, Kubernetes) -- it's a general software-engineering skill (how to add or fix coverage tooling in any repository, in any language) that happened to come out of ScyllaDB driver work. The README currently describes this repo as ScyllaDB-specific ("deep knowledge of ScyllaDB Cloud, ScyllaDB CQL data modeling, and ScyllaDB Vector Search"), so I want to flag that before this merges rather than assume it fits. Options as I see them:

  • Merge as-is if the repo's scope is meant to be "skills useful to people building on/with ScyllaDB" more broadly (this skill would apply to anyone maintaining a ScyllaDB driver, not just end users).
  • Adjust the README's framing if it's kept.
  • Decide it doesn't belong here and close this instead.

Why this content, and why it leads with bugs instead of tool docs

Every high-value finding in the four driver PRs came from actually running the coverage tool and checking the output artifact had real, non-zero content -- not from reading a log line and assuming success. SKILL.md leads with that principle, and each language reference documents the specific bugs found and fixed, since a generic "how coverage.py/JaCoCo/cargo-llvm-cov works" writeup wouldn't have caught any of them:

  • Python: a stale compiled extension (.so) silently shadowing the instrumented pure-Python source on import, producing a 0% report with no error.
  • Go: go test's flag parser silently defaulting the package pattern to . the moment it hits a custom flag it doesn't recognize -- meaning coverage flags placed afterward (or a package meant to be tested) are silently ignored, not errored.
  • Java: jacoco:prepare-agent setting the argLine Maven property correctly, only for a module's own Surefire/Failsafe config to silently overwrite it with just its own JVM flags -- coverage was never actually being collected for that module, despite a log line saying it was.
  • Recurring across Go, Rust, and Java in three different disguises: a test runner's/build tool's default fail-fast behavior discarding every other test's coverage data after the first failure, not just failing that one test.

Validation

./tests/validate-skills.sh skills/code-coverage-setup/

passes (skill-validator check --strict), after fixing three "potentially unreferenced file" warnings by linking each reference file explicitly from the table in SKILL.md.

Distills a hands-on session that added code coverage measurement (unit +
integration tests, combined into one report) to four ScyllaDB driver
repositories -- Python, Go, Rust, and Java -- across two rounds of
maintainer code review.

Not a ScyllaDB-product skill like the others in this repo: it's a general
software-engineering skill (how to add/fix coverage tooling in any
repository) that happened to come out of ScyllaDB driver work. Opening as
a draft to discuss whether/how it fits this repo's scope before merging.

The content leads with empirically-verified gotchas rather than generic
tool docs, since that's what actually made the difference in practice:
a stale compiled extension shadowing instrumented Python source, a test
runner's default fail-fast behavior silently discarding coverage data
for every test after the first failure (three different disguises, in
Go/nextest/Maven), a Maven argLine override silently clobbering JaCoCo's
injected agent flag, and a coverage counter-mode clash when merging
runs recorded under different modes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant