Skip to content

feat: add local Safari testing via safaridriver - #5131

Draft
marcoscaceres wants to merge 3 commits into
mainfrom
chore/safari-launcher
Draft

feat: add local Safari testing via safaridriver#5131
marcoscaceres wants to merge 3 commits into
mainfrom
chore/safari-launcher

Conversation

@marcoscaceres

@marcoscaceres marcoscaceres commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

Adds a minimal custom karma launcher at tests/karma.safari.cjs so pnpm start --browser Safari works locally and Safari can run in CI. It talks to safaridriver over the WebDriver HTTP protocol directly, adding no dependency: the published karma-safari-launcher uses a redirect.html hack that modern Safari treats as a download, and the @onslap replacement needs a native wd@1.x build that does not compile on Node 24+.

Two fixes the Safari run surfaced: core/pubsubhub posts to the parent frame with targetOrigin: "/" rather than falling back to "*" when reading parent.location.origin throws on Safari's opaque srcdoc origin, and the unit test helper awaits the plugin document's own run promise instead of racing it.

Written with AI: this change was generated by Claude. Per AI_POLICY.md.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a custom Karma launcher to enable running the project’s unit tests locally in Safari via safaridriver, replacing the deprecated/broken third-party Safari launcher dependency.

Changes:

  • Introduces tests/karma.safari.cjs, a minimal W3C WebDriver client/launcher that spawns safaridriver and navigates to the Karma URL.
  • Wires the custom launcher into the shared Karma config (tests/karma.conf.base.cjs).
  • Removes karma-safari-launcher from dependencies/lockfile.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File Description
tests/karma.safari.cjs New custom Safari launcher using safaridriver + Node http.
tests/karma.conf.base.cjs Replaces karma-safari-launcher plugin with the local Safari launcher.
package.json Drops karma-safari-launcher dependency.
pnpm-lock.yaml Removes karma-safari-launcher entries.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/karma.safari.cjs Outdated
Comment thread tests/karma.safari.cjs Outdated
Comment thread tests/karma.safari.cjs Outdated
Comment thread tests/karma.safari.cjs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/karma.safari.cjs Outdated
Comment thread tests/karma.safari.cjs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/karma.safari.cjs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/karma.safari.cjs Outdated
Comment thread src/core/markdown.js Outdated
@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot i think we should enable this by default to run alongside Chrome and Firefox, even if it’s more expensive to run on macOS in CI. We care that we don’t regress on WebKit/Safari so it’s worth it.

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

@copilot i think we should enable this by default to run alongside Chrome and Firefox, even if it’s more expensive to run on macOS in CI. We care that we don’t regress on WebKit/Safari so it’s w...

Done in bd0debf — Safari is now enabled in CI on macos-latest runners alongside Chrome and Firefox:

  • pr.yml: extended the test-karma matrix to include Safari / macos-latest (with sudo safaridriver --enable as a conditional step before installing deps).
  • push.yml: added a new test-karma-safari job on macos-latest that runs both test:unit and test:integration with BROWSERS: Safari.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot Can you check and fix ci?

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

@copilot Can you check and fix ci?

Fixed in a7d2784. The CI failure had two root causes on cold macOS runners:

  1. Fixed 500ms startup wait was too short — safaridriver wasn't listening yet, so POST /session got ECONNREFUSED immediately. Replaced with a waitForReady() helper that polls GET /status every 250ms for up to 15s, only proceeding once safaridriver responds.
  2. 10s POST /session timeout was too short — Safari itself takes time to launch on a cold runner. Increased to 30s for that call.

The waitForReady also properly races against the process error/exit events (with a settle-once guard), so early failures are still reported quickly with a clear message.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot Couple of lint errors to fix. See ci.

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

@copilot Couple of lint errors to fix. See ci.

Fixed in dab0d02 — two prettier formatting errors in tests/karma.safari.cjs:

  1. Long reject(new Error(...)) call on line 108 → wrapped to multi-line
  2. Curried arrow const finish = fn => (...args) => { on lines 157–161 → reformatted to prettier's preferred style for curried arrows

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

Seeing a lot ci fails @copilot. Do your thing, my good every so helpful little digital marvel.

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Seeing a lot ci fails @copilot. Do your thing, my good every so helpful little digital marvel.

Fixed in ad409ca — two root causes identified from the Safari CI log:

  1. src/core/pubsubhub.js: window.parent.location.origin throws SecurityError in Safari when srcdoc iframes are treated as opaque-origin. Wrapped in try/catch with "*" as a safe fallback (these messages are test-only, contain no sensitive data).

  2. tests/unit/SpecHelper.js: the waitReady message handler used ev.source.document, which also throws SecurityError cross-origin in Safari, silently preventing the "end-all" message from being handled and leaving tests hanging. Replaced with ev.source !== iframe.contentWindow (safe cross-origin Window identity comparison). Also changed the timeout from throw inside setTimeout (which Safari converts to an unhandled rejection with undefined, crashing jasmine-core) to a proper reject() inside the Promise.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot can you review this again and fix any issues you find and any resolve any unresolved comments?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (2)

pnpm-lock.yaml:129

  • The lockfile no longer lists lint-staged under the root importer’s devDependencies, but package.json still references npx lint-staged via simple-git-hooks and includes a lint-staged config. This will cause pnpm i --frozen-lockfile to omit lint-staged, breaking the configured pre-commit hook. Re-add lint-staged to the root devDependencies (and regenerate the lockfile), or remove the hook/config if lint-staged is intentionally being dropped.
    package.json:56
  • lint-staged was removed from devDependencies, but the repo still configures simple-git-hooks.pre-commit to run npx lint-staged and includes a lint-staged config block later in this same file. With this change, fresh installs won’t have lint-staged available locally, and pre-commit will either fail or unexpectedly download from the network. Either keep lint-staged as a devDependency or remove/update the simple-git-hooks and lint-staged configuration to match.
    "karma-jasmine": "^5.1.0",
    "karma-jasmine-html-reporter": "^2.2.0",
    "karma-mocha-reporter": "^2.2.5",
    "loading-indicator": "^2.0.0",
    "pluralize": "^8.0.0",
    "prettier": "^3.8.2",
    "prompt": "^1.3.0",

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/karma.safari.cjs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)

package.json:56

  • lint-staged was removed from devDependencies, but simple-git-hooks.pre-commit still runs npx lint-staged and the lint-staged config remains in this file. This will make pre-commit either fail (no local binary) or rely on network installs via npx. Re-add lint-staged to devDependencies (preferred) or update/remove the hook/config so it doesn't reference lint-staged.
    "karma": "^6.4.4",
    "karma-chrome-launcher": "^3.2.0",
    "karma-firefox-launcher": "^2.1.3",
    "karma-jasmine": "^5.1.0",
    "karma-jasmine-html-reporter": "^2.2.0",
    "karma-mocha-reporter": "^2.2.5",
    "loading-indicator": "^2.0.0",
    "pluralize": "^8.0.0",
    "prettier": "^3.8.2",
    "prompt": "^1.3.0",

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot Can you confirm all your feedback has been addressed? Can you fix anything that was not addressed?

Copilot AI commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

@copilot Can you confirm all your feedback has been addressed? Can you fix anything that was not addressed?

Done in 039e560. All feedback is now addressed:

  • ev.data guard (SpecHelper.js:142) — already uses ev.data?.topic (optional chaining), guards non-object payloads.
  • postMessage("*") security (pubsubhub.js) — the previous fix went too far: it returned early for all topics when window.parent.location.origin throws, which also dropped the end-all signal and caused all tests to time out. Now only end-all (which publishes detail = undefined) falls back to targetOrigin: "*"; all other topics still skip postMessage when the origin is unavailable. This keeps sensitive config from leaking while restoring test completion in Safari.
  • SAFARIDRIVER_PORT range validation — now uses an IIFE that validates the parsed integer is in 1–65535, throwing a clear error for invalid values (e.g. 0, -1, non-numeric strings).
  • PR description / CI mismatch — already updated to reflect Safari runs in CI.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot can you fix the CI issues?

Copilot AI commented May 14, 2026

Copy link
Copy Markdown
Contributor

@copilot can you fix the CI issues?

Fixed in 9a55b96. The 4 Safari-specific test failures had the same root cause: Safari returns '' for innerText on elements inside closed <details> elements (no layout is computed for hidden content).

  • data-tests-spec.js (3 failures): File/emoji anchor elements live inside <details class="respec-tests-details"> which is closed by default — changed to textContent.
  • issues-notes-spec.js (1 failure at line 447): The processed .note div — changed div.innerText to div.textContent.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot still seeing issues?

Copilot AI commented May 14, 2026

Copy link
Copy Markdown
Contributor

@copilot still seeing issues?

Fixed in 79aec16. The root cause was that tests/spec/SpecHelper.js had the same Safari postMessage bug that was previously fixed in tests/unit/SpecHelper.jsev.source.document throws SecurityError in Safari for opaque-origin srcdoc iframes, causing the "end-all" message handler to fail silently and tests to time out.

The fix applies the same pattern: settle() guard for single resolution, try/catch around the ev.source.document check, ev.data type guard, and a 100ms polling fallback on doc.respec.ready.

@marcoscaceres marcoscaceres added the AI Contains AI-generated logic; see AI_POLICY.md label Aug 5, 2026
@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot resolve the merge conflicts in this pull request

Copilot AI commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Done — merged latest main and resolved the two conflicts in .github/workflows/pr.yml and .github/workflows/push.yml. The only changes were taking setup-node@v7 (and single-quote style) from main while preserving our Safari matrix entry and test-karma-safari job.

@marcoscaceres

Copy link
Copy Markdown
Contributor Author

@copilot can you fix the lint issues?

Copilot AI commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@copilot can you fix the lint issues?

Fixed in the latest commit — removed the /* eslint-env node */ comment from tests/karma.safari.cjs. ESLint's flat config no longer supports eslint-env inline comments; node globals are already covered by the project-wide globals.node in eslint.config.js.

@marcoscaceres
marcoscaceres force-pushed the chore/safari-launcher branch from cdaf07b to c48aea0 Compare August 13, 2026 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Contains AI-generated logic; see AI_POLICY.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants