Skip to content

fix(google_ads): keep quota exhaustion out of error tracking - #89598

Open
Gilbert09 wants to merge 2 commits into
masterfrom
posthog/google-ads-quota-exhausted-retryable
Open

fix(google_ads): keep quota exhaustion out of error tracking#89598
Gilbert09 wants to merge 2 commits into
masterfrom
posthog/google-ads-quota-exhausted-retryable

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

A Google Ads sync that hits the account's request quota keeps getting flagged in error tracking as a bug, even though the sync recovers on its own. See error tracking issue 01a03f10-643d-7080-8a99-dd2085bfd75b.

google_ads.py already rides out a quota/rate-limit RESOURCE_EXHAUSTED response with in-process backoff (_call_with_transient_retry). Once that budget is exhausted it re-raises, and Temporal retries the whole sync activity — but GoogleAdsSource had no get_retryable_errors() override, so the exception still got logged as an exception and reported as a fresh bug on every quota window that outlasts the in-process budget.

Changes

  • GoogleAdsSource.get_retryable_errors() now matches the quota-exhausted message, so this failure is logged as a warning instead of an exception once the in-process retry budget is used up.
  • Mechanical: no change to retry counts, backoff, or Temporal's activity retry policy — only which log level and error-tracking captures this specific message.

Follows the same pattern already used by other ad sources for self-recovering failures (meta_ads, tiktok_ads, linkedin_ads, stripe).

How did you test this code?

Added TestGoogleAdsRetryableErrors, covering:

  • the quota-exhausted message matches get_retryable_errors() — guards against this becoming error-tracking noise again
  • the deterministic "message too large" RESOURCE_EXHAUSTED variant does NOT match — that one isn't retried in-process and must stay visible

Ran the full google_ads source test suite locally: 340 passed. Ruff lint/format clean. Not run: mypy repo-wide (change only adds a method matching the base class's existing set[str] signature, no new types) and hogli review (Greptile not signed in in this sandbox — did a manual self-review of the diff instead, no findings).

Automatic notifications

  • Publish to changelog?

Docs update

None — no user-facing or documented behavior changes.

🤖 Agent context

Autonomy: Fully autonomous

Investigated via PostHog error tracking (query-error-tracking-issue, query-error-tracking-issue-events) to confirm the failure originates in google_ads.py's GoogleAdsSearchService.search, then read the source and its existing transient-retry handling. Skills invoked: /writing-tests, /writing-pr-descriptions, /reviewing-before-pr (Greptile unavailable, fell back to a manual self-review). Checked for duplicate open PRs (gh pr list --search, and every open PR by this automation) — none address this issue.


Created with PostHog Desktop

Google Ads quota exhaustion (RESOURCE_EXHAUSTED) is already retried in-process with
backoff. Once that budget is exhausted it re-raises and Temporal retries the whole
activity, but the source had no `get_retryable_errors()` override, so the exception was
still logged and reported to error tracking as if it were a bug.

Add the stable "Resource has been exhausted (e.g. check quota)" message to
`get_retryable_errors()`, matching the pattern already used by other ad sources
(meta_ads, tiktok_ads, linkedin_ads, stripe).

Generated-By: PostHog Desktop
Task-Id: 70b36f4c-1152-47ea-a296-ada86ca97b3f
@trunk-io

trunk-io Bot commented Aug 26, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

Copy link
Copy Markdown
Contributor

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 26, 2026 17:22
@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

@Gilbert09 Gilbert09 added the stamphog Request AI approval (no full review) label Aug 26, 2026 — with PostHog

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, contained fix by an owning-team author that reclassifies a self-recovering quota error out of error tracking; includes tests and no risky-territory surface (no API/data/auth/billing/CI changes).

  • Author wrote 0% of the modified lines and has 10 merged PRs in these paths (familiarity MODERATE).
Gate mechanics and policy version
Gate Result
prerequisites all clear
deny-list no deny categories matched
size 9L, 1F substantive, 36L/2F incl. docs/generated/snapshots — within ceiling
tier T1-agent / T1b-small (36L, 2F, single-area, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 45adf7a · reviewed head 45adf7a

@trunk-io

trunk-io Bot commented Aug 26, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Request AI approval (no full review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant