Skip to content

fix(tess2): test Apple TARGET_OS_* macros by value, not defined() - #561

Open
smeyfroi wants to merge 1 commit into
openframeworks:bleedingfrom
smeyfroi:fix/tess2-16bit-index-on-macos
Open

fix(tess2): test Apple TARGET_OS_* macros by value, not defined()#561
smeyfroi wants to merge 1 commit into
openframeworks:bleedingfrom
smeyfroi:fix/tess2-16bit-index-on-macos

Conversation

@smeyfroi

@smeyfroi smeyfroi commented Jul 31, 2026

Copy link
Copy Markdown

Problem

Since #552, every macOS build gets 16-bit TESSindex — and with it a 16-bit ofIndexType in openFrameworks (ofConstants.h: typedef TESSindex ofIndexType). Downstream, the OF renderers hardcode GL_UNSIGNED_INT for indexed draws on desktop, so every indexed/instanced draw reads 2× past the uploaded index buffer: out-of-range vertex fetches that render as garbage triangles or nothing at all, varying with GPU heap layout — non-deterministic across launches and relinks. It also caps indexed meshes at 65k vertices on macOS. Diagnosed on macOS 26 / Apple silicon against of_v20260730 nightlies; of_v20251123 (predating #552's artifacts) is unaffected.

Cause

TargetConditionals.h defines every TARGET_OS_* macro on every Apple platform — as 1 or 0. On macOS, TARGET_OS_IPHONE is defined as 0, so defined(TARGET_OS_IPHONE) is true. And this very patch adds #include <TargetConditionals.h> to the top of tesselator.h, so the macros are always in scope.

History:

  • Tess2 fixes for emscripten #552 (064906de, "Tess2 fixes for emscripten") rewrote the index-width condition from its original value-form — #if TARGET_OS_IPHONE || ANDROID || __ARMEL__ || EMSCRIPTEN — to a defined()-form. The value-form was correct: undefined identifiers evaluate to 0 in #if, and Apple documents TARGET_OS_* macros as value-tested. The rewrite flipped macOS to 16-bit indices.
  • tess2 apple silicon fix arm defines #558 (9a6d0b83, "tess2 apple silicon fix arm defines") noticed Apple builds landing in the 16-bit branch, but attributed it to the __arm__-family defines and guarded those with !defined(__APPLE__). __arm__ / __aarch32__ aren't defined on Apple arm64, so the actual macOS trigger — defined(TARGET_OS_IPHONE) — survived that fix.

Fix

Test the Apple TARGET_OS_* macros by value in the canonical defined(X) && X form (robust under -Wundef), keeping plain defined() for the non-Apple macros and #558's arm guard intact.

A companion openFrameworks PR makes the renderers derive the GL index type from sizeof(ofIndexType) so they stay correct under either header: openframeworks/openFrameworks#8526. Both fixes stand alone.

Verification

Applied both the current and the patched tess2.patch to a fresh libtess2 v1.0.2 checkout with the formula's own command (patch -p1 -u -N; both apply with zero rejects), then compiled _Static_assert(sizeof(TESSindex) == EXPECT) probes against the resulting headers, all with -Wundef -Werror:

tess2.patch Target sizeof(TESSindex)
current (bleeding) macOS 2 — the bug
this PR macOS 4 — fixed
this PR iOS (xcrun --sdk iphoneos, arm64) 2 — unchanged
this PR no Apple macros defined (Linux/Windows-like) 4 — unchanged

-Wundef -Werror staying silent confirms the defined(X) && X idiom is warning-clean on every target.

TargetConditionals.h defines EVERY TARGET_OS_* macro on EVERY Apple
platform, as 1 or 0 — and this very patch adds
#include <TargetConditionals.h> to the top of tesselator.h, so the
macros are always in scope. On macOS, TARGET_OS_IPHONE is defined as 0,
which makes defined(TARGET_OS_IPHONE) true and silently narrows tess2's
TESSindex — and with it openFrameworks' ofIndexType (ofConstants.h:
typedef TESSindex ofIndexType) — to unsigned short on every macOS build.

History of the regression:

- 064906d ('Tess2 fixes for emscripten', openframeworks#552, 2026-04-24) rewrote the
  index-width condition from its original value-form
      #if TARGET_OS_IPHONE || ANDROID || __ARMEL__ || EMSCRIPTEN
  to a defined()-form. The value-form was correct: undefined
  identifiers evaluate to 0 in #if, and Apple documents TARGET_OS_*
  macros as value-tested. The rewrite flipped macOS to 16-bit indices.

- 9a6d0b8 ('tess2 apple silicon fix arm defines', openframeworks#558, 2026-06-13)
  noticed Apple builds landing in the 16-bit branch, but attributed it
  to the __arm__-family defines and guarded those with
  !defined(__APPLE__). __arm__ / __aarch32__ are not defined on Apple
  arm64, so the actual macOS trigger — defined(TARGET_OS_IPHONE) —
  survived that fix.

Downstream effect in openFrameworks: ofGLRenderer and
ofGLProgrammableRenderer hardcode GL_UNSIGNED_INT for indexed draws on
desktop, so a 16-bit ofIndexType makes every indexed/instanced draw
read 2x past the uploaded index buffer — out-of-range vertex fetches
that render as garbage triangles or nothing at all, varying with GPU
heap layout, i.e. non-deterministic across launches and relinks.
Diagnosed on macOS 26 / Apple silicon against of_v20260730 nightlies;
of_v20251123 (which predates openframeworks#552's artifacts) is unaffected. A
renderer-side fix deriving the GL index type from sizeof(ofIndexType)
is proposed separately in openFrameworks; both fixes stand on their
own, and this one also restores the full 32-bit vertex range for
indexed meshes on macOS.

Fix: test the Apple TARGET_OS_* macros by value in the canonical
defined(X) && X form (robust under -Wundef), keeping plain defined()
for the non-Apple macros and openframeworks#558's arm guard intact.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant