face: fix facemark landmarks returned as all zeros on OpenCV 5

[aarochu]

Fixes opencv/opencv#29703

Problem

Facemark::fit() returns the correct number of landmarks with every point left at (0,0) on OpenCV 5, while the same code works on 4.x. The reporter hit this with FacemarkLBF and the pretrained lbfmodel.yaml.

It only happens when the output is a std::vector<std::vector<Point2f>> — which is what the documentation, the samples and the reporter's code all pass — so landmarks[i].size() looks right (68) and every coordinate is zero.

Root cause

_copyVector2Output() writes each result through a Mat header obtained from OutputArray::getMat(i). That header only aliases the destination vector's storage, so the source has to match its shape already: copyTo() calls create() first, and when the shape differs the header is reallocated, so the copy lands in a fresh buffer that is discarded instead of in the caller's vector.

The source was built as Mat(Mat(vec[i]).t()) to match the 1 x N header OpenCV 4 hands out for a vector<vector<T>>:

// modules/core/src/matrix_wrap.cpp, _InputArray::getMat_()
// 4.x
return v->empty() ? Mat() : Mat(1, int(v->size()), t, v->data());
// 5.x
int v_sz = int(v->size());
return v->empty() ? Mat() : Mat(1, &v_sz, t, v->data());

OpenCV 5 returns a 1-D header there instead, so the transposed source no longer matches and every landmark is silently dropped.

Fix

Build the source as a 1 x N two-channel row with reshape(), which matches the destination header on both 4.x and 5.x, and size the destination from the actual landmark count instead of a hard-coded 68 (so models with a different number of landmarks work too).

The helper is duplicated in the LBF, AAM and Kazemi implementations and all three were affected, so all three are fixed.

The Mat/UMat output branches are left alone: they bind the real destination object rather than an aliasing header, so a reallocating copyTo() still delivers the data there. Changing them would alter their output shape without fixing a bug.

Testing

Verified on a local Windows/MSVC build of 5.x + contrib (5.1.0-dev), using the reporter's exact sequence (cascade → detectMultiScale → FacemarkLBF::create/loadModel/fit) with the pretrained lbfmodel.yaml from the issue:

Before:

Faces: [136 x 136 from (235, 237)]
Number of landmarks: 68
Non-zero landmarks: 0 / 68
First 5 landmarks: [0, 0] [0, 0] [0, 0] [0, 0] [0, 0]

After:

Faces: [136 x 136 from (235, 237)]
Number of landmarks: 68
Non-zero landmarks: 68 / 68
First 5 landmarks: [213.177, 280.037] [213.544, 301.395] [214.891, 322.97] [218.014, 343.195] [227.687, 361.095]

I also reproduced the underlying shape mismatch in isolation against opencv_core 5.x and confirmed the replacement copies correctly for 5, 68 and 194 landmarks as well as the empty case.

The existing CV_Face_FacemarkLBF.test_workflow and CV_Face_FacemarkAAM.test_workflow only asserted the landmark count, which stayed correct throughout — that is why this went unnoticed. They now also assert the points actually reached the caller. Confirmed this is a real regression test: with the source fix reverted and only the test change applied, it fails with

  nonzero
    Which is: 0
  facial_points[0].size()
    Which is: 68

Full opencv_test_face suite passes with the fix (19/19, with OPENCV_TEST_DATA_PATH plus the face_landmark_model.dat the module downloads at configure time).

[asmorkalov]

Please drop irrelevant comments from the code.

[aarochu]

Comments should be gone!

[fallenmi]

Reviewed with OpenAI Codex assistance; I independently built and tested the exact revision and verified the results below.

The OpenCV 5 output conversion is fixed. Keeping the new regression assertions while rolling back only the three production conversion hunks made both FacemarkAAM.test_workflow and FacemarkLBF.test_workflow fail deterministically: each returned 68 landmarks and all 68 were (0, 0). Restoring exact head 05f89d83f99373869fd9f4200443a2b2b7cc5c03 made both tests pass with every returned landmark nonzero.

I built the face module against current OpenCV 5.x core and ran the complete face test binary with the matching opencv_extra data: 19/19 tests passed. The dynamic vec[i].size() allocation also removes the hard-coded 68-point assumption while the two-channel reshape preserves the actual Point2f layout. git diff --check is clean.

The published Actions matrix is still red (3 success, 9 failure), so those jobs should be resolved or rerun before merge; my approval is based on the exact local base/head regression and full face-module suite above.

[stealthylabs]

Hi I am the reporter of the original bug, and was wondering what is needed to move this forward, or how can we help further.

Thank you @aarochu