Skip to content

XNNPACK delegate accepts rank-7 FULLY_CONNECTED then fails during Prepare #11036

Description

@laolvfan

Summary

The default XNNPACK delegate claims a rank-7 FULLY_CONNECTED node and then
fails during delegate preparation. This aborts interpreter setup, although
the same FlatBuffer runs correctly with the builtin kernels when default
delegates are disabled.

Environment

  • LiteRT: ai-edge-litert 2.1.6
  • TensorFlow converter: tf-nightly 2.22.0.dev20260808 (also reproduced on
    TensorFlow 2.20.0)
  • Linux x86_64 CPU, Python 3.11

Minimal reproduction

import tensorflow as tf
from ai_edge_litert.interpreter import Interpreter, OpResolverType

x1 = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], shape=[1, 3, 2])


class Model(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.w1 = tf.Variable([1.0] * 6)

    def call(self, x):
        w = tf.reshape(self.w1, (6, 1, 1, 1))
        v = tf.reshape(x, (1, 1, 1, 6))
        y = tf.tensordot(v, w, axes=1)
        return tf.expand_dims(y, axis=-1)  # output rank: 7


model = Model()
print("eager:", model(x1).shape, model(x1).numpy().flatten())
flatbuffer = tf.lite.TFLiteConverter.from_keras_model(model).convert()


def run(**kwargs):
    interpreter = Interpreter(model_content=flatbuffer, **kwargs)
    interpreter.allocate_tensors()
    input_detail = interpreter.get_input_details()[0]
    interpreter.set_tensor(input_detail["index"], x1.numpy())
    interpreter.invoke()
    return interpreter.get_tensor(interpreter.get_output_details()[0]["index"])


print(
    "builtin only:",
    run(
        experimental_op_resolver_type=
        OpResolverType.BUILTIN_WITHOUT_DEFAULT_DELEGATES
    ).flatten(),
)
print("default delegates:", run().flatten())

Actual result

eager: (1, 1, 1, 1, 1, 1, 1) [21.]
builtin only: [21.]
RuntimeError: failed to delegate FULLY_CONNECTED node #1
Node number 2 (TfLiteXNNPackDelegate) failed to prepare.

The default-delegate run raises from allocate_tensors() or invoke() rather
than returning a result.

Expected result

The delegate should either support this node or decline to claim it during
partitioning/support checking. In the latter case, the builtin CPU kernel
should execute the node automatically, as demonstrated by the successful
BUILTIN_WITHOUT_DEFAULT_DELEGATES run. A preparation failure after the node
has been delegated makes an otherwise runnable model fail to load.

Context

The same behavior was reproduced through both tf.lite.Interpreter and
ai_edge_litert. LiteRT maintainers reproduced the failure and asked that it
be tracked in XNNPACK: google-ai-edge/LiteRT#9215

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions