Summary
The default XNNPACK delegate claims a rank-7 FULLY_CONNECTED node and then
fails during delegate preparation. This aborts interpreter setup, although
the same FlatBuffer runs correctly with the builtin kernels when default
delegates are disabled.
Environment
- LiteRT:
ai-edge-litert 2.1.6
- TensorFlow converter:
tf-nightly 2.22.0.dev20260808 (also reproduced on
TensorFlow 2.20.0)
- Linux x86_64 CPU, Python 3.11
Minimal reproduction
import tensorflow as tf
from ai_edge_litert.interpreter import Interpreter, OpResolverType
x1 = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], shape=[1, 3, 2])
class Model(tf.keras.Model):
def __init__(self):
super().__init__()
self.w1 = tf.Variable([1.0] * 6)
def call(self, x):
w = tf.reshape(self.w1, (6, 1, 1, 1))
v = tf.reshape(x, (1, 1, 1, 6))
y = tf.tensordot(v, w, axes=1)
return tf.expand_dims(y, axis=-1) # output rank: 7
model = Model()
print("eager:", model(x1).shape, model(x1).numpy().flatten())
flatbuffer = tf.lite.TFLiteConverter.from_keras_model(model).convert()
def run(**kwargs):
interpreter = Interpreter(model_content=flatbuffer, **kwargs)
interpreter.allocate_tensors()
input_detail = interpreter.get_input_details()[0]
interpreter.set_tensor(input_detail["index"], x1.numpy())
interpreter.invoke()
return interpreter.get_tensor(interpreter.get_output_details()[0]["index"])
print(
"builtin only:",
run(
experimental_op_resolver_type=
OpResolverType.BUILTIN_WITHOUT_DEFAULT_DELEGATES
).flatten(),
)
print("default delegates:", run().flatten())
Actual result
eager: (1, 1, 1, 1, 1, 1, 1) [21.]
builtin only: [21.]
RuntimeError: failed to delegate FULLY_CONNECTED node #1
Node number 2 (TfLiteXNNPackDelegate) failed to prepare.
The default-delegate run raises from allocate_tensors() or invoke() rather
than returning a result.
Expected result
The delegate should either support this node or decline to claim it during
partitioning/support checking. In the latter case, the builtin CPU kernel
should execute the node automatically, as demonstrated by the successful
BUILTIN_WITHOUT_DEFAULT_DELEGATES run. A preparation failure after the node
has been delegated makes an otherwise runnable model fail to load.
Context
The same behavior was reproduced through both tf.lite.Interpreter and
ai_edge_litert. LiteRT maintainers reproduced the failure and asked that it
be tracked in XNNPACK: google-ai-edge/LiteRT#9215
Summary
The default XNNPACK delegate claims a rank-7
FULLY_CONNECTEDnode and thenfails during delegate preparation. This aborts interpreter setup, although
the same FlatBuffer runs correctly with the builtin kernels when default
delegates are disabled.
Environment
ai-edge-litert 2.1.6tf-nightly 2.22.0.dev20260808(also reproduced onTensorFlow 2.20.0)
Minimal reproduction
Actual result
The default-delegate run raises from
allocate_tensors()orinvoke()ratherthan returning a result.
Expected result
The delegate should either support this node or decline to claim it during
partitioning/support checking. In the latter case, the builtin CPU kernel
should execute the node automatically, as demonstrated by the successful
BUILTIN_WITHOUT_DEFAULT_DELEGATESrun. A preparation failure after the nodehas been delegated makes an otherwise runnable model fail to load.
Context
The same behavior was reproduced through both
tf.lite.Interpreterandai_edge_litert. LiteRT maintainers reproduced the failure and asked that itbe tracked in XNNPACK: google-ai-edge/LiteRT#9215