Proposed 2.0 schema for AI/ML BOM

[mrutkows]

Note: I rebased from "2.0-dev" and then manually brought in schemas from "2.0-threadmodeling" and also from "2.0-blueprints" and this PR is against master...

The only files that matter (if other branches are merged) are:

• schema/2.0/model/cyclonedx-ai-ml-2.0.schema.json
• .gitignore (to ignore files from MacOS and IDEs going forward)

Plus, want to delete the 1.7 port of:

• schema/2.0/model/cyclonedx-ai-modelcard-2.0.schema.json

[mrutkows]

Please note that the patterns found that begin with "^cdx:ai-ml:" can be further constrained to specific taxonomic categories in the referenced AI/ML v2.0 CycloneDX Property Tax. proposal PR (draft). This is included in the proposed schema and reflected in the corollary PR: CycloneDX/cyclonedx-property-taxonomy#175

[mrutkows]

@stevespringett Here is the "off-the-top-of-my-head" list of things that perhaps need thought around risk schema...

Proposed updates the risk schema's "assessment" object

goal: to be used for AI/ML or other types of assessments per regulatory requirements changes needed tot he "risk" schema's assessment object to address the needs for AI regulations (in these terms):

• lists of "harms" and "benefits" relative to using the component (model) assumed to help assess risk scoring
  - i.e., the "pros" and "cons" found by the assessment

Currently, they are declared in "consideration":

        "benefits": {
          "type": "string",
          "title": "Benefits",
          "description": "Expected benefits to the identified groups at risk."
        },
        "harms": {
          "type": "string",
          "title": "Harms",
          "description": "Expected harms to the identified groups at risk."
        },

Promote measure (metrics) and type groupings to "common":

 "measure": {
      "type": "object",
      "title": "Measure",
      "description": "A generic measurement with a value and unit.",
      "required": [
        "value",
        "unit"
      ],
      "additionalProperties": false,
      "properties": {
        "value": {
          "type": "number",
          "title": "Value",
          "description": "The numeric value of the measurement."
        },
        "unit": {
          "type": "string",
          "title": "Unit",
          "description": "The unit of measurement.",
          "anyOf": [
            {"$ref": "#/$defs/timeUnits"},
            {"$ref": "#/$defs/throughputUnits"},
            {"$ref": "#/$defs/computeUnits"},
            {"$ref": "#/$defs/memoryUnits"},
            {"$ref": "#/$defs/ratioUnits"},
            {"$ref": "#/$defs/energyUnits"},
            {"$ref": "#/$defs/co2Units"},
            {"$ref": "#/$defs/customUnitPattern"}
          ]
        }
      }
    },

Note: these include metrics for "enviromentalConsiderations" which have been candidates for move to "common" as well.

Input/Output Formats and encodings

Similarly, "format" and "encoding" objects and values are candidates for promotion to "common":

 "format": {
          "type": "object",
          "title": "Format",
          "$comment": "TODO: There should be a common place in the CycloneDX Property Taxonomy to define encoding types",
          "description": "Describes the format and encoding of the parameter data.",
          "additionalProperties": false,
          "properties": {
            "dataType": {
              "type": "string",
              "title": "Data Type",
              "description": "The data type of the parameter.",
              "examples": [
                "string",
                "float32",
                "fp16",
                "bf16",
                "int64",
                "uint8",
                "q8_0"
              ]
            },
            "encoding": {
              "type": "string",
              "title": "Encoding",
              "description": "The encoding format of the parameter data.",
              "anyOf": [
                {
                  "enum": [
                    "utf-8",
                    "ascii",
                    "base64",
                    "jpeg",
                    "png",
                    "wav",
                    "mp3",
                    "flac",
                    "raw",
                    "tokenized",
                    "other"
                  ],
                  "meta:enum": {
                    "utf-8": "UTF-8 text encoding.",
                    "ascii": "ASCII text encoding.",
                    "base64": "Base64 encoded binary data.",
                    "jpeg": "JPEG image format.",
                    "png": "PNG image format.",
                    "wav": "WAV audio format.",
                    "mp3": "MP3 audio format.",
                    "flac": "FLAC lossless audio format.",
                    "raw": "Raw binary data without specific encoding.",
                    "tokenized": "Pre-tokenized input (token IDs).",
                    "other": "Other encoding format not explicitly listed."
                  }
                },
                {
                  "pattern": "^cdx:ai-ml:[a-z0-9:-]+$"
                }
              ],
              "examples": [
                "utf-8",
                "tokenized",
                "jpeg",
                "wav"
              ]
            },

[mrutkows]

@stevespringett added examples to all simple, non-nested schema type within the ai-ml schema file.

[mrutkows]

mrutkows force-pushed the 2.0-dev-ai-ml branch from 8d2ab37 to 8ec6842 1 minute ago

Rebased on latest 2.0-dev branch.

[JunHwan-Kwon]

The proposed modelParameter captures format.dataType, format.encoding, shape, and constraints, but it has no typed field for the affine quantization parameters that give a quantized tensor its numerical meaning.

The schema's own image example illustrates the gap:

{
  "name": "image",
  "format": {
    "dataType": "uint8",
    "encoding": "jpeg",
    "mimeType": "image/jpeg"
  },
  "shape": [null, 3, 224, 224]
}

As a reproducible example, I parsed the MobileNetV2 model contained in Google's hosted mobilenet_v2_1.0_224_quant.tgz archive [1].

Archive SHA-256:
d6a04d780f76f656c902413be432eb349ec4a458240e3739119eb44977f77a79

TFLite member SHA-256:
f08d447cde49b4e0446428aa921aff0a14ea589fa9c5817b31f83128e9a43c1d

Google's LiteRT Interpreter API (ai-edge-litert 2.1.4) reports:

• input: uint8 [1,224,224,3], scale 0.0078125, zero-point 128
• output: uint8 [1,1001], scale 0.09889253973960876, zero-point 58

Using the LiteRT affine mapping f = (q - zeroPoint) * scale [2], the full uint8 code domains map to the following real-value ranges, rounded to four decimal places:

• input: [-1.0000, 0.9922]
• output: [-5.7358, 19.4818]

Therefore, identical dataType and shape declarations can still have different affine numerical interpretations. From the declaration alone, a BOM consumer cannot recover the mapping between integer tensor codes and the represented real values.

In this TFLite case, these values are not merely narrative, author-supplied documentation. LiteRT documents that the converter populates the quantization parameters and stores them in the TFLite model file, where they are available through the Tensor API [2]. Reading these serialized artifact facts requires no inference run, runtime measurement, or heuristic reconstruction.

The underlying numerical contract is not TFLite-specific. ONNX QuantizeLinear likewise represents quantized tensors using scale, zero-point, and explicit granularity semantics [3].

On the available extension points:

format itself is additionalProperties: false with no properties member, so quantization data cannot be attached where dataType lives. It would have to be lifted to modelParameter.properties as name-value strings, where a scale vector would require an ad-hoc serialization and could not be typed or validated by the schema.

The property taxonomy already registers cdx:ai-ml:model:hyperparameter:quantization for model-level storage precision, such as bf16 or q4_k_m. That model-level classification is distinct from the tensor-specific affine parameters discussed here.

The corollary taxonomy PR (CycloneDX/cyclonedx-property-taxonomy#175) does not currently define a typed representation for tensor-level scales, zero-points, or quantization axes. format.dataType is itself only a string and cannot structurally associate those parameters with the tensor or validate their cardinality.

Would an optional, framework-neutral affine-quantization object be in scope?

{
  "name": "image",
  "quantization": {
    "scheme": "affine",
    "granularity": "per-tensor",
    "scales": [0.0078125],
    "zeroPoints": [128]
  }
}

Granularity is not binary. ONNX defines per-tensor, per-axis, and blocked quantization. An axis applies to the latter two, while blocked quantization additionally has block-size semantics [3]. TFLite currently uses per-tensor and per-axis quantization. The example above is intentionally per-tensor; whether axis and block size should be required or have defaults is a schema-design question that this comment does not presume.

If this seems in scope, I would be happy to contribute a concrete schema change with aligned JSON, XML, and protobuf definitions, examples, and validation tests, either as a PR against this branch if that's the preferred route, or in whatever form is most useful.

This proposal covers only the tensor's numerical representation. It does not claim to identify the full application preprocessing pipeline.

[1] https://storage.googleapis.com/download.tensorflow.org/models/tflite_11_05_08/mobilenet_v2_1.0_224_quant.tgz
[2] https://developers.google.com/edge/litert/conversion/tensorflow/metadata
[3] https://onnx.ai/onnx/operators/onnx__QuantizeLinear.html

[mrutkows]

affine-quantization

@JunHwan-Kwon,

I had not considered needing to describe quantization to that detail for the initial design as it was not called out by regulatory agencies (and esp. not affine quantization which I have not personally used in models I encounter within my company). I could see an argument that there might be some additional security/compliance features when matching models to runtimes/frameworks/harnesses/etc. to judge if there was some mismatch. Quant., again in general, might be some loose indicator of trust given loss of precision; however, benchmarks and such would be a better indicator (when provider per-quant.) of suitability of model given precision drop-off.

In addition, the proposed addition assumes a single-file model; whereas, a single model repo. may have many quants. besides the full precision model; therefore, this information would need to be made available on a per-file basis and not simply a model repo. which would require some thought and refactoring as simply adding a single quantization schema would not suffice.

I am curious going back to my first reaction... can you explain, in terms of model security and compliance and apart from my suppositions above, what having the quantization means for evaluating model risk? Again, we are not trying to "run" the model and perform operations on the tensor data...

I am leaning towards perhaps allowing. such add. quantization info. being applied via additional property taxonomy values which would allow them to be applied on a per-file basis... additionally, I will give a hard look at why regulations are asking for overall "model inputs and outputs" with not much specificity and seems to be more about higher-order suitability of purpose.

Would you agree we could use the property taxonomy to optionally annotate model files with these values?

---

Here is an example of what I am thinking:

Preserve the "macro" (Hyperparameter) value : cdx:ai-ml:model:hyperparameter:quantization = "Q4_K_M" for generalized model risk assessment.

Then using "properties" for any possible technical serialization/runtime execution keys (i.e., any compiler/runtime optimizations) which can be applied on a per-parameter (tensor) basis on "modelParameter"; specifically, augmenting its "format":

{
  "name": "image",
  "description": "LiteRT MobileNet input layer",
  "modality": "image",
  "format": {
    "dataType": "uint8",
    "encoding": "raw",
    "properties": [
      {
        "name": "cdx:ai-ml:model:parameter:quantization:scheme",
        "value": "affine_asymmetric"
      },
      {
        "name": "cdx:ai-ml:model:parameter:quantization:scale",
        "value": "0.0078125"
      },
      {
        "name": "cdx:ai-ml:model:parameter:quantization:zeroPoint",
        "value": "128"
      }
    ]
  },
  "shape": [1, 3, 224, 224],
  "processingStage": "post-processing"
}

[JunHwan-Kwon]

@mrutkows

Yes, and thank you for sketching this out concretely. Provided that each model variant or file is represented as a separately hash-identified component, I agree that optional property-taxonomy annotations at the per-file, per-parameter level are an appropriate representation for these values. Keeping cdx:ai-ml:model:hyperparameter:quantization as the macro-level classification while placing serialization facts under each parameter's format.properties keeps the two levels apart. It also resolves the limitation I raised about the closed format object, since the properties array now has an explicit place inside format.

I also agree with your distinction between model suitability and configuration integrity. I am not proposing quantization as a general trust score or as a substitute for accuracy benchmarks. The narrower value is that a deployment system could compare the numerical contract declared by a harness with the contract recorded for a hash-identified model file, without running inference. Without that comparison, a contract mismatch does not surface at build or load time, since the tensor remains valid for its declared dtype and shape.

I extracted 62 complete affine contracts from the external interfaces of 50 public TFLite files. The corpus ledger records the quantization scheme and granularity, together with the scale and zero-point values. Mapped to the proposed property taxonomy, these correspond to scheme, scale, zeroPoint, and granularity. Your example already covers the first three; granularity distinguishes per-tensor from per-axis representations. dataType and shape are already represented by format and modelParameter.

All 62 observed external contracts were per-tensor: each contained one scale and one zero-point. Their serialized TFLite quantized_dimension field was 0, but it did not carry per-axis semantics in those cases.

For future per-axis support, an axis property would become relevant, and scale and zero-point may be vectors rather than scalars. Since property values are strings, a canonical serialization rule would be important: for example, a JSON numeric array with a defined ordering and a cardinality rule tied to the selected axis. Otherwise, two producers could encode the same vector differently. Whether axis should be mandatory or have a defined default is a taxonomy-design question.

One detail in the worked example may be worth checking. If it is intended to describe the same Google-hosted MobileNetV2 artifact cited above, that artifact exposes an NHWC input shape of [1, 224, 224, 3], rather than [1, 3, 224, 224].

I am less certain that processingStage: "post-processing" is an error. Under the current enum descriptions, a raw source image would be pre-processing, while the tensor presented to the model may already be post-processing relative to the input pipeline. It may be useful to clarify what that field is relative to before changing the example.

For context, I also ran a paired experiment on a deterministic 1,000-image ImageNetV2 subset. For each target model, the artifact, images, resize and crop procedure, and runtime were fixed while only the affine input encoder was replaced with the other model's contract. EfficientNet-Lite0 changed from 59.3% to 50.1% top-1 accuracy (two-sided exact McNemar p = 5.5e-14), while MobileNetV2 changed from 58.1% to 56.8% and was not statistically significant (p = 0.182). This is not a prevalence estimate; it demonstrates that a structurally valid contract mismatch can produce a material accuracy change for one target artifact while remaining statistically non-significant for another.

The corpus, artifact hashes, interface ledgers, per-image predictions, and verification scripts are available here:
https://github.com/JunHwan-Kwon/tensor-quantization-metadata-study

For context on where my interest comes from: my work is in medical device software, where small numerical differences can matter more than they would elsewhere, and where a change you cannot see is harder to manage than one you can. That is my own vantage point rather than a general argument, and I would not want the case to rest on it. SBOM and ML-BOM have been genuinely useful to me in that setting, and I am aware I am arriving late to work you have already carried a long way.

I would be glad to help draft the taxonomy entries and worked examples for full-precision and per-tensor affine files, and to help define a canonical per-axis representation, following whichever contribution format is most useful for CycloneDX/cyclonedx-property-taxonomy#175.