Add GGML_OPENVINO_ENABLE_WEIGHT_RELEASE env variable for GPU plugin

[zhaixuejun1993]

This pull request introduces a mechanism for releasing (freeing) host memory used for OpenVINO model weights after the model has been compiled and cached, primarily to reduce memory usage in long-running processes. The changes add tracking and statistics for weight buffers, carefully manage when and how host memory can be released, and ensure that attempts to reuse or rebuild graphs after release are properly guarded. Additionally, new environment variables and logging are included to control and monitor this feature.

The most important changes are:

OpenVINO Weight Release Mechanism:

• Added infrastructure to track, identify, and release host memory backing model weights after compilation, including new types (RELEASED_WEIGHT) and statistics structures, and a mechanism to safely free memory and prevent further use. (ggml-openvino/ggml-openvino.cpp, ggml-openvino/ggml-openvino-extra.h, ggml-openvino/ggml-decoder.cpp) [1] [2] [3] [4]

Runtime Context and Safety Guards:

• Introduced new runtime context states to track weight release, enforced cache requirements after release, and added assertions to prevent uncached or dynamic graph usage after host memory is freed. (ggml-openvino/utils.cpp) [1] [2] [3]

APIs and Logging:

• Exposed new API functions for collecting weight names, getting weight release stats, and releasing weight candidates, as well as detailed logging and environment variable controls for enabling/disabling the feature. (ggml-openvino/ggml-decoder.h, ggml-openvino/ggml-openvino-extra.h, ggml-openvino/ggml-openvino-extra.cpp, ggml-openvino/utils.cpp) [1] [2] [3] [4]

Code Structure and Integration:

• Registered buffer contexts globally for tracking, ensured thread safety with mutexes, and integrated the release mechanism into the OpenVINO backend lifecycle. (ggml-openvino/ggml-openvino.cpp) [1] [2] [3]

Safety and Error Handling:

• Added explicit aborts and error messages if attempts are made to recreate or access released weights, ensuring safe and predictable behavior. (ggml-openvino/ggml-decoder.cpp)

These changes together enable significant memory savings for OpenVINO graph execution, especially on GPU, while maintaining safety and debuggability.…E to enable const node weight release for GPU plugin

Overview

Additional information

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure:

[zhaixuejun1993]

replaced by #275