Skip to content

feat(taxii2): migrate connector to manager-supported mode (#6859) - #7340

Open
Hugo Dupras (jabesq) wants to merge 8 commits into
masterfrom
feat/6859-manager-supported-taxii2
Open

feat(taxii2): migrate connector to manager-supported mode (#6859)#7340
Hugo Dupras (jabesq) wants to merge 8 commits into
masterfrom
feat/6859-manager-supported-taxii2

Conversation

@jabesq

Copy link
Copy Markdown
Member

Proposed changes

  • Normalize docker-compose.yml and config.yml.sample (required vars uncommented as ChangeMe, defaulted vars commented out; fix a quoted boolean)
  • Add Pydantic settings mirroring the connector's existing configuration 1:1 — 54 TAXII2_* variables, with SecretStr for credentials and ListFromString for comma-separated fields
  • Wire the settings into the existing ConfigConnector in place — the helper is now built from to_helper_config()
  • Set manager_supported: true in the connector manifest
  • Generate __metadata__/connector_config_schema.json and CONNECTOR_CONFIG_DOC.md
  • Give connector.id a unique UUIDv4 default (the SDK declares it required with no default)
  • Add unit tests covering settings validation and the manager-supported wiring
  • Drop the now-unused PyYAML dependency

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

Structure-preserving ("lite") migration: ConfigConnector keeps its name and file,
and the entry point, client, converter, scheduling and logging are untouched. The
connector is manager-supported but not yet "verified".

CONNECTOR_UPDATE_EXISTING_DATA was dropped — it was read into an attribute that was
never used anywhere and is not part of the SDK connector config. PyYAML became dead
once the migration removed the last yaml.load call; it is still installed
transitively via pycti, so runtime behaviour is unchanged.

Validation: isort, black, flake8 --ignore=E,W and the custom STIX-ID pylint checker
(10.00/10) all pass; 13 unit tests pass. Every connector module still imports from a
venv built only from src/requirements.txt. Regenerating the config schema produces
no diff against the committed version.

Copilot AI lite review requested due to automatic review settings August 21, 2026 13:35
@jabesq Hugo Dupras (jabesq) added the filigran team Item from the Filigran team. label Aug 21, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Unused dependencies detected

The following packages appear to be unused:

  • antlr4-python3-runtime in external-import/taxii2/src/requirements.txt
  • chardet in external-import/taxii2/src/requirements.txt
  • datefinder in external-import/taxii2/src/requirements.txt
  • idna in external-import/taxii2/src/requirements.txt
  • pika in external-import/taxii2/src/requirements.txt
  • python-dateutil in external-import/taxii2/src/requirements.txt
  • python-magic in external-import/taxii2/src/requirements.txt
  • pytz in external-import/taxii2/src/requirements.txt
  • regex in external-import/taxii2/src/requirements.txt
  • simplejson in external-import/taxii2/src/requirements.txt
  • six in external-import/taxii2/src/requirements.txt
  • sseclient in external-import/taxii2/src/requirements.txt
  • stix2-patterns in external-import/taxii2/src/requirements.txt
  • urllib3 in external-import/taxii2/src/requirements.txt

How to fix: (1) if the package is used but under a different import name (e.g. PyYAMLyaml), add a mapping line to .github/deptry-package-map.txt. (2) if truly unused, remove it from the connector's requirements.txt.

@github-actions

Copy link
Copy Markdown

🔴 Connector Linter errors detected

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the external-import/taxii2 connector to a manager-supported configuration model by introducing connectors-sdk/Pydantic-based settings, wiring those settings into the existing ConfigConnector, and regenerating the connector’s metadata (schema + docs) accordingly.

Changes:

  • Introduces Pydantic settings models for all TAXII2 configuration variables and uses to_helper_config() to build the OpenCTIConnectorHelper config.
  • Normalizes sample configuration files (docker-compose.yml, config.yml.sample) and marks the connector as manager_supported.
  • Adds unit tests for settings validation and for verifying the manager-supported wiring path.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
external-import/taxii2/tests/tests_connector/test_settings.py Adds unit tests for settings validation, defaults, list parsing, and SecretStr handling.
external-import/taxii2/tests/tests_connector/init.py Initializes the tests subpackage.
external-import/taxii2/tests/test-requirements.txt Adds pytest + reuses connector runtime requirements for tests.
external-import/taxii2/tests/test_main.py Adds integration-style unit tests to validate helper/config/connector wiring.
external-import/taxii2/tests/conftest.py Adds sys.path setup so tests can import connector code from src/.
external-import/taxii2/src/requirements.txt Adds pydantic and connectors-sdk; removes unused PyYAML.
external-import/taxii2/src/connector/settings.py Introduces ConnectorSettings plus TAXII2-specific config model defaults and types.
external-import/taxii2/src/connector/connector.py Switches helper initialization to use to_helper_config() output.
external-import/taxii2/src/connector/config_variables.py Replaces legacy YAML/env parsing with validated ConnectorSettings and secret unwrapping.
external-import/taxii2/src/connector/init.py Exposes ConnectorSettings from the connector package.
external-import/taxii2/src/config.yml.sample Normalizes sample config layout and documents required vs defaulted values.
external-import/taxii2/src/init.py Exposes ConnectorSettings at the src package level.
external-import/taxii2/docker-compose.yml Normalizes docker-compose env var documentation for manager-supported usage.
external-import/taxii2/metadata/connector_manifest.json Sets manager_supported: true.
external-import/taxii2/metadata/connector_config_schema.json Adds generated JSON schema for connector configuration.
external-import/taxii2/metadata/CONNECTOR_CONFIG_DOC.md Adds generated configuration documentation.
Suppressed comments (1)

external-import/taxii2/docker-compose.yml:36

  • The comment says TAXII2_INTERVAL is only used when CONNECTOR_DURATION_PERIOD is not set, but CONNECTOR_DURATION_PERIOD has a default (PT60M) in the new settings, so that fallback won't normally be reachable. This is misleading in the docker-compose example.
      # - TAXII2_INTERVAL=1 # Default 1 (hour). Only used when CONNECTOR_DURATION_PERIOD is not set

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

# v21: true # Default true, set to false for a TAXII 2.0 server
# collections: '*.*' # Default *.*
# initial_history: 24 # Default 24 (hours)
# interval: 1 # Default 1 (hour), only used when connector.duration_period is not set
# - TAXII2_USERNAME=ChangeMe # Username for basic authentication
# - TAXII2_PASSWORD=ChangeMe # Password for basic authentication
# - TAXII2_USE_TOKEN=false # Default false. Use bearer token authentication instead of basic auth
# - TAXII2_TOKEN=ChangeMe # Bearer token, used when TAXII2_USE_TOKEN is true
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

filigran team Item from the Filigran team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(TAXII2): migrate connector to the catalog

3 participants