Skip to content

fix(codecs): support Avro Date and TimeMillis logical types - #26112

Open
omwbennett wants to merge 2 commits into
vectordotdev:masterfrom
omwbennett:support-avro-date-timemillis
Open

fix(codecs): support Avro Date and TimeMillis logical types#26112
omwbennett wants to merge 2 commits into
vectordotdev:masterfrom
omwbennett:support-avro-date-timemillis

Conversation

@omwbennett

Copy link
Copy Markdown
Contributor

Summary

Support encoding and decoding Avro Date and TimeMillis values.
Due to the way VRL stores integers, an extra step was added to the avro encoder path to coerce Avro types based on the schema field's logical type.

References

Closes #24773

Vector configuration

sources:
  kafka_avro:
    type: kafka
    bootstrap_servers: "kafka:9092"
    group_id: "vector"
    topics:
      - test_topic
    decoding:
      codec: avro
      avro:
        schema: |
          {
            "type": "record",
            "name": "test",
            "fields": [
              {
                "name": "date_field",
                "type": { "type": "int", "logicalType": "date" }
              }
            ]
          }
        strip_schema_id_prefix: false

sinks:
  console_out:
    type: console
    inputs:
      - kafka_avro
    encoding:
      codec: json

And similar for TimeMillis.

How did you test this PR?

Manual testing with the above configs, added unit tests for coerce_logical_types and enabled the existing date/timemillis avro round trip tests. All tests passing.

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

Contributor Guidelines

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them. To catch issues early, add a pre-push hook (template) or run the following locally before pushing:
    • make fmt
    • make check-clippy (auto-fix with make clippy-fix)
    • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

@omwbennett
omwbennett requested review from a team as code owners August 14, 2026 13:12
@github-actions github-actions Bot added docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation labels Aug 14, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0ca77c6bfb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread lib/codecs/src/encoding/format/avro.rs Outdated
Comment thread lib/codecs/src/encoding/format/avro.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: external docs Anything related to Vector's external, public documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avro decoder doesn't support Date values

2 participants