Skip to content

Allow Name Aliasing on SpecifcAvroReader #3670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
snoz2191 opened this issue Apr 14, 2025 · 0 comments
Open

Allow Name Aliasing on SpecifcAvroReader #3670

snoz2191 opened this issue Apr 14, 2025 · 0 comments

Comments

@snoz2191
Copy link

Problematic

Currently we have an error where the SpecificAvroReader is not able to deserialize correctly an Avro file while the flag for Specific Avro Reader is set to true.

Context

We have the following Avro schema:

{
  "namespace": "com.mycompany.data.schemas.avro",
  "type": "record",
  "name": "OnlineStatusChangedEvent",
  "aliases": ["OnlineStatusChange"],
  "fields": [
    {
      "name": "Id",
      "type": "long"
    },
    {
      "name": "currentStatus",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "previousStatus",
      "type": [
        "null",
        "string"
      ],
      "default": null
    }
  ]
}

And this generates, through the avro-maven-plugin a corresponding class which would fall under com.mycompany.data.schemas.avro.OnlineStatusChangedEvent. Now this change is both forwards and backwards compatible based on the latest Avro specification.

Nevertheless, when a consumer tries to deseriliaze an event written in this new schema, while having the previous schema and the associated old class, we get the following error:

Caused by: org.apache.kafka.common.errors.SerializationException: Could not find class com.mycompany.data.schemas.avro.OnlineStatusChangedEvent specified in writer's schema whilst finding reader's schema for a SpecificRecord.
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.getSpecificReaderSchema(AbstractKafkaAvroDeserializer.java:336)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.getReaderSchema(AbstractKafkaAvroDeserializer.java:307)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.access$000(AbstractKafkaAvroDeserializer.java:57)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer$1.load(AbstractKafkaAvroDeserializer.java:73)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer$1.load(AbstractKafkaAvroDeserializer.java:68)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
	... 24 common frames omitted

Goal/Expectation

If the Avro specification allows for name aliasing, the deserializer should be able to, in case the original class is not available, to search if one of the aliases mentioned has a corresponding class. Hence, we would ensure that our Deserializer allows for the same behaviour as the Avro schemas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant