kafkastore.topic unbounded and impacting startup time

[SgtPepperLHCB]

We have a user of our schema-registry cluster who, through their continous-automation, adds and removes a "test" schema subject. After many months of this, it's become clear the kafkastore.topic grows unbounded and it's impacting our startup times.

For example, 2/3 of the records in the kafkastore.topic are associated with these removed subjects. It's understandable why the schemas are never deleted, but not why the DELETE_SUBJECT artifacts are not.

To mitigate, we've increased kafkastore.timeout.ms=120000, but this will continue to be an issue.

[rayokota]

@SgtPepperLHCB , you can now use DELETE /subjects/{subject}/versions/{version}?permanent=true and the removed schemas will be tombstoned and later removed during compaction

[dylanmei]

@rayokota is there an option to make this the default behavior?

[rayokota]

Not currently

[dmariassy]

Hi,

I know that this is an old issue but it's the only one I was able to find that concerns DELETE_SUBJECT messages.

We have tried to do a mass clean up of our SR instance, and deleted a ton of subjects with ?permanent=true. These requests produced the expected tombstones, and will get compacted away.

However, the non-permanent deletes that needed to precede the ?permanent=true requests have generated a lot of DELETE_SUBJECT messages. These have the same cardinality as the input subject versions, and don't have corresponding tombstones.

As a result, it looks like we will be left with the exact same number of messages in our _schemas topic as when we started the clean up exercise.

Is this intentional? Are we missing something?