Skip to content

Gson support #10104

Closed
Closed

Description

@frank-montyne

Is your enhancement request related to a problem? Please describe.
In our projects we are using Gson not Jackson for Json serialization/deserialization. On top of that we make use of immutables (immutables.org) to define our data objects. Getters and setters are of the form field(), field(type data) not getField(), setField(...). When passing these data classes to JSONSchema.of() the fields are not recognized and we end up with null data in the topic when sending messages using a producer.
Another problem is that in some cases we use json adapters to create a json representation that is different from how the fields are represented in the data class.
At this point it is not possible to set your own JSON schema and pass the JSON string representation instead of the data object.
The SchemaBuilder does not seem to support schema's of type "JSON" .
I tried passing a schema that was created by the JSONSchema class using
String json = JSONSchema.of(Data.class).getSchemaInfo().toString() and then passing the resulting string to
SchemaDefinition.builder().withJsonDef(json).build()
This produces org.apache.avro.SchemaParseException: Type not supported: JSON)

Describe the solution you'd like
Support of JSON schema's in the SchemaDefinition builder. Some documentation on JSON schema's would also be nice.
Support for passing the json representation (string) instead of a POJO.
Direct support for Gson and immutables would be nice.
It would also be nice to be able to specify a limited set of fields to be stored. The json representation of the data object should then be stored as a separate string field.

Describe alternatives you've considered
Switching to Jackson is not an option. Our whole code base is based on Gson for Json related stuff.
Creating mirror classes to convert our immutables to plain and simple POJO's but that's way too much work and a maintenance nightmare.

Additional context
Using version 2.7.1

Activity

  1. added
    type/enhancementThe enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
    on Mar 31, 2021
  2. gmethvin commented on Apr 3, 2021

    @gmethvin
    Member

    I don't think this issue is so much about Gson support as it is about making it easy to generate schemas via other means, using other libraries. We've had similar challenges with pulsar4s and generating JSON schemas to support various Scala JSON libraries.

    In my opinion the JSON schema support should be entirely pluggable. The core Schema interface should exist in the pulsar client library, and there should be separate libraries that implement it for Jackson, Gson, json4s, etc.

  3. frank-montyne commented on Apr 4, 2021

    @frank-montyne
    Author

    I completely agree. Another problem is the current reliance on getter/setter method to determine the properties of the POJO. Immutables (immutables.org) work with builders instead of setter methods and by default the getter methods for properties are not prefixed with 'get'.

  4. codelipenghui commented on Mar 4, 2022

    @codelipenghui
    Contributor

    The issue had no activity for 30 days, mark with Stale label.

  5. locked and limited conversation to collaborators on Dec 7, 2022
  6. converted this issue into a discussion #18786 on Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    lifecycle/staletype/enhancementThe enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions