Search code examples
amazon-web-servicesapache-flinkamazon-kinesis-analytics

aws kinesis data analytics application (flink) change property originally located at flink-conf.yaml


As the runtime of my flink application I use managed flink by AWS (Kinesis Data Analytics Application)

I added functionality (sink) for write processed events from kinesis queue in S3 in a parquet format. Locally everything works for me, but when I try to run the application in the cloud I get the following exception:

    "throwableInformation": [
        "com.esotericsoftware.kryo.KryoException: Error constructing instance of class: org.apache.avro.Schema$LockableArrayList",
        "Serialization trace:",
        "types (org.apache.avro.Schema$UnionSchema)",
        "schema (org.apache.avro.Schema$Field)",
        "fieldMap (org.apache.avro.Schema$RecordSchema)",

After finding a solution to the problem, I found that I need to change following property (checked this on a local cluster):

classloader.resolve-order: child-first -> classloader.resolve-order: parent-first

Is it possible to change this configuration when using AWS managed Fink (not EMR, Kinesis Data Analytics applications) in any way?


Solution

  • aws support answer: No. This property cannot be changed.