Summary
Currently, for Parquet Dagger Source, depot has been added to minimalJar. Depot comes with its own transitive dependencies on org.apache.httpcomponents:httpclient with version 4.5.13. However, dagger-functions module uses org.elasticsearch.client:elasticsearch-rest-client:6.6.1 which has a transitive dependency on org.apache.httpcomponents:httpclient with version 4.5.6. This causes the classloader in Flink jobmanager to get confused at runtime when the job is submitted, as there are two different modules using 2 versions of the same dependency. It throws a LinkageError.
Proposed solution
To exclude org.apache.httpcomponents:httpclient dependency from depot in build.gradle of dagger-core module.
Additional context
This is the error that comes
Summary
Currently, for Parquet Dagger Source, depot has been added to minimalJar. Depot comes with its own transitive dependencies on
org.apache.httpcomponents:httpclientwith version4.5.13. However,dagger-functionsmodule usesorg.elasticsearch.client:elasticsearch-rest-client:6.6.1which has a transitive dependency onorg.apache.httpcomponents:httpclientwith version4.5.6. This causes the classloader in Flink jobmanager to get confused at runtime when the job is submitted, as there are two different modules using 2 versions of the same dependency. It throws a LinkageError.Proposed solution
To exclude
org.apache.httpcomponents:httpclientdependency from depot inbuild.gradleofdagger-coremodule.Additional context
This is the error that comes