Skip to content

Exclude jackson-databind from shaded jar#50

Open
jrkinley wants to merge 1 commit intocloudera-labs:branch-0.7from
jrkinley:branch-0.7
Open

Exclude jackson-databind from shaded jar#50
jrkinley wants to merge 1 commit intocloudera-labs:branch-0.7from
jrkinley:branch-0.7

Conversation

@jrkinley
Copy link
Copy Markdown

@jrkinley jrkinley commented Mar 1, 2021

Exclude jackson-databind from shaded .jar as it clashes with Spark's version in Cloudera Runtime 7.1.x. Looking at the dependency tree Spark includes jackson-databind:2.6.5 whereas Envelope 0.7.2 is including jackson-databind:2.3.1 as part of a kitesdk dependency.

@jrkinley
Copy link
Copy Markdown
Author

jrkinley commented Mar 1, 2021

Spark throws this exception when both versions of the jar are on the classpath:

Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;
	at org.apache.hadoop.hive.common.StatsSetupConst$ColumnStatsAccurate.<clinit>(StatsSetupConst.java:176)
	at org.apache.hadoop.hive.common.StatsSetupConst.parseStatsAcc(StatsSetupConst.java:353)
	at org.apache.hadoop.hive.common.StatsSetupConst.setBasicStatsState(StatsSetupConst.java:241)
	at org.apache.hadoop.hive.common.StatsSetupConst.setStatsStateForCreateTable(StatsSetupConst.java:345)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:2443)
	at org.apache.hadoop.hive.ql.metadata.Hive.lambda$loadDynamicPartitions$4(Hive.java:2909)
	... 4 more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants