Hive3中orc包升级 报错Error: java.lang.ClassNotFoundException: org.threeten.extra.chrono.AbstractDate
·
背景:Hive3.1.3升级orc版本为了支持zstd,orc版本从1.5.x -> 1.6.x
解决:加上jar依赖 add jar hdfs://xxx/xxx/threeten-extra-1.7.0.jar;
原因:原因是ORC 为了处理混合历法/闰秒等边界情况,使用了 ThreeTen-Extra 的 HybridChronology,进而需要 org.threeten.extra.chrono.AbstractDate。MR 任务容器里没有该类,于是抛出 ClassNotFoundException
报错原文:
Error: java.lang.ClassNotFoundException: org.threeten.extra.chrono.AbstractDate
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.threeten.extra.chrono.HybridChronology.date(HybridChronology.java:346)
at org.threeten.extra.chrono.HybridChronology.date(HybridChronology.java:88)
at java.time.format.DateTimePrintContext.adjust(DateTimePrintContext.java:164)
at java.time.format.DateTimePrintContext.<init>(DateTimePrintContext.java:119)
at java.time.format.DateTimeFormatter.formatTo(DateTimeFormatter.java:1744)
at java.time.format.DateTimeFormatter.format(DateTimeFormatter.java:1720)
at java.time.ZonedDateTime.format(ZonedDateTime.java:2143)
at org.apache.orc.impl.ConvertTreeReaderFactory$StringGroupFromTimestampTreeReader.setConvertVectorElement(ConvertTreeReaderFactory.java:1241)
at org.apache.orc.impl.ConvertTreeReaderFactory$ConvertTreeReader.convertVector(ConvertTreeReaderFactory.java:307)
at org.apache.orc.impl.ConvertTreeReaderFactory$StringGroupFromTimestampTreeReader.nextVector(ConvertTreeReaderFactory.java:1258)
at org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextBatch(TreeReaderFactory.java:2047)
at org.apache.orc.impl.RecordReaderImpl.nextBatch(RecordReaderImpl.java:1219)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.ensureBatch(RecordReaderImpl.java:87)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.hasNext(RecordReaderImpl.java:103)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:253)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:228)
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360)
at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:167)
at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:52)
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229)
at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:142)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:206)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:192)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:52)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:466)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:350)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
更多推荐

所有评论(0)