Pig 0.17.0运行报错ERROR 2998：缺失org/python/google/common/collect/Lists类

Problem Statement #

I'm new to Apache Pig and encountered multiple issues while running my first program. I've tried the following two Stack Overflow solutions without success:

• ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org/apache/hadoop/hbase/filter/WritableByteArrayComparable
• Pig Installation Error: ERROR pig.Main: ERROR 2998: Unhandled internal error

My Pig version is 0.17.0, and the error stack trace is as follows:

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. org/python/google/common/collect/Lists

java.lang.NoClassDefFoundError: org/python/google/common/collect/Lists
    at org.apache.pig.tools.pigstats.mapreduce.MRJobStats.getTaskReports(MRJobStats.java:533)
    at org.apache.pig.tools.pigstats.mapreduce.MRJobStats.addMapReduceStatistics(MRJobStats.java:355)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.addSuccessJobStats(MRPigStatsUtil.java:232)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.accumulateStats(MRPigStatsUtil.java:164)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:379)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:290)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1475)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1460)
    at org.apache.pig.PigServer.storeEx(PigServer.java:1119)
    at org.apache.pig.PigServer.store(PigServer.java:1082)
    at org.apache.pig.PigServer.openIterator(PigServer.java:995)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:782)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:383)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
    at org.apache.pig.Main.run(Main.java:630)
    at org.apache.pig.Main.main(Main.java:175)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: java.lang.ClassNotFoundException: org.python.google.common.collect.Lists
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 24 more
================================================================================

Solutions #

This error stems from a missing or unresolvable Guava dependency—org.python.google.common.collect.Lists is a class from the Guava library, which Pig 0.17.0 depends on. Follow these steps to resolve it:

1. Check for Guava Jar in Pig's Lib Directory #

• Navigate to your Pig installation's lib folder (usually $PIG_HOME/lib). Look for a guava-*.jar file. Pig 0.17.0 requires Guava 11.0.2 or a compatible version.
• If the jar is missing, download the correct version and place it in the lib directory.

2. Add Guava to PIG_CLASSPATH #

• If the jar exists but isn't being loaded, explicitly include it in Pig's classpath:

  export PIG_CLASSPATH=$PIG_HOME/lib/guava-11.0.2.jar:$PIG_CLASSPATH
  

• Alternatively, specify the classpath when launching Pig:

  pig -cp $PIG_HOME/lib/guava-11.0.2.jar
  

3. Verify Hadoop Compatibility #

• Pig 0.17.0 is built to work with Hadoop 2.7.x and newer. Confirm your Hadoop version meets this requirement—older versions can trigger dependency conflicts.

4. Reinstall Pig (If Corruption is Suspected) #

• If you think installation files are corrupted, re-extract the official Pig 0.17.0 binary package. Avoid manual modifications to the lib directory unless necessary, as this can break dependency chains.

5. Validate Python Environment (For UDF Users) #

• If you're using Python UDFs, ensure your Python environment is properly configured. While this error is primarily Java-related, misconfigured Python setups can occasionally cause secondary dependency issues.

Content sourced from Stack Exchange, question author: Siva Jagadesh