Java – NoSuchMethodError uses Builder to write Avro objects to HDFS

This exception occurred while writing an object to HDFS:

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.avro.Schema$Parser.parse(Ljava/lang/String; [Ljava/lang/String;)Lorg/apache/avro/Schema;
        at com.blah.SomeType.<clinit>(SomeType.java:10)

The line it references in the generated code looks like this:

public class SomeType extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse ..........

The call in my code looks like this:

val someTypeBuilder = SomeType.newBuilder()
      .setSomeField("some value")
       set a whole load of other fields

Calling newBuilder() in the test code is perfectly fine.

jar is running on HDFS nodes using hadoop jar commands.

Any idea what could be wrong here?

org.apache.avroSchema.Parser.parse(String s, String… more) was not introduced until Avro 1.7.5. Note that it appears in 1.7.4 Schema.Parser documentation does not exist. Update your HDFS node to use a newer version of Avro, or something like add the string itself like the current 1.7 version of Schema. Do and use the method of taking a single string:

public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse(String.join("", prevArg1, prevArg2));

Edit:
I’ve never tried upgrading Avro on a Hadoop installation, but I guess if it’s a minor version upgrade, you might be able to find jars and replace them manually.

~/dev/hadoop/hadoop-2.7.1$ find . -name '*avro*.jar'
./share/hadoop/tools/lib/avro-1.7.4.jar
./share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/avro-1.7.4.jar
./share/hadoop/common/lib/avro-1.7.4.jar
./share/hadoop/mapreduce/lib/avro-1.7.4.jar
./share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/avro-1.7.4.jar

Try downloading Avro 1.7.7 and replace all found jars with downloaded jars. I also back up my Hadoop installation in case something goes wrong.