Java – How to build Hadoop source code under Windows?

How to build Hadoop source code under Windows?… here is a solution to the problem.

How to build Hadoop source code under Windows?

Try building Hadoop from a source under Windows 7 x64. According to the instructions Hadoop2OnWindows and BUILDING

I cloned the Hadoop source code from git, checkout to origin/branch-2.5 (SHA-1: fa3bb675a728105d69614f53abe4339958550adf)
Then from the Windows console I run:

  1. Set platform=x64
  2. Fresh install -Pdist, native-win -DskipTests -Dtar

and get error – [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.5.0-SNAPSHOT:protoc (compile-protoc) on project hadoop-common: org.apache . maven.plugin.MojoExecutionException: 'protoc --version' does not return version -> [help 1].

Is there any solution?

Solution

The first thing Hadoop-Maven-Plugins’ Protoc Mojo does is it uses Java’s java.lang.Process class to try to execute the following command:

protoc --version

Basically, it does this to check if the version of the Protocol Buffer compiler (protoc) on your system matches the version of the protobuf JAR.

This has 4 results:

  • It could not find the protoc command to execute (it received an exit value of 127).
  • It does not receive an exit value of 127, but it also does not return a version value
  • A version is returned, but it doesn’t match the version of Protobuf jar
  • Returns a version that does match protobuf jar

You have the second problem.

Does it work if you try to run protoc --version from the command line?

As they state on their website, if you have multiple versions of Protoc on your system, you can set the HADOOP_PROTOC_PATH environment variable in your build shell to point to the one you want to use for your Hadoop build. If you do not define this environment variable, you look for protoc in PATH. Is it available on your PATH?

If you do not already have the Protocol Buffer compiler installed, you can download it from the following location:
https://developers.google.com/protocol-buffers/docs/downloads

Related Problems and Solutions