Run the Hadoop Pi sample
I need some help running the Hadoop Pi sample in Windows. I’ve successfully set up Hadoop on Windows and can see all NameNode, SecondaryNameNode, JobTracker, and TaskTracker running. I can also click on URL:
http://localhost:50070/, http://localhost:50030 and http://localhost:50090/.
However, when I try to run the built-in Pi sample, I get the following exception.
$ hadoop jar hadoop/hadoop-examples-1.2.1.jar pi 4 100
Not a valid JAR: C:\cygwin\home\chandeln\hadoop\hadoop-examples-1.2.1.jar
I also set up both Hadoop and Java in my path.
$ java -version
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
$ hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /C:/cygwin/usr/local/hadoop/hadoop-core-1.2.1.jar
My .bashrc looks like this:
export JAVA_HOME=/cygdrive/c/Java/jdk1.7.0_40
export HADOOP_PREFIX=/cygdrive/c/usr/local/hadoop
export PATH=$PATH:$HADOOP_PREFIX/bin
Can anyone shed light on this?
Solution
Although Hadoop is written in Java, it doesn’t work well on Windows. If you really want to get Hadoop up and running in the shortest possible time, I recommend Linux. You can download the VMWare Player application for free and install it on Windows. Then, download the latest version on Ubuntu and install it as a virtual machine using VMWare Player. You can then use your Ubuntu virtual machine to install a single node cluster of Hadoop .
If this seems like too much work, open your <a href=”http://aws.amazon.com/” rel=”noreferrer noopener nofollow” > your Amazon AWS account and find Pre-configured Hadoop machines work together.