Java – Host and port used to list directories in hdfs

First, I used the HortonWorks Sandbox as a Hadoop dist with no custom configuration at all.

Once connected to the sandbox, I can list the files for the HDFS directory:

[[email protected] ~]# hadoop fs -ls hdfs:///user/guest

But if I try to specify the host and port, I only get the error message:

[[email protected] ~]# hadoop fs -ls hdfs://localhost:8020/user/guest
ls: calls from sandbox.hortonworks.com/10.0.2.15 to localhost:8020 fail on connection exception: java.net.ConnectException: Connexion refusée; For more information, see: http://wiki.apache.org/hadoop/ConnectionRefused

[[email protected] ~]# hadoop fs -ls hdfs://localhost:9000/user/guest
ls: calls from sandbox.hortonworks.com/10.0.2.15 to localhost:9000 fail on connection exception: java.net.ConnectException: Connexion refusée; For more information, see: http://wiki.apache.org/hadoop/ConnectionRefused

Once I know the correct host and port to use, I can use them in my Java call:

Path pt = new Path("hdfs://host:port/user/guest/test-text-file.txt");

Check the value of the property fs.defaultFS in core-site.xml, which contains the IP address/hostname and port on which the NameNode daemon binds (bind) started.

I see you’re using the hortonworks sandbox, which is a property in core-site.xml which is located .xml /etc/hadoop/conf/core-site

<property>
  <name>fs.defaultFS</name>
  <value>hdfs://sandbox.hortonworks.com:8020</value>
</property>

So, you can try something like this:

hadoop fs -ls hdfs://sandbox.hortonworks.com:8020/user/guest 

Or you can also replace the IP address of sandbox.hortonworks.com from the corresponding entry in /etc/hosts and look like this on my virtual machine:

127.0.0.1       localhost.localdomain localhost
192.168.1.3 sandbox.hortonworks.com sandbox

So, I can try this too:

hadoop fs -ls hdfs://192.168.1.3:8020/user/guest