Host and port used to list directories in hdfs
First, I used the HortonWorks Sandbox as a Hadoop dist with no custom configuration at all.
Once connected to the sandbox, I can list the files for the HDFS directory:
[[email protected] ~]# hadoop fs -ls hdfs:///user/guest
But if I try to specify the host and port, I only get the error message:
[[email protected] ~]# hadoop fs -ls hdfs://localhost:8020/user/guest
ls: calls from sandbox.hortonworks.com/10.0.2.15 to localhost:8020 fail on connection exception: java.net.ConnectException: Connexion refusée; For more information, see: http://wiki.apache.org/hadoop/ConnectionRefused
[[email protected] ~]# hadoop fs -ls hdfs://localhost:9000/user/guest
ls: calls from sandbox.hortonworks.com/10.0.2.15 to localhost:9000 fail on connection exception: java.net.ConnectException: Connexion refusée; For more information, see: http://wiki.apache.org/hadoop/ConnectionRefused
Once I know the correct host and port to use, I can use them in my Java call:
Path pt = new Path("hdfs://host:port/user/guest/test-text-file.txt");
Solution
Check the value of the property fs.defaultFS
in core-site.xml
, which contains the IP address/hostname and port on which the NameNode daemon binds (bind) started.
I see you’re using the hortonworks sandbox, which is a property in core-site.xml which is located .xml /etc/hadoop/conf/core-site
<property>
<name>fs.defaultFS</name>
<value>hdfs://sandbox.hortonworks.com:8020</value>
</property>
So, you can try something like this:
hadoop fs -ls hdfs://sandbox.hortonworks.com:8020/user/guest
Or you can also replace the IP address of sandbox.hortonworks.com
from the corresponding entry in /etc/hosts
and look like this on my virtual machine:
127.0.0.1 localhost.localdomain localhost
192.168.1.3 sandbox.hortonworks.com sandbox
So, I can try this too:
hadoop fs -ls hdfs://192.168.1.3:8020/user/guest