The path to the file in hdfs… here is a solution to the problem.
The path to the file in hdfs
I want to read files from the Hadoop file system.
To get the correct file path, I need the hostname and port address of hdfs
.
So in the end my file path looks like
this
Path path = new Path("hdfs://123.23.12.4344:9000/user/filename.txt")
Now I want to know how to extract HostName = “123.23.12.4344” & port: 9000?
Basically, I
want to access the file system on Amazon EMR, however, when I use
FileSystem
fs = FileSystem.get(getConf());
I get
You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path
So I decided to use the URI. (I have to use the URI) but I’m not sure how to access the URI.
Solution
You can use either of these two methods to resolve your error.
1.
String infile = "file.txt";
Path ofile = new Path(infile);
FileSystem fs = ofile.getFileSystem(getConf());
2.
Configuration conf = getConf();
System.out.println("fs.default.name : - " + conf.get("fs.default.name"));
It prints uri as : hdfs://10.214.15.165:9000 or something
String uri = conf.get("fs.default.name");
FileSystem fs = FileSystem.get(uri,getConf());