How do I update or write to a file system on Hadoop?… here is a solution to the problem.
How do I update or write to a file system on Hadoop?
So I’m using a filesystem on Hadoop and I need to make some updates to that filesystem using paths. In this file system, I have different txt format files. I can access these files using the path. Below I think I can use BufferedReader to read a text file on Hadoop, but it hasn’t been tested yet.
fs = FileSystem.get(URI.create(path), conf);
Path pt = new Path(out + "/" + name + ".txt");
BufferedReader each_br = new BufferedReader(new InputStreamReader(fs.open(pt)));
I want to write a specific file using the path. I can’t figure it out. Any ideas?
Thanks,
Solution
You’ve memorized your reading, and writing is just as easy.
fs = FileSystem.get(URI.create(path), conf);
FSDataOutputStream out = fs.create(new Path("myOutFile"));
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));
This will provide you with a standard java BufferedWriter. From there you should be able to do whatever you want.