Java – How do I update or write to a file system on Hadoop?

How do I update or write to a file system on Hadoop?… here is a solution to the problem.

How do I update or write to a file system on Hadoop?

So I’m using a filesystem on Hadoop and I need to make some updates to that filesystem using paths. In this file system, I have different txt format files. I can access these files using the path. Below I think I can use BufferedReader to read a text file on Hadoop, but it hasn’t been tested yet.

fs = FileSystem.get(URI.create(path), conf);
Path pt = new Path(out + "/" + name  + ".txt");
BufferedReader each_br = new BufferedReader(new InputStreamReader(fs.open(pt)));

I want to write a specific file using the path. I can’t figure it out. Any ideas?

Thanks,

Solution

You’ve memorized your reading, and writing is just as easy.

fs = FileSystem.get(URI.create(path), conf);
FSDataOutputStream out = fs.create(new Path("myOutFile"));
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(out));

This will provide you with a standard java BufferedWriter. From there you should be able to do whatever you want.

Related Problems and Solutions