Is Java – hadoop DistributedFileSystem thread-safe?

hadoop DistributedFileSystem thread-safe?… here is a solution to the problem.

hadoop DistributedFileSystem thread-safe?

I’m using Hadoop to write to the data I grab.

I have a spring service called from multiple threads to write something to HDFS.

@Service
public class WriteService
{
    public void write(String path, String content)
    {
        FileSystem fs = FileSystem.get(conf);
    }
}

I’m not sure if a FileSystem object can be a member of WriteService, and I haven’t found out if it’s thread-safe.
I’m using the DistributedFileSystem object.

Do you know if it’s thread-safe? Can I use it as a member of my service?

Thanks

Solution

Hadoop DFS uses the so-called WORM-Model.

However, to answer this question, it is generally not safe. You still need to consider concurrency control requirements.

Related Problems and Solutions