Java – Set the space quota for HDFS in Java

Set the space quota for HDFS in Java… here is a solution to the problem.

Set the space quota for HDFS in Java

I’m trying to set a space quota for HDFS using the Hadoop Java API, however, I can only find the getSpaceQuota method in the ContentSummary class. Basically, I want the code to implement the same functionality as the sudo -u hdfs hdfs dfsadmin -setSpaceQuota 1k/quotasdir command. Is there an API for setting space quota methods? Or what’s a better idea? Thanks in advance.

Solution

In the Apache Hadoop repository, the code for the hdfs dfsadmin -setSpaceQuota command is in DFSAdmin in class. If you read through the code, you’ll see that it ended up delegated to DistributedFileSystem#setQuota。 Method. The method performs RPC on NameNode to modify the quota.

If you want to reimplement this functionality in your Java program, you may need to get an instance of FileSystem, downcast it to DistributedFileSystem, and then call DistributedFileSystem#setQuota.

Note that Apache Hadoop does not provide strong backward compatibility guarantees for DistributedFileSystem classes. This means that after upgrading to a new Hadoop version, your code may break. The class is annotated as LimitedPrivate and Unstable. Apache Hadoop Compatibility The documentation describes the meaning of these comments in detail. There is currently no guarantee that a public, stable API can be used to change quotas from a custom program.

Related Problems and Solutions