Set the space quota for HDFS in Java
I’m trying to set a space quota for HDFS using the Hadoop Java API, however, I can only find the getSpaceQuota method in the ContentSummary class. Basically, I want the code to implement the same functionality as the sudo -u hdfs hdfs dfsadmin -setSpaceQuota 1k/quotasdir
command. Is there an API for setting space quota methods? Or what’s a better idea? Thanks in advance.
Solution
In the Apache Hadoop repository, the code for the hdfs dfsadmin -setSpaceQuota
command is in DFSAdmin
in class. If you read through the code, you’ll see that it ended up delegated to DistributedFileSystem#setQuota
。 Method. The method performs RPC on NameNode to modify the quota.
If you want to reimplement this functionality in your Java program, you may need to get an instance of FileSystem
, downcast it to DistributedFileSystem, and then call DistributedFileSystem#setQuota
.
Note that Apache Hadoop does not provide strong backward compatibility guarantees for DistributedFileSystem
classes. This means that after upgrading to a new Hadoop version, your code may break. The class is annotated as LimitedPrivate
and Unstable
. Apache Hadoop Compatibility The documentation describes the meaning of these comments in detail. There is currently no guarantee that a public, stable API can be used to change quotas from a custom program.