Java – Hadoop sequence files cannot be created on the local file system

I found this example how to write to the local file system, but it throws this exception:

Exception in thread "main" java.io.IOException: (null) entry in command string: null chmod 0644 C:\temp\test.seq
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:770)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:866)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:849)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:733)
    at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:225)
    at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209)
    at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307)
    at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:296)
    at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:328)
    at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:398)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:778)
    at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1168)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)

Run this program on a Windows 10 machine. I even tried using the msys git bash shell, thinking it might help the JVM emulate chmod operations. Nothing has changed. Any suggestions on how to do this on Windows?

I also got this error, which was solved after following the steps. (Note: I’m using Spark 2.0.2 and Hadoop 2.7).

1. Verify that you encounter “java.io.IOException: Unable to find executable null\bin\winutils.exe in Hadoop binaries. ”。 You can check it by running the “spark-shell” command.
2. I got the above error. This is because I didn’t add “HADOOP_HOME” to the environment variables. After adding “HADOOP_HOME”, same as “SPARK_HOME” in my case, the problem is solved.