Java – Unable to load org.apache.hadoop.util.ShutdownHookManager when shutting down the tomcat server

I have a simple web application running on tomcat. This web application reads and writes files to HDFS.

The problem I’m facing is every time I stop the server with ./bin/shutdown.sh. I get a failed to load hadoop shutdownHookManager exception.

I’m sure hadoop-common (including ShutDownManager) is in the tomcat classpath.

Can anyone help me?

The exception I get :

Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [com.ebay.kernel.calwrapper.CalTransactionHelper$1] (value [com.ebay.kernel.calwrapper.CalTransactionHelper$1 @3d5a1c5f]) and a value of type [com.ebay.kernel.calwrapper.CalTransactionHelper.Stack] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop 
INFO: Stopping ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load org.apache.hadoop.util.ShutdownHookManager$2.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Exception in thread "Thread-9" java.lang.NoClassDefFoundError: org/apache/hadoop/util/ShutdownHookManager$2
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.ShutdownHookManager$2
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1714)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  ... 2 more

I’m having the same issue in the Maven Mojo I built. I think the problem is caused by the context class loader that loads the hadoop class in your web application. The WebAppClassLoader classloader does not see these definitions. When a JVM shutdown occurs, the shutdown hook is being executed and the shutdownHookManager is being called, but it is loaded into the now destroyed context classloader, and that’s the problem.

Putting the hadoop jar in Tomcat’s lib directory may help. Or set some *_CLASSPATH Java options when starting Tomcat. This way, WebAppClassLoader will have the definition of your class.

Otherwise, you can manually perform Hadoop’s close hook and ignore the above exception before destroying the context.