Java – pig : Hadoop jobs Fail

I have a pig script that queries data from a csv file.

The script has been tested locally with small and large .csv files.

In a small cluster:
It starts with processing the script and fails after 40% of the calls are complete

The error is just,
Unable to read data from File Path

My inference is that the script can read the file, but the connection is dropped and the message is lost

But I only get the above error.

The answer to the general question is to change the error level in the configuration file to add these two lines to the mapred-site .xml

log4j.logger.org.apache.hadoop = error,A 
log4j.logger.org.apache.pig= error,A

In my case, it’s an OutOfMemory exception