Hive UDTF returns the ArrayList column… here is a solution to the problem.
Hive UDTF returns the ArrayList column
I am new to Hive UDTF. I have a requirement that I have to pass the string value as a Paratmeter in the UDTF, and the returned Column should be an ArrayList.
I wrote the following code:
public StructObjectInspector initialize(ObjectInspector[] arg0)
throws UDFArgumentException {
ArrayList<String> fieldNames = new ArrayList<String>();
ArrayList<ObjectInspector> fieldOIs = new ArrayList<ObjectInspector>();
fieldNames.add("col1");
stringOI = (PrimitiveObjectInspector) arg0[0];
listOi=(ListObjectInspector) arg0[0];
fieldOIs.add(listOi.getListElementObjectInspector());
return ObjectInspectorFactory.getStandardStructObjectInspector(fieldNames, fieldOIs);
}
@Override
public void process(Object[] record) throws HiveException {
TODO Auto-generated method stub
String document = (String) stringOI.getPrimitiveJavaObject(record[0]);
if (document == null) {
return;
}
firstColumn=(String) stringOI.getPrimitiveJavaObject(record[0]);
secondColumn=(String) stringOI.getPrimitiveJavaObject(record[1]);
if(outputMapper.containsKey(firstColumn))
{
ArrayList<String> tempList=new ArrayList<String>();
tempList=outputMapper.get(firstColumn);
tempList.add(secondColumn);
outputMapper.put(firstColumn,tempList);
}
else
{
childVendorList=new ArrayList<String>();
childVendorList.add(secondColumn);
outputMapper.put(firstColumn,childVendorList);
}
forward(outputMapper.get(firstColumn));
}
I get the following exception:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyStringObjectInspector cannot be converted to org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector
Who can help???
Solution
listOi=(ListObjectInspector) arg0[0];
fieldOIs.add(listOi.getListElementObjectInspector());
return ObjectInspectorFactory.getStandardStructObjectInspector(fieldNames, fieldOIs);
This arg0[0] is a primitive object inspector. Using listOi.getListElementObjectInspector() just gives you a similar PrimitiveObjectInspector (like String, Integer is not List). It should be
fieldOIs.add(ObjectInspectorFactory.getStandardListObjectInspector(stringOI ))
This specifies the output column with a list of stringOI types.