hive udf-java字符串异常

qkf9rpyu  于 2021-05-29  发布在  Hadoop
关注(0)|答案(2)|浏览(329)

我已经编写了udf来解码cookie并返回字符串列表。不幸的是,我在处理时遇到了配置单元运行时错误
这是我的密码:

@Override
public ObjectInspector initialize(ObjectInspector[] input) throws UDFArgumentException {

    ObjectInspector cookieContent = input[0];
    if (!(isStringOI(cookieContent))){
        throw new UDFArgumentException("only string");
    }
    this.cookieValue = (StringObjectInspector) cookieContent;
    return ObjectInspectorFactory.getStandardListObjectInspector
            (PrimitiveObjectInspectorFactory.javaStringObjectInspector);
}

public Object evaluate(DeferredObject[] input) throws HiveException {

    String encoded = cookieValue.getPrimitiveJavaObject(input[0].get());
    try {
        result = decode(encoded);
    } catch (CodeException e) {
        throw new UDFArgumentException();
    }

    return result;
}
public List<String> decode(String encoded) throws CodeException {

    decodedBase64 = Base64.decodeBase64(encoded);
    String decompressedArray = new String(getKadrs(decodedBase64));
    String kadr= decompressedArray.substring(decompressedArray.indexOf("|") + 1);
    List<String> kadrsList= new ArrayList(Arrays.asList(kadr.split(",")));
    return kadrsList;
}

private byte[] getKadrs(byte[] compressed) throws CodeException {
    Inflater decompressor = new Inflater();
    decompressor.setInput(compressed);
    ByteArrayOutputStream outPutStream = new ByteArrayOutputStream(compressed.length);
    byte temp [] = new byte[1024];
    while (!decompressor.finished()) {
        try {
            int count = decompressor.inflate(temp);
            outPutStream.write(temp, 0, count);
        }
        catch (DataFormatException e) {
            throw new CodeException ("Wrong data format", e);
        }
    }
    try {
        outPutStream.close();
    } catch (IOException e) {
        throw new CodeException ("Cant close outPutStream ", e);
    }
    return outPutStream.toByteArray();
}

其结果是:
“kadr1,kadr20,kadr35,kadr12”。单元测试工作得很好,但当我尝试在hive中使用此函数时,我得到了以下结果:

Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.hadoop.io.Text
  at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableStringObjectInspector.getPrimitiveWritableObject(WritableStringObjectInspector.java:41)

我很难调试,因为其他人必须实现我的jar才能看到结果,所以所有的建议将不胜感激。

wvt8vs2t

wvt8vs2t1#

你的 evaluate 方法当前返回 String ,它不是hadoop数据类型。你应该把你的绳子包起来 Text 反对说 return new Text(result) .

mw3dktmi

mw3dktmi2#

拉文德拉是对的
我有个朋友
返回objectinspectorfactory.getstandardlistobjectinspector(primitiveobjectinspectorfactory.writablestringobjectinspector);
writablestringobjectinspector返回文本
我把它改成了javastringobjectinspector,它返回字符串,一切正常谢谢

相关问题