pig udf piggeolocationudf不工作

koaltpgm  于 2021-06-04  发布在  Hadoop
关注(0)|答案(0)|浏览(128)

有类似的问题,我的问题上的stackoverflow,但他们没有帮助我,特别是。。。
我从“hadoop in practice”中复制了一些代码,并添加了一些代码(getcachefiles)来编写piggeolocationudf。。。但是,当我将代码打包到jar并在gruntshell(mapreduce模式)中执行它时,这就行不通了
有人能帮我吗?我的失败在哪里?

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.DataType;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.logicalLayer.schema.Schema;
import com.maxmind.geoip.LookupService;

public class PigGeolocationUDF extends EvalFunc<String> {
    private LookupService geoloc;
    private static final String COUNTRY = "country";

    public List<String> getCacheFiles() {
        List<String> list = new ArrayList<String>(1);
        list.add("/tmp/GeoLiteCity.dat#GeoLiteCity.dat");
        return list;
    }

    public String exec(Tuple input) throws IOException {

        if (input == null || input.size() == 0) {
            return null;
        }

        Object object = input.get(0);
        if (object == null) {
            return null;
        }

        String ip = (String) object;

        return lookup(ip);
    }

    protected String lookup(String ip) throws IOException {

        if (geoloc == null) {

            geoloc = new LookupService("GeoLiteCity.dat", LookupService.GEOIP_MEMORY_CACHE);

        }

        String country = geoloc.getLocation(ip).countryCode + "\t"
                + geoloc.getLocation(ip).countryName + "\t"
                + geoloc.getLocation(ip).region + "\t"
                + geoloc.getLocation(ip).postalCode + "\t"
                + geoloc.getLocation(ip).city;

        if ("N/A".equals(country)) {
            return null;
        }

        return country;

    }

    @Override
    public Schema outputSchema(Schema input) {
        return new Schema(new Schema.FieldSchema(COUNTRY, DataType.CHARARRAY));
    }

}

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题