java—hadoop中用于多个双值的自定义可写类

我试图发出4个数字值作为键。我为同一个类编写了定制的可写可比类，但我仍然坚持使用compare（）方法stackoverflow站点中提到了几种解决方案。但这并没有解决我的问题。
我的可写类是

public class DimensionWritable implements WritableComparable {
    private double keyRow;
    private double keyCol;

    private double valRow;
    private double valCol;

    public  DimensionWritable(double keyRow, double keyCol,double valRow, double valCol) {
        set(keyRow, keyCol,valRow,valCol);
    }
    public void set(double keyRow, double keyCol,double valRow, double valCol) {
        //row dimension
        this.keyRow = keyRow;
        this.keyCol = keyCol;
        //column dimension
        this.valRow = valRow;
        this.valCol = valCol;
    }

    @Override
    public void write(DataOutput out) throws IOException {
        out.writeDouble(keyRow);
        out.writeDouble(keyCol);

        out.writeDouble(valRow);
        out.writeDouble(valCol);
    }
    @Override
    public void readFields(DataInput in) throws IOException {
        keyRow = in.readDouble();
        keyCol = in.readDouble();

        valRow = in.readDouble();
        valCol = in.readDouble();
    }
    /**
     * @return the keyRow
     */
    public double getKeyRow() {
        return keyRow;
    }
    /**
     * @param keyRow the keyRow to set
     */
    public void setKeyRow(double keyRow) {
        this.keyRow = keyRow;
    }
    /**
     * @return the keyCol
     */
    public double getKeyCol() {
        return keyCol;
    }
    /**
     * @param keyCol the keyCol to set
     */
    public void setKeyCol(double keyCol) {
        this.keyCol = keyCol;
    }
    /**
     * @return the valRow
     */
    public double getValRow() {
        return valRow;
    }
    /**
     * @param valRow the valRow to set
     */
    public void setValRow(double valRow) {
        this.valRow = valRow;
    }
    /**
     * @return the valCol
     */
    public double getValCol() {
        return valCol;
    }
    /**
     * @param valCol the valCol to set
     */
    public void setValCol(double valCol) {
        this.valCol = valCol;
    }

    //compare - confusing

}

例如，compare语句背后的逻辑是什么？它是hadoop中的密钥交换，对吗？
如何实现上述4个双值相同。
更新我编辑了我的代码为“isnot2bad”说，但显示

java.lang.Exception: java.lang.RuntimeException: java.lang.NoSuchMethodException: edu.am.bigdata.svmmodel.DimensionWritable.<init>()
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: edu.am.bigdata.svmmodel.DimensionWritable.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
    at org.apache.hadoop.io.WritableComparator.newKey(WritableComparator.java:113)
    at org.apache.hadoop.io.WritableComparator.<init>(WritableComparator.java:99)
    at org.apache.hadoop.io.WritableComparator.get(WritableComparator.java:55)
    at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:819)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:836)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:376)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:85)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:584)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:656)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NoSuchMethodException: edu.am.bigdata.svmmodel.DimensionWritable.<init>()
    at java.lang.Class.getConstructor0(Class.java:2721)
    at java.lang.Class.getDeclaredConstructor(Class.java:2002)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)

我做错什么了吗？

如果您想在hadoop中使用您的类型作为键，它必须是可比较的（您的类型必须是完全有序的），即两个示例 a 以及 b 的 DimensionWritable 必须相等，或 a 必须大于或小于 b （无论这意味着什么，都取决于执行情况）。
通过实施 compareTo 您可以定义示例如何自然地相互比较。这是通过比较要比较的示例的字段来完成的：

public int compareTo(DimensionWritable o) { 
    int c = Double.compare(this.keyRow, o.keyRow);
    if (c != 0) return c;
    c = Double.compare(this.keyCol, o.keyCol);
    if (c != 0) return c;
    c = Double.compare(this.valRow, o.valRow);
    if (c != 0) return c;
    c = Double.compare(this.valCol, o.valCol);
    return c;
}

请注意 hashCode 还必须实施，因为它必须符合你对平等的定义（根据法律，两个示例被认为是平等的） compareTo hadoop要求密钥的哈希代码在不同的jvm之间保持不变。所以我们再次使用这些字段来计算散列码：

public int hashCode() {
    final int prime = 31;
    int result = 1;
    result = prime * result + Double.hashCode(keyRow);
    result = prime * result + Double.hashCode(keyCol);
    result = prime * result + Double.hashCode(valRow);
    result = prime * result + Double.hashCode(valCol);
    return result;
}

java—hadoop中用于多个双值的自定义可写类

1条答案

相关问题

热门标签

最新问答