随机林模型加载错误

nwlqm0z1  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(234)

我的spark程序从elasticsearch读取数据(以javapairdd>的形式)。这个javapairdd使用以下内部类转换为javardd

static class Transformer implements
        Function<Tuple2<String, Map<String, Object>>, LabeledPoint> {

    @Override
    public LabeledPoint call(Tuple2<String, Map<String, Object>> arg0)
            throws Exception {
        HashingTF tf = new HashingTF();
        Map<String, Object> map = (Map<String, Object>) arg0._2();

        // get values from Map
        Set<String> keys = map.keySet();
        List<Object> valuesList = new ArrayList<Object>();
        for (Iterator<String> i = keys.iterator(); i.hasNext();) {
            String key = (String) i.next();
            Object value = (Object) map.get(key);
            valuesList.add(value);
        }
        return new LabeledPoint(1d, tf.transform(valuesList));
    }
}

使用javardd数据生成随机森林模型并保存到同一台机器上的特定位置。模型保存成功。
但在加载模型时,我遇到了以下异常,
sun.reflect.nativemethodaccessorimpl.invoke0(本机方法)处的java.lang.reflect.invokeTargetException at sun.reflect.nativemethodaccessorimpl.invoke(本机方法)。java:62)在sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl。java:43)在java.lang.reflect.method.invoke(方法。java:497)在org.codehaus.mojo.exec.execjavamojo$1.run(execjavamojo。java:293)在java.lang.thread.run(线程。java:745)原因:org.json4s.package$mappingexception:在org.json4s.reflect.package$.fail(package)中未找到可转换为java.lang.string的值。scala:96)在org.json4s.extraction$.convert(extraction。scala:554)在org.json4s.extraction$.extract(提取。scala:331)在org.json4s.extraction$.extract(extraction。scala:42)在org.json4s.extractablejsonastnode.extract(extractablejsonastnode。scala:21)在org.apache.spark.mllib.tree.model.decisiontreemodel$.load(decisiontreemodel。scala:326)在org.apache.spark.mllib.tree.model.decisiontreemodel.load(decisiontreemodel.scala)位于co.nttd.integration.sparkesintegration.main(sparkesintegration)。java:96)
为了保存模型,编写了以下代码

JavaPairRDD<String, Map<String, Object>> esRDD = JavaEsSpark
            .esRDD(javaSparkContext);
    // Transform data in JavaRDD<LabeledPoint>
    JavaRDD<LabeledPoint> data = esRDD.map(new Transformer());

    final RandomForestModel model = RandomForest.trainClassifier(
                inputTrainingData, numClasses, categoricalFeaturesInfo, numTrees,
                featureSubsetStrategy, impurity, maxDepth, maxBins, seed);

    model.save(JavaSparkContext.toSparkContext(jsc),
            "myPath");

要加载模型,decisiontreemodel model=decisiontreemodel.load(sc.sc(),“file:///mypath”);

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题