如何在pig查询中展平嵌套的avro记录?

e5nqia27  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(203)

avro架构如下所示:

{
  "type" : "record",
  "name" : "name1",
  "fields" :
  [
    {
      "name" : "f1",
      "type" : "string"
    },
    {
      "name" : "f2",
      "type" :
      {
        "type" : "array",
        "items" :
        {
          "type" : "record",
          "name" : "name2",
          "fields" :
          [
            {
              "name" : "time",
              "type" : [ "float", "int", "double", "long" ]
            },
          ]
        }
      }
    }
  ]
}

在pig中阅读后:

grunt> A = load 'data' using AvroStorage();
grunt> DESCRIBE A;
A: {f1: chararray,f2: {ARRAY_ELEM: (time: (FLOAT: float,INT: int,DOUBLE: double,LONG: long))}}

我想要的可能是一袋 (f1:chararray, timestamp:double) . 我就是这么做的:

grunt> B = FOREACH A GENERATE f1, f2.time AS timestamp;
grunt> DESCRIBE B;
B: {f1: chararray,timestamp: {(time: (FLOAT: float,INT: int,DOUBLE: double,LONG: long))}}

那我该怎么把这张唱片压平呢?
我是新来的Pig,阿芙罗和不知道我想做什么,甚至是有意义的。谢谢你的帮助。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题