在hortonworks沙盒上使用pig

我一直试图在hortonworks提供的沙盒上使用currenttime（），但无法使其工作。
这就是我在Pig剧本里的全部内容：

<code>
REGISTER zookeeper.jar
REGISTER piggybank.jar
REGISTER hbase-common-0.98.4.2.2.0.0-2041-hadoop2.jar
REGISTER hbase-common-0.98.4.2.2.0.0-2041-hadoop2-tests.jar
REGISTER hbase-client-0.98.4.2.2.0.0-2041-hadoop2.jar
REGISTER guava.jar
a = CurrentTime();
dump a;
</code>

我在日志中看到的错误是：

15/04/13 21:23:22 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
15/04/13 21:23:22 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
15/04/13 21:23:22 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2015-04-13 21:23:22,428 [main] INFO  org.apache.pig.Main - Apache Pig version 0.14.0.2.2.0.0-2041 (rexported) compiled Nov 19 2014, 15:24:46
2015-04-13 21:23:22,429 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/hue/appcache/application_1428957295391_0006/container_1428957295391_0006_01_000002/pig_1428960202427.log
2015-04-13 21:23:24,041 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2015-04-13 21:23:24,615 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2015-04-13 21:23:27,864 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. <file script.pig, line 10> Cannot expand macro 'CurrentTime'. Reason: Macro must be defined before expansion.
Failed to parse: <file script.pig, line 10> Cannot expand macro 'CurrentTime'. Reason: Macro must be defined before expansion.
    at org.apache.pig.parser.PigMacro.macroInline(PigMacro.java:455)
    at org.apache.pig.parser.QueryParserDriver.inlineMacro(QueryParserDriver.java:301)
    at org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:290)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:183)
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
    at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
    at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:741)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:170)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
2015-04-13 21:23:27,871 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file script.pig, line 10> Cannot expand macro 'CurrentTime'. Reason: Macro must be defined before expansion.
Details at logfile: /hadoop/yarn/local/usercache/hue/appcache/application_1428957295391_0006/container_1428957295391_0006_01_000002/pig_1428960202427.log
2015-04-13 21:23:27,908 [main] INFO  org.apache.pig.Main - Pig script completed in 5 seconds and 684 milliseconds (5684 ms)

我包括了那些注册行，因为我想也许自定义项不包括在内。我不知道现在该怎么办

我认为你必须使用当前时间的完整路径方法。下面有一个例子，我将给定的时间转换为unix日期格式。

register '/usr/lib/pig/piggybank.jar' ;
DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();

date_filter = FOREACH parsed_log GENERATE ISOToUnix(date) AS unixTime:long;
 STORE date_filter INTO '/root/pig/output/parselogdate/';

我认为对于你的应用程序，你必须使用下面的，只要尝试一下，如果它工作。您的currenttime将以元组作为输入，并返回当前时间作为输出。

//Users is a file with tuples in it.
     a = load users;
     b = foreach a generate org.apache.pig.builtin.CurrentTime(); 
     dump b;

我尝试执行上面的命令并得到了输出。你可以找到java文档。

在hortonworks沙盒上使用pig

1条答案

相关问题

热门标签

最新问答