在Java中使用ANTLR和gradle解析Python代码

6jjcrrmo  于 5个月前  发布在  Java
关注(0)|答案(1)|浏览(74)

上下文

我正在尝试解析一些Python代码,并使用ANTLR在.py文件中找到一个函数。然而,我在理解如何在java中使用ANTLR时遇到了一些困难。

构建.分级

下面的build.gradle文件内容用于包含ANTLR:

plugins {
    ...

    // Allow using antler.
    id 'antlr'
}

dependencies {
...

    // Include support for ANTLR runtime dependency.
    implementation 'org.antlr:antlr4-runtime:4.13.1'

    }

字符串
this example 1的基础上,我加入了Python3Parser.g4Python3Lexer.g4文件,它们和下面的EditPython.java文件在同一个目录下。在this example 2的基础上,我加入了Python3LexerBase.javaPython3ParserBase.java(我相信当运行时,最终会产生Python3Lexer.g4Python3Parser.g4文件:

python transformGrammar.py
antlr4 -Dlanguage=Python3 -o gen *.g4

MWE(MWE)

这是一个试图使用Python3Parser对象的最小类。

import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.TerminalNode;

public class EditPython {

  public static void main(String[] args) {
    if (args.length == 0) {
      System.out.println("Please provide a Python code snippet as an argument.");
      return;
    }

    String pythonCode = args[0];
    modifyFunctionDocumentation(pythonCode);
  }

  public static void modifyFunctionDocumentation(String pythonCode) {
    // Parse Python code to get the tree
    Python3Lexer lexer = new Python3Lexer(CharStreams.fromString(pythonCode));
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    Python3Parser parser = new Python3Parser(tokens);
    ParseTree tree = parser.file_input();

    // Traverse the tree to find and modify documentation
    Python3Parser.File_inputContext ctx = parser.file_input();
    Python3Parser.StmtContext stmtContext = ctx.stmt(0); // Assuming it's the first statement
    if (stmtContext.funcdef() != null && stmtContext.funcdef().NAME().getText().equals("some_function")) {
      // Assuming docstring is represented as STRING() in AST
      Python3Parser.FuncdefContext funcDefContext = stmtContext.funcdef();
      TerminalNode stringNode = funcDefContext.STRING();
      if (stringNode != null) {
        String docstring = stringNode.getText();
        System.out.println("Found the function: some_function");
        System.out.println("Docstring: " + docstring);
      } else {
        System.out.println("Function some_function found, but no docstring.");
      }
    } else {
      System.out.println("Function some_function not found in the provided code.");
    }
  }
}


问题
然而,当我编译它时,它会传回:

^
EditPython.java:20: error: cannot find symbol
    Python3Lexer lexer = new Python3Lexer(CharStreams.fromString(pythonCode));
    ^


因为我没有导入Python3Lexer。但是,当我导入com.someproject.somefolder.Python3Lexer时,它会说:

EditPython.java:5: error: cannot find symbol
import com.someproject.somefolder.Python3Lexer;
                                  ^


因为Python3Lexer.g4文件不是Python3Lexer.java文件。
问题
如何在Java代码中包含Python3Lexer文件,以便使用Python3Parser对象?

j2cgzkjk

j2cgzkjk1#

你不需要运行transformGrammar.py,只有当你使用Python作为目标语言时才需要运行transformGrammar.py(Java是你的目标语言)。
要在Java中解析python源文件并打印.py文件中的函数名称,请执行以下操作:

1.下载文件

创建一个目录并下载以下文件:

wget https://www.antlr.org/download/antlr-4.13.1-complete.jar
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/python/python3/Python3Lexer.g4
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/python/python3/Python3Parser.g4
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/python/python3/Java/Python3LexerBase.java
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/python/python3/Java/Python3ParserBase.java

字符串
(我使用了wget命令行,但你也可以通过浏览器下载它们)

2. main类

在同一目录中,创建一个名为Main.java的文件,内容如下:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;

public class Main {

    public static void main(String[] args) {
        
        String source = "def foo():\n    pass\n\ndef quared(n):\n    return n * n";

        System.out.println(source + "\n");
        
        Python3Lexer lexer = new Python3Lexer(CharStreams.fromString(source));
        Python3Parser parser = new Python3Parser(new CommonTokenStream(lexer));

        ParseTreeWalker.DEFAULT.walk(new Python3ParserBaseListener() {

            // funcdef
            //  : 'def' name parameters ('->' test)? ':' block
            //  ;
            @Override
            public void enterFuncdef(Python3Parser.FuncdefContext ctx) {
                System.out.println("name=" + ctx.name().getText());
            }

        }, parser.file_input());
    }
}

3.生成解析器

要生成lexer和parser类,请执行以下命令:

java -jar antlr-4.13.1-complete.jar *.g4

4.编译java

要编译所有Java源文件,请执行以下命令:

javac -cp antlr-4.13.1-complete.jar *.java

5.运行main类

要运行Main类,请在Linux或Mac上执行以下命令:

java -cp .:antlr-4.13.1-complete.jar Main


在Windows上执行:

java -cp .;antlr-4.13.1-complete.jar Main


您应该看到以下内容打印到您的控制台:

def foo():
    pass

def quared(n):
    return n * n

name=foo
name=quared

相关问题