debugging 如何使用Vscode调试llama2推断命令?

xqnpmsa8  于 6个月前  发布在  Vscode
关注(0)|答案(1)|浏览(97)

我尝试在vscode调试模式下运行LLAMA 2推理脚本(如下所示):

torchrun --nproc_per_node 1 example_text_completion.py \
    --ckpt_dir models/7B-Chat \
    --tokenizer_path tokenizer.model \
    --max_seq_len 128 --max_batch_size 4

字符串
在此之前,我可以成功地运行它与我的命令行界面,这表明我的python环境是正确的。
我已经尝试了下面的两个调试工具:
1.

{
     "name": "Python: run_llama2_inference",
     "type": "python",
     "request": "launch",
     "module": "torchrun",
     "args": [
         "--nproc_per_node=1",
         "example_chat_completion.py",
         "--ckpt_dir=models/7B-Chat/",
         "--tokenizer_path=tokenizer.model",
         "--max_seq_len=512",
         "--max_batch_size=4",
     ],
     "console": "integratedTerminal",
     "justMyCode": true,
     "env": {
         "PYTHONPATH": "${workspaceFolder}"
     }
 },


对应的错误消息:“No module named torchrun”
1.

{
     "name": "Python: run_llama2_inference",
     "type": "python",
     "request": "launch",
     "module": "torch.distributed.launch",
     "args": [
         "--use-env",
         "example_chat_completion.py",
         "--nproc_per_node=1",
         "--ckpt_dir=models/7B-Chat/",
         "--tokenizer_path=tokenizer.model",
         "--max_seq_len=512",
         "--max_batch_size=4",
     ],
     "console": "integratedTerminal",
     "justMyCode": true,
     "env":
         "PYTHONPATH": "${workspaceFolder}"
     }
 },


对应的错误消息:“无法使用参数:--nproc_per_node=1”
这两种方法都不像预期的那样有效。我想寻求在线Maven的建议。提前感谢您的想法或建议!

s6fujrry

s6fujrry1#

您可以使用"program"字段指定要运行的Python脚本(example_text_completion.py),并使用"args"字段传递其余参数。
以下是如何修改启动配置的示例:

{
    "name": "Python: run_llama2_inference",
    "type": "python",
    "request": "launch",
    "program": "${workspaceFolder}/example_chat_completion.py",
    "args": [
        "--ckpt_dir=models/7B-Chat/",
        "--tokenizer_path=tokenizer.model",
        "--max_seq_len=512",
        "--max_batch_size=4",
    ],
    "console": "integratedTerminal",
    "justMyCode": true,
    "env": {
        "PYTHONPATH": "${workspaceFolder}"
    }
}

字符串

相关问题