2.4K Star 8.2K Fork 4.4K

GVPMindSpore / mindspore

 / 详情

[ST][MS][2.3][mindformers][readme][llama2][大模型] generate多卡推理和pipeline多卡推理失败,TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.

DONE
Bug-Report
创建于  
2024-05-08 16:49
name about labels
Bug Report Use this template for reporting a bug kind/bug

Describe the current behavior / 问题描述 (Mandatory / 必填)

generate多卡推理和pipeline多卡推理失败,报如下错误:
TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.

Environment / 环境信息 (Mandatory / 必填)

  • Hardware Environment(Ascend/GPU/CPU) / 硬件环境:

Please delete the backend not involved / 请删除不涉及的后端:
/device ascend

  • Software Environment / 软件环境 (Mandatory / 必填):
    -- MindSpore version (e.g., 1.7.0.Bxxx) :
    -- Python version (e.g., Python 3.7.5) :
    -- OS platform and distribution (e.g., Linux Ubuntu 16.04):
    -- GCC/Compiler version (if compiled from source):
    ms版本:commit_id = '[sha1]:84aa3010,[branch]:(HEAD,origin/master,origin/HEAD,master)'
    run包:runpkg_version:Milan_C17/20240414
    mf版本: commit de0dda0752cbee19900e1a9116dcb4924bb1b41e (HEAD -> dev, origin/dev, origin/HEAD)

  • Excute Mode / 执行模式 (Mandatory / 必填)(PyNative/Graph):

Please delete the mode not involved / 请删除不涉及的模式:
/mode pynative
/mode graph

Related testcase / 关联用例 (Mandatory / 必填)

不涉及

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

不涉及

执行命令:
source /usr/local/Ascend/latest/bin/setenv.bash
source /home/miniconda3/bin/activate ci
export ASCEND_HOME_PATH=/usr/local/Ascend/latest
export PYTHONPATH=/home/jenkins0/mindformers/:$PYTHONPATH

bash scripts/msrun_launcher.sh "predict_custom.py
--yaml_file path/to/config_yaml
--checkpoint_path path/to/shard_checkpoint_dir
--model_type llama2_70b" 4

Describe the expected behavior / 预期结果 (Mandatory / 必填)

正常推理

Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

报错日志:
Traceback (most recent call last):
File "predict_custom_generate.py", line 83, in
main(args)
File "predict_custom_generate.py", line 53, in main
warm_up_model.infer_predict_layout(ms.Tensor(np.ones(shape=(model_config.batch_size, model_config.seq_length)), ms.int32))
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 1896, in infer_predict_layout
predict_net.compile(*predict_data)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 997, in compile
jit_config_dict=self._jit_config_dict, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/common/api.py", line 1642, in compile
result = self._graph_executor.compile(obj, args, kwargs, phase, self._use_vm_mode())
TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.
The reason may be: lack of definition of type cast, or incorrect type when creating the node.


  • Framework Unexpected Exception Raised:

This exception is caused by framework's unexpected error. Please create an issue at https://gitee.com/mindspore/mindspore/issues to get help.


  • C++ Call Stack: (For framework developers)

mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:1263 CheckArgsSizeAndType

Special notes for this issue/备注 (Optional / 选填)

走给 谭伟城

评论 (5)

wenli 创建了Bug-Report
wenli 添加了
 
kind/bug
标签
wenli 添加了
 
v2.3.0.rc2
标签
wenli 添加了
 
sig/mslite
标签
wenli 添加了
 
attr/function
标签
wenli 添加了
 
stage/func-debug
标签
wenli 添加了
 
device/ascend
标签
wenli 添加协作者wangxingyan
wenli 添加协作者BJ-WANG
展开全部操作日志

Please assign maintainer to check this issue.
请为此issue分配处理人。
@wenli

感谢您的提问,您可以评论//mindspore-assistant更快获取帮助:

  1. 如果您刚刚接触MindSpore,或许您可以在教程找到答案
  2. 如果您是资深Pytorch用户,您或许需要:
  1. 如果您遇到动态图问题,可以设置set_context(pynative_synchronize=True)查看报错栈协助定位
  2. 模型精度调优问题可参考官网调优指南
  3. 如果您反馈的是框架BUG,请确认您在ISSUE中提供了MindSpore版本、使用的后端类型(CPU、GPU、Ascend)、环境、训练的代码官方链接以及可以复现报错的代码的启动方式等必要的定位信息
  4. 如果您已经定位出问题根因,欢迎提交PR参与MindSpore开源社区,我们会尽快review

原issue [ST][MS][2.3][mindformers][readme][llama2][大模型] 文档描述不清晰,启动方式需改成msrun启动,单卡推理失败
https://e.gitee.com/mind_spore/issues/list?issue=I9MJ1R 部分回归不通过,新提单跟踪

tan-wei-cheng 任务状态TODO 修改为VALIDATION
tan-wei-cheng 负责人tan-wei-cheng 修改为wenli
i-robot 添加了
 
gitee
标签

回归版本:
ms版本:commit_id = '[sha1]:1a55df60,[branch]:(HEAD,origin/master,origin/HEAD,master)'
run包:runpkg_version:Milan_C17/20240414
mf版本:commit 6ad0c08f0a46c374f9cc4549713b0309ddd7d77a (HEAD -> dev, origin/dev, origin/HEAD)
回归步骤:参考issue复现步骤
基本功能:param_init_type: "float16"设置后,跑测正常
2024-05-09 16:15:17,223 - mindformers[mindformers/generation/text_generator.py:888] - INFO - total time: 299.9470839500427 s; generated tokens: 1514 tokens; generate speed: 5.047556989259353 tokens/s
2024-05-09 16:15:17,242 - mindformers[mindformers/modules/block_tables.py:129] - INFO - Clear block table cache engines.
I love Beijing, because it is a city with a long history and a rich culture. It is the capital of China, and it is also the political, economic and cultural center of China.

测试结论:回归通过

wenli 任务状态VALIDATION 修改为DONE
tan-wei-cheng 移除了
 
kind/bug
标签
tan-wei-cheng 移除了
 
kind/bug
标签
tan-wei-cheng 添加了
 
rca/others
标签
tan-wei-cheng 添加了
 
ctl/solutiontest
标签
tan-wei-cheng 移除了
 
v2.3.0.rc2
标签
tan-wei-cheng 移除了
 
v2.3.0.rc2
标签
tan-wei-cheng 添加了
 
rct/oldrelease
标签

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(5)
Python
1
https://gitee.com/mindspore/mindspore.git
git@gitee.com:mindspore/mindspore.git
mindspore
mindspore
mindspore

搜索帮助