name | about | labels |
---|---|---|
Bug Report | Use this template for reporting a bug | kind/bug |
generate多卡推理和pipeline多卡推理失败,报如下错误:
TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.
Ascend
/GPU
/CPU
) / 硬件环境:Please delete the backend not involved / 请删除不涉及的后端:
/device ascend
Software Environment / 软件环境 (Mandatory / 必填):
-- MindSpore version (e.g., 1.7.0.Bxxx) :
-- Python version (e.g., Python 3.7.5) :
-- OS platform and distribution (e.g., Linux Ubuntu 16.04):
-- GCC/Compiler version (if compiled from source):
ms版本:commit_id = '[sha1]:84aa3010,[branch]:(HEAD,origin/master,origin/HEAD,master)'
run包:runpkg_version:Milan_C17/20240414
mf版本: commit de0dda0752cbee19900e1a9116dcb4924bb1b41e (HEAD -> dev, origin/dev, origin/HEAD)
Excute Mode / 执行模式 (Mandatory / 必填)(PyNative
/Graph
):
Please delete the mode not involved / 请删除不涉及的模式:
/mode pynative
/mode graph
不涉及
不涉及
执行命令:
source /usr/local/Ascend/latest/bin/setenv.bash
source /home/miniconda3/bin/activate ci
export ASCEND_HOME_PATH=/usr/local/Ascend/latest
export PYTHONPATH=/home/jenkins0/mindformers/:$PYTHONPATH
bash scripts/msrun_launcher.sh "predict_custom.py
--yaml_file path/to/config_yaml
--checkpoint_path path/to/shard_checkpoint_dir
--model_type llama2_70b" 4
正常推理
报错日志:
Traceback (most recent call last):
File "predict_custom_generate.py", line 83, in
main(args)
File "predict_custom_generate.py", line 53, in main
warm_up_model.infer_predict_layout(ms.Tensor(np.ones(shape=(model_config.batch_size, model_config.seq_length)), ms.int32))
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 1896, in infer_predict_layout
predict_net.compile(*predict_data)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 997, in compile
jit_config_dict=self._jit_config_dict, **kwargs)
File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/common/api.py", line 1642, in compile
result = self._graph_executor.compile(obj, args, kwargs, phase, self._use_vm_mode())
TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.
The reason may be: lack of definition of type cast, or incorrect type when creating the node.
This exception is caused by framework's unexpected error. Please create an issue at https://gitee.com/mindspore/mindspore/issues to get help.
mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:1263 CheckArgsSizeAndType
走给 谭伟城
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
感谢您的提问,您可以评论//mindspore-assistant更快获取帮助:
原issue [ST][MS][2.3][mindformers][readme][llama2][大模型] 文档描述不清晰,启动方式需改成msrun启动,单卡推理失败
https://e.gitee.com/mind_spore/issues/list?issue=I9MJ1R 部分回归不通过,新提单跟踪
回归版本:
ms版本:commit_id = '[sha1]:1a55df60,[branch]:(HEAD,origin/master,origin/HEAD,master)'
run包:runpkg_version:Milan_C17/20240414
mf版本:commit 6ad0c08f0a46c374f9cc4549713b0309ddd7d77a (HEAD -> dev, origin/dev, origin/HEAD)
回归步骤:参考issue复现步骤
基本功能:param_init_type: "float16"设置后,跑测正常
2024-05-09 16:15:17,223 - mindformers[mindformers/generation/text_generator.py:888] - INFO - total time: 299.9470839500427 s; generated tokens: 1514 tokens; generate speed: 5.047556989259353 tokens/s
2024-05-09 16:15:17,242 - mindformers[mindformers/modules/block_tables.py:129] - INFO - Clear block table cache engines.
I love Beijing, because it is a city with a long history and a rich culture. It is the capital of China, and it is also the political, economic and cultural center of China.
测试结论:回归通过
登录 后才可以发表评论