2.4K Star 8.2K Fork 4.4K

GVPMindSpore / mindspore

 / 详情

[ST][MS][教程]resnet50图像分类在910A上graph模式训练精度达不到官网的0.74

TODO
Bug-Report
创建于  
2024-04-30 17:32
name about labels
Bug Report Use this template for reporting a bug kind/bug

Describe the current behavior / 问题描述 (Mandatory / 必填)

resnet50图像学习在910A上graph模式训练精度达不到官网的0.74

Environment / 环境信息 (Mandatory / 必填)

  • Hardware Environment(Ascend/GPU/CPU) / 硬件环境:

Please delete the backend not involved / 请删除不涉及的后端:
/device ascend

  • Software Environment / 软件环境 (Mandatory / 必填):
    -- MindSpore version (e.g., 1.7.0.Bxxx) :
    -- Python version (e.g., Python 3.7.5) :
    -- OS platform and distribution (e.g., Linux Ubuntu 16.04):
    -- GCC/Compiler version (if compiled from source):

  • Excute Mode / 执行模式 (Mandatory / 必填)(PyNative/Graph):

Please delete the mode not involved / 请删除不涉及的模式:
/mode graph

Related testcase / 关联用例 (Mandatory / 必填)

用例路径:solution_test/cases/03subject_test/06document/02network_cases
关联用例:test_ms_tutorial_cv_resnet50_image_classification_0001.py

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

  1. 进入官网,https://www.mindspore.cn/tutorials/application/zh-CN/master/cv/resnet50.html
  2. 下载样例代码,使用python执行
  3. 验证教程网络训练是否成功,日志中是否存在告警日志

Describe the expected behavior / 预期结果 (Mandatory / 必填)

网络训练成功,精度达到0.74

Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

Epoch: [  1/  5], Steps: [  1/196], Train Loss: [2.312]
Epoch: [  1/  5], Steps: [101/196], Train Loss: [1.825]
Epoch: [  1/  5], Steps: [196/196], Train Loss: [1.425]
--------------------------------------------------
Epoch: [  1/  5], Average Train Loss: [1.873], Accuracy: [0.472]
--------------------------------------------------
Epoch: [  2/  5], Steps: [  1/196], Train Loss: [1.543]
Epoch: [  2/  5], Steps: [101/196], Train Loss: [1.393]
Epoch: [  2/  5], Steps: [196/196], Train Loss: [1.366]
--------------------------------------------------
Epoch: [  2/  5], Average Train Loss: [1.348], Accuracy: [0.564]
--------------------------------------------------
Epoch: [  3/  5], Steps: [  1/196], Train Loss: [1.150]
Epoch: [  3/  5], Steps: [101/196], Train Loss: [1.208]
Epoch: [  3/  5], Steps: [196/196], Train Loss: [1.011]
--------------------------------------------------
Epoch: [  3/  5], Average Train Loss: [1.175], Accuracy: [0.600]
--------------------------------------------------
Epoch: [  4/  5], Steps: [  1/196], Train Loss: [1.045]
Epoch: [  4/  5], Steps: [101/196], Train Loss: [1.151]
Epoch: [  4/  5], Steps: [196/196], Train Loss: [0.923]
--------------------------------------------------
Epoch: [  4/  5], Average Train Loss: [1.100], Accuracy: [0.617]
--------------------------------------------------
Epoch: [  5/  5], Steps: [  1/196], Train Loss: [0.969]
Epoch: [  5/  5], Steps: [101/196], Train Loss: [0.941]
Epoch: [  5/  5], Steps: [196/196], Train Loss: [1.063]
--------------------------------------------------
Epoch: [  5/  5], Average Train Loss: [1.069], Accuracy: [0.623]

Special notes for this issue/备注 (Optional / 选填)

走给吕昱峰

评论 (5)

baimz 创建了Bug-Report
baimz 添加了
 
kind/bug
标签
baimz 添加了
 
attr/function
标签
baimz 添加了
 
stage/coding
标签
baimz 添加了
 
v2.3.0.rc2
标签
baimz 添加了
 
device/ascend
标签
baimz 添加了
 
sig/doc
标签
展开全部操作日志

Please assign maintainer to check this issue.
请为此issue分配处理人。
@baimz

感谢您的提问,您可以评论//mindspore-assistant更快获取帮助:

  1. 如果您刚刚接触MindSpore,或许您可以在教程找到答案
  2. 如果您是资深Pytorch用户,您或许需要:
  1. 如果您遇到动态图问题,可以设置set_context(pynative_synchronize=True)查看报错栈协助定位
  2. 模型精度调优问题可参考官网调优指南
  3. 如果您反馈的是框架BUG,请确认您在ISSUE中提供了MindSpore版本、使用的后端类型(CPU、GPU、Ascend)、环境、训练的代码官方链接以及可以复现报错的代码的启动方式等必要的定位信息
  4. 如果您已经定位出问题根因,欢迎提交PR参与MindSpore开源社区,我们会尽快review
xiangminshan 负责人xiangminshan 修改为lvyufenghw
baimz 修改了描述
tuoyuanyuan 负责人lvyufenghw 修改为zhaoyu
tuoyuanyuan 添加协作者lvyufenghw
tuoyuanyuan 添加协作者周剑

训练到11个epoch才达到官网的指标
输入图片说明

i-robot 添加了
 
foruda
标签
dongronghua 修改了标题

CCB结论:
遗留原因:910A从VM切换到GE后架构发生变动
风险:外部用户使用本教程时设为GRAPH_MODE时会碰到精度收敛慢的问题
规避措施:本版本遗留,630版本继续解决

fangwenyi 添加了
 
ccb/bug
标签
fangwenyi 移除了
 
v2.3.0.rc2
标签
fangwenyi 添加了
 
master
标签

二分法定位到commitID为0140d5d8eea76c016454fac1d779492dd84ca818导致出现的问题,相关PRhttps://gitee.com/mindspore/mindspore/pulls/67037

i-robot 添加了
 
gitee
标签
zhaoyu 添加协作者zhaoyu
zhaoyu 负责人zhaoyu 修改为fary86
zhaoyu 取消协作者zhaoyu

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(7)
10403258 zhaoyu2012 1660048930 11016979 xiangmd 1654824581
Python
1
https://gitee.com/mindspore/mindspore.git
git@gitee.com:mindspore/mindspore.git
mindspore
mindspore
mindspore

搜索帮助