| 查看: 2096 | 回复: 1 | ||
qh203铜虫 (小有名气)
|
[求助]
root和普通用户下并行计算问题
|
|
在root用户下，用openmpi并行计算cpi 这个算例，6个节点，每个节点8个cpu。输出正常，如下 [root@node1 examples]# mpirun -np 40 -machinefile test ./cpi Process 3 on node2 Process 38 on node6 Process 18 on node4 Process 32 on node6 Process 20 on node4 Process 2 on node2 Process 35 on node6 Process 34 on node6 Process 22 on node4 Process 7 on node2 Process 23 on node4 Process 5 on node2 Process 4 on node2 Process 37 on node6 Process 33 on node6 Process 30 on node5 Process 8 on node3 Process 26 on node5 Process 10 on node3 Process 15 on node3 Process 27 on node5 Process 31 on node5 Process 28 on node5 Process 24 on node5 Process 19 on node4 Process 21 on node4 Process 17 on node4 Process 6 on node2 Process 16 on node4 Process 25 on node5 Process 9 on node3 Process 11 on node3 Process 13 on node3 Process 14 on node3 Process 0 on node2 Process 1 on node2 Process 36 on node6 Process 39 on node6 Process 12 on node3 Process 29 on node5 pi is approximately 3.1416009869231245, Error is 0.0000083333333314 wall clock time = 0.128546 在普通用户下用openmpi并行计算cpi这个算例，输出则变成 [aojjj@node1 examples]$ mpirun -np 40 -machinefile test ./cpi libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. -------------------------------------------------------------------------- The OpenFabrics (openib) BTL failed to register memory in the driver. Please check /var/log/messages or dmesg for driver specific failure reason. The failure occured here: Local host: mthca0 Device: openib_reg_mr Function: Cannot allocate memory() Errno says: You may need to consult with your system administrator to get this problem fixed. -------------------------------------------------------------------------- -------------------------------------------------------------------------- The OpenFabrics (openib) BTL failed to initialize while trying to allocate some locked memory. This typically can indicate that the memlock limits are set too low. For most HPC installations, the memlock limits should be set to "unlimited". The failure occured here: Local host: node4 OMPI source: btl_openib_component.c:1161 Function: ompi_free_list_init_ex_new() Device: mthca0 Memlock limit: 32768 You may need to consult with your system administrator to get this problem fixed. This FAQ entry on the Open MPI web site may also be helpful: http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages -------------------------------------------------------------------------- -------------------------------------------------------------------------- WARNING: There was an error initializing an OpenFabrics device. Local host: node4 Local device: mthca0 -------------------------------------------------------------------------- libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. Process 26 on node5 Process 8 on node3 Process 28 on node5 Process 1 on node2 Process 29 on node5 Process 4 on node2 Process 22 on node4 Process 2 on node2 Process 15 on node3 Process 25 on node5 Process 31 on node5 Process 38 on node6 Process 14 on node3 Process 30 on node5 Process 32 on node6 Process 39 on node6 Process 37 on node6 Process 33 on node6 Process 36 on node6 Process 35 on node6 Process 16 on node4 Process 18 on node4 Process 10 on node3 Process 21 on node4 Process 19 on node4 Process 20 on node4 Process 11 on node3 Process 17 on node4 Process 9 on node3 Process 0 on node2 Process 7 on node2 Process 6 on node2 Process 5 on node2 Process 23 on node4 Process 24 on node5 Process 3 on node2 Process 27 on node5 Process 34 on node6 Process 12 on node3 Process 13 on node3 pi is approximately 3.1416009869231245, Error is 0.0000083333333314 wall clock time = 3.002147 [node1:02112] 39 more processes have sent help message help-mpi-btl-openib.txt / mem-reg-fail [node1:02112] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages [node1:02112] 36 more processes have sent help message help-mpi-btl-openib.txt / init-fail-no-mem [node1:02112] 39 more processes have sent help message help-mpi-btl-openib.txt / error in device init 也计算出来了，但是多了许多warniing 和error的提示。 在各个节点修改了/etc/security/limits.conf 和/etc/init.d/sshd, 还是不行。 到底问题在哪里？ |
回复此楼» 猜你喜欢
浙大物院表面物理课题组长期招收普博及直博学生
已经有259人回复
-
【征稿】Springer Nature热电专题 Discover Materials / Discover Chemistry
已经有47人回复
-
物理学I论文润色/翻译怎么收费?
已经有277人回复
-
25届硕士申请，材料计算方向
已经有2人回复
-
课题组招1-2名硕士
已经有1人回复
-
机时越来越贵之后，我把一部分 DFT 流程搬到了 AI 计算平台上
已经有0人回复
-
斯坦福大学与Elsevier世界前2%学者新排名查询
已经有1人回复
-
27申博推荐：南京航空航天大学国际前沿院光学方向招收博士生！
已经有19人回复
-
华南师范大学+双一流学科物理学+广东省杰青团队2027年博士招生+机器学习驱动材料设计
已经有1人回复
-
apl投稿状态
已经有2人回复
-
华南师范大学+双一流学科物理学+广东省杰青团队2027年博士招生+机器学习驱动材料设计
已经有0人回复
» 本主题相关价值贴推荐，对您同样有帮助:
- 请做CUDA并行计算的大神来指点下工作站的配置
已经有7人回复
- Fluent 14.5 并行计算问题
已经有10人回复
- 单机多核并行计算下UDF的问题
已经有9人回复
- 请教并行计算linux的问题
已经有19人回复
- LAMMPS并行计算的问题（cpu——time关系）
已经有17人回复
- VS20？？+intel visual fortran2011XE做并行计算的，能介绍一下经验吗？
已经有19人回复
- 集群配置ssh，需要要给每个用户都单独配置吗？
已经有26人回复
- ivf在windows下如何实现omp并行计算，vs2008应如何设置
已经有3人回复
- gaussian 03Revison-E01版在Window 下不能并行计算啊!
已经有9人回复
- ansys 13 fluent 并行计算问题
已经有13人回复
- 【求助成功】在linux下装MS出现问题，求教
已经有5人回复
- 【求助】fluent模拟气固流化床采用欧拉模型并行计算出现问题
已经有12人回复
- 【求助】关于fluent多机并行计算连接问题
已经有6人回复
qh203
铜虫 (小有名气)
- 应助: 0 (幼儿园)
- 金币: 2817.1
- 散金: 765
- 红花: 1
- 帖子: 123
- 在线: 215.3小时
- 虫号: 1401474
- 注册: 2011-09-14
- 专业: 凝聚态物性 II ：电子结构
2楼2013-10-13 21:22:41
50