| 查看: 2846 | 回复: 10 | |||
| 当前只显示满足指定条件的回帖，点击这里查看本话题的所有回帖 | |||
[交流]
vasp编译测试@天河2号 HELP 已有4人参与
|
|||
|
节点:2路Ivy Bridge Xeon E5-2692 54GB 编译器：Intel Fortran 14.0.1+MKL MPI： 天河2优化的MPICH 3版本 几天测试, 得到的加速不明显,具体如下: 1 NPAR设置非常重要,对速度影响很大, 保证NPAR>节点数, 并且最优结果是 NPAR=2 for 1 nodes NPAR=4 for 2 nodes NPAR=4 for 3 nodes NPAR=8 for 4 nodes 基本是保证12 cores per group， 硬件上即是一路CPU为一组能带 2 关闭scaLAPACK对1个node稍有加速，开启scaLAPACK对多节点没有加速帮助 3 测试不同cachesize，几乎没有差异 4 测试不同MPI_Block, 几乎没有差异 5 radical.F need to be lowered to -O1, 否则spin=2计算会报错 6 BLACS如果用libmkl_blacs_lp64.a居然报错, 但MPICH 3应该对应的这个, 而只有libmkl_blacs_intelmpi_lp64.a没有报错, 表示不能理解 7 Cluster 版本的FFT加速不明显 目前怀疑天河2的infiniBand速度很快，testbench没有遇到通信瓶颈，瓶颈还是在节点的计算负载 any idea？ 欢迎讨论 2楼贴makefile |
回复此楼» 本帖已获得的红花（最新10朵）
naohjohn» 猜你喜欢
在坚冰还盖着北海的时候，我看到了怒放的梅花。
已经有3人回复
-
如果此刻你正在为国基感到焦虑，不妨来听听这首《基金之外》
已经有8人回复
-
没有任何消息-是不是就凉了
已经有10人回复
-
人气不行了
已经有11人回复
-
2026国自然函评费到账
已经有22人回复
-
导师吐槽：我怎么摊上了这么个极品研究生！
已经有3人回复
-
2026年的国家社科基金项目通讯评审的新规则与新动向、新挑战
已经有7人回复
-
今天基金会出结果吗？20260819
已经有18人回复
-
我面上完蛋了
已经有14人回复
-
范进中举一文的中心思想
已经有8人回复
» 本主题相关价值贴推荐，对您同样有帮助:
- 求助：VASP编译问题
已经有10人回复
- vasp.5.2编译不过去，装不上，求大神帮忙
已经有8人回复
- 求助，VASP5.3.5的编译
已经有4人回复
- vasp 编译专用的可执行文件
已经有15人回复
- vasp 编译主程序时候出错，求好心人看一下
已经有17人回复
- vasp编译错误求助
已经有6人回复
- 关于 VASP 的编译经验分享
已经有16人回复
- 单机编译安装vasp求指导
已经有10人回复
- 【已解决】VASP内存占用疯涨……看来得重新编译了……
已经有14人回复
- 求助vasp编译好后无法运行问题？？
已经有24人回复
- [VASP编译]一 什么是.so文件、.a文件？
已经有11人回复
- 编译vasp主程序时，makefile中的链接库选择问题
已经有15人回复
- vasp5.2 openmpi+intel+mkl编译，能量正值问题
已经有7人回复
- vasp5.2 并行编译如何修改makefile
已经有5人回复
- 【求助】vasp编译 OFLAG
已经有7人回复
- 【求助】VASP 编译出错
已经有24人回复
liqizuiyang
木虫 (著名写手)
- 应助: 59 (初中生)
- 金币: 6859.5
- 散金: 261
- 红花: 94
- 帖子: 2093
- 在线: 708.9小时
- 虫号: 2366686
- 注册: 2013-03-21
- 性别: GG
- 专业: 凝聚态物性 II ：电子结构
3楼2014-12-20 13:47:47
|
.SUFFIXES: .inc .f .f90 .F #----------------------------------------------------------------------- # Makefile for Intel Fortran compiler for Pentium/Athlon/Opteron # based systems # we recommend this makefile for both Intel as well as AMD systems # for AMD based systems appropriate BLAS (libgoto) and fftw libraries are # however mandatory (whereas they are optional for Intel platforms) # For Athlon we recommend # ) to link against libgoto (and mkl as a backup for missing routines) # ) odd enough link in libfftw3xf_intel.a (fftw interface for mkl) # feedback is greatly appreciated # # The makefile was tested only under Linux on Intel and AMD platforms # the following compiler versions have been tested: # - ifc.7.1 works stable somewhat slow but reliably # - ifc.8.1 fails to compile the code properly # - ifc.9.1 recommended (both for 32 and 64 bit) # - ifc.10.1 partially recommended (both for 32 and 64 bit) # tested build 20080312 Package ID: l_fc_p_10.1.015 # the gamma only mpi version can not be compiles # using ifc.10.1 # - ifc.11.1 partially recommended (some problems with Gamma only and intel fftw) # Build 20090630 Package ID: l_cprof_p_11.1.046 # - ifort.12.1 strongly recommended (we use this to compile vasp) # Version 12.1.5.339 Build 20120612 # # it might be required to change some of library path ways, since # LINUX installations vary a lot # # Hence check ***ALL*** options in this makefile very carefully #----------------------------------------------------------------------- # # BLAS must be installed on the machine # there are several options: # 1) very slow but works: # retrieve the lapackage from ftp.netlib.org # and compile the blas routines (BLAS/SRC directory) # please use g77 or f77 for the compilation. When I tried to # use pgf77 or pgf90 for BLAS, VASP hang up when calling # ZHEEV (however this was with lapack 1.1 now I use lapack 2.0) # 2) more desirable: get an optimized BLAS # # the two most reliable packages around are presently: # 2a) Intels own optimised BLAS (PIII, P4, PD, PC2, Itanium) # http://developer.intel.com/software/products/mkl/ # this is really excellent, if you use Intel CPU's # # 2b) probably fastest SSE2 (4 GFlops on P4, 2.53 GHz, 16 GFlops PD, # around 30 GFlops on Quad core) # Kazushige Goto's BLAS # http://www.cs.utexas.edu/users/kgoto/signup_first.html # http://www.tacc.utexas.edu/resources/software/ # #----------------------------------------------------------------------- # all CPP processed fortran files have the extension .f90 SUFFIX=.f90 #----------------------------------------------------------------------- # fortran compiler and linker #----------------------------------------------------------------------- #FC=ifort # fortran linker #FCL=$(FC) #----------------------------------------------------------------------- # whereis CPP ?? (I need CPP, can't use gcc with proper options) # that's the location of gcc for SUSE 5.3 # # CPP_ = /usr/lib/gcc-lib/i486-linux/2.7.2/cpp -P -C # # that's probably the right line for some Red Hat distribution: # # CPP_ = /usr/lib/gcc-lib/i386-redhat-linux/2.7.2.3/cpp -P -C # # SUSE X.X, maybe some Red Hat distributions: CPP_ = ./preprocess <$*.F | /usr/bin/cpp -P -C -traditional >$*$(SUFFIX) # this release should be fpp clean # we now recommend fpp as preprocessor # if this fails go back to cpp CPP_=fpp -f_com=no -free -w0 $*.F $*$(SUFFIX) #----------------------------------------------------------------------- # possible options for CPP: # NGXhalf charge density reduced in X direction # wNGXhalf gamma point only reduced in X direction # avoidalloc avoid ALLOCATE if possible # PGF90 work around some for some PGF90 / IFC bugs # CACHE_SIZE 1000 for PII,PIII, 5000 for Athlon, 8000-12000 P4, PD # RPROMU_DGEMV use DGEMV instead of DGEMM in RPRO (depends on used BLAS) # RACCMU_DGEMV use DGEMV instead of DGEMM in RACC (depends on used BLAS) # tbdyn MD package of Tomas Bucko #----------------------------------------------------------------------- #CPP = $(CPP_) -DHOST=\"LinuxIFC\" \ -DCACHE_SIZE=12000 -DPGF90 -Davoidalloc -DNGXhalf \ # -DRPROMU_DGEMV -DRACCMU_DGEMV #----------------------------------------------------------------------- # general fortran flags (there must a trailing blank on this line) # byterecl is strictly required for ifc, since otherwise # the WAVECAR file becomes huge #----------------------------------------------------------------------- FFLAGS = -FR -names lowercase -assume byterecl -xAVX #----------------------------------------------------------------------- # optimization # we have tested whether higher optimisation improves performance # -axK SSE1 optimization, but also generate code executable on all mach. # xK improves performance somewhat on XP, and a is required in order # to run the code on older Athlons as well # -xW SSE2 optimization # -axW SSE2 optimization, but also generate code executable on all mach. # -tpp6 P3 optimization # -tpp7 P4 optimization #----------------------------------------------------------------------- # ifc.9.1, ifc.10.1 recommended OFLAG=-O2 -ip OFLAG_HIGH = $(OFLAG) OBJ_HIGH = OBJ_NOOPT = DEBUG = -FR -O0 INLINE = $(OFLAG) #----------------------------------------------------------------------- # the following lines specify the position of BLAS and LAPACK # we recommend to use mkl, that is simple and most likely # fastest in Intel based machines #----------------------------------------------------------------------- # mkl path for ifc 11 compiler #MKL_PATH=$(MKLROOT)/lib/em64t # mkl path for ifc 12 compiler MKLROOT=/vol-th/intel/composer_xe_2013_sp1.1.106/mkl MKL_PATH=$(MKLROOT)/lib/intel64 #MKL_FFTW_PATH=/vol-th/home/ac_sim_zfdi_1/xhshi/lib/fftw3xf MKL_FFTW_PATH=/vol-th/home/ac_sim_zfdi_1/xhshi/lib/fftw3x_cdft # BLAS # setting -DRPROMU_DGEMV -DRACCMU_DGEMV in the CPP lines usually speeds up program execution BLAS= -Wl,--start-group $(MKL_PATH)/libmkl_scalapack_lp64.a $(MKL_PATH)/libmkl_cdft_core.a $(MKL_PATH)/libmkl_blacs_intelmpi_lp64.a $(MKL_PATH)/libmkl_intel_lp64.a $(MKL_PATH)/libmkl_intel_thread.a $(MKL_PATH)/libmkl_core.a -Wl,--end-group -liomp5 # faster linking and available from at least version 11 #BLAS= -liomp5 -mkl # LAPACK, use vasp.5.lib/lapack_double #LAPACK= ../vasp.5.lib/lapack_double.o # LAPACK from mkl, usually faster and contains scaLAPACK as well #LAPACK= $(MKL_PATH)/libmkl_intel_lp64.a # here a tricky version, link in libgoto and use mkl as a backup # also needs a special line for LAPACK # this is the best thing you can do on AMD based systems !!!!!! #BLAS = -Wl,--start-group /opt/libs/libgoto/libgoto.so $(MKL_PATH)/libmkl_intel_thread.a $(MKL_PATH)/libmkl_core.a -Wl,--end-group -liomp5 #LAPACK= /opt/libs/libgoto/libgoto.so $(MKL_PATH)/libmkl_intel_lp64.a #----------------------------------------------------------------------- #LIB = -L../vasp.5.lib -ldmy \ ../vasp.5.lib/linpack_double.o $(LAPACK) \ $(BLAS) # options for linking, nothing is required (usually) #LINK = #----------------------------------------------------------------------- # fft libraries: # VASP.5.2 can use fftw.3.1.X (http://www.fftw.org) # since this version is faster on P4 machines, we recommend to use it #----------------------------------------------------------------------- #FFT3D = fft3dfurth.o fft3dlib.o # alternatively: fftw.3.1.X is slighly faster and should be used if available #FFT3D = fftw3d.o fft3dlib.o /opt/libs/fftw-3.1.2/lib/libfftw3.a # you may also try to use the fftw wrapper to mkl (but the path might vary a lot) # it seems this is best for AMD based systems #FFT3D = fftw3d.o fft3dlib.o $(MKL_FFTW_PATH)/libfftw3xf_intel.a #INCS = -I$(MKLROOT)/include/fftw #======================================================================= # MPI section, uncomment the following lines until # general rules and compile lines # presently we recommend OPENMPI, since it seems to offer better # performance than lam or mpich # # !!! Please do not send me any queries on how to install MPI, I will # certainly not answer them !!!! #======================================================================= #----------------------------------------------------------------------- # fortran linker for mpi #----------------------------------------------------------------------- FC=/usr/local/mpi3/bin/mpif90 FCL=$(FC) #----------------------------------------------------------------------- # additional options for CPP in parallel version (see also above): # NGZhalf charge density reduced in Z direction # wNGZhalf gamma point only reduced in Z direction # scaLAPACK use scaLAPACK (recommended if mkl is available) # avoidalloc avoid ALLOCATE if possible # PGF90 work around some for some PGF90 / IFC bugs # CACHE_SIZE 1000 for PII,PIII, 5000 for Athlon, 8000-12000 P4, PD # RPROMU_DGEMV use DGEMV instead of DGEMM in RPRO (depends on used BLAS) # RACCMU_DGEMV use DGEMV instead of DGEMM in RACC (depends on used BLAS) # tbdyn MD package of Tomas Bucko #----------------------------------------------------------------------- #----------------------------------------------------------------------- CPP = $(CPP_) -DMPI -DHOST=\"LinuxIFC\" -DIFC \ -DCACHE_SIZE=2000 -DPGF90 -Davoidalloc -DNGZhalf \ -DMPI_BLOCK=64000 -Duse_collective -DscaLAPACK\ -DRPROMU_DGEMV -DRACCMU_DGEMV -DnoSTOPCAR #----------------------------------------------------------------------- # location of SCALAPACK # if you do not use SCALAPACK simply leave this section commented out #----------------------------------------------------------------------- # usually simplest link in mkl scaLAPACK #BLACS= -lmkl_blacs_openmpi_lp64 #SCA= $(MKL_PATH)/libmkl_scalapack_lp64.a $(BLACS) #----------------------------------------------------------------------- # libraries #----------------------------------------------------------------------- LIB = -L../vasp.5.lib -ldmy \ ../vasp.5.lib/linpack_double.o \ $(SCA) $(LAPACK) $(BLAS) #----------------------------------------------------------------------- # parallel FFT #----------------------------------------------------------------------- # FFT: fftmpi.o with fft3dlib of Juergen Furthmueller #FFT3D = fftmpi.o fftmpi_map.o fft3dfurth.o fft3dlib.o # alternatively: fftw.3.1.X is slighly faster and should be used if available #FFT3D = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o /opt/libs/fftw-3.1.2/lib/libfftw3.a # you may also try to use the fftw wrapper to mkl (but the path might vary a lot) # it seems this is best for AMD based systems FFT3D = fftmpiw.o fftmpi_map.o fftw3d.o fft3dlib.o $(MKL_FFTW_PATH)/libfftw3x_cdft_lp64.a INCS = -I$(MKLROOT)/include/fftw #----------------------------------------------------------------------- # general rules and compile lines #----------------------------------------------------------------------- BASIC= symmetry.o symlib.o lattlib.o random.o SOURCE= base.o mpi.o smart_allocate.o xml.o \ constant.o jacobi.o main_mpi.o scala.o \ asa.o lattice.o poscar.o ini.o mgrid.o xclib.o vdw_nl.o xclib_grad.o \ radial.o pseudo.o gridq.o ebs.o \ mkpoints.o wave.o wave_mpi.o wave_high.o spinsym.o \ $(BASIC) nonl.o nonlr.o nonl_high.o dfast.o choleski2.o \ mix.o hamil.o xcgrad.o xcspin.o potex1.o potex2.o \ constrmag.o cl_shift.o relativistic.o LDApU.o \ paw_base.o metagga.o egrad.o pawsym.o pawfock.o pawlhf.o rhfatm.o hyperfine.o paw.o \ mkpoints_full.o charge.o Lebedev-Laikov.o stockholder.o dipol.o pot.o \ dos.o elf.o tet.o tetweight.o hamil_rot.o \ chain.o dyna.o k-proj.o sphpro.o us.o core_rel.o \ aedens.o wavpre.o wavpre_noio.o broyden.o \ dynbr.o hamil_high.o rmm-diis.o reader.o writer.o tutor.o xml_writer.o \ brent.o stufak.o fileio.o opergrid.o stepver.o \ chgloc.o fast_aug.o fock_multipole.o fock.o mkpoints_change.o sym_grad.o \ mymath.o internals.o npt_dynamics.o dynconstr.o dimer_heyden.o dvvtrajectory.o vdwforcefield.o \ nmr.o pead.o subrot.o subrot_scf.o \ force.o pwlhf.o gw_model.o optreal.o steep.o davidson.o david_inner.o \ electron.o rot.o electron_all.o shm.o pardens.o paircorrection.o \ optics.o constr_cell_relax.o stm.o finite_diff.o elpol.o \ hamil_lr.o rmm-diis_lr.o subrot_cluster.o subrot_lr.o \ lr_helper.o hamil_lrf.o elinear_response.o ilinear_response.o \ linear_optics.o \ setlocalpp.o wannier.o electron_OEP.o electron_lhf.o twoelectron4o.o \ mlwf.o ratpol.o screened_2e.o wave_cacher.o chi_base.o wpot.o \ local_field.o ump2.o ump2kpar.o fcidump.o ump2no.o \ bse_te.o bse.o acfdt.o chi.o sydmat.o dmft.o \ rmm-diis_mlr.o linear_response_NMR.o wannier_interpol.o linear_response.o vasp: $(SOURCE) $(FFT3D) $(INC) main.o rm -f vasp $(FCL) -o vasp main.o $(SOURCE) $(FFT3D) $(LIB) $(LINK) makeparam: $(SOURCE) $(FFT3D) makeparam.o main.F $(INC) $(FCL) -o makeparam $(LINK) makeparam.o $(SOURCE) $(FFT3D) $(LIB) zgemmtest: zgemmtest.o base.o random.o $(INC) $(FCL) -o zgemmtest $(LINK) zgemmtest.o random.o base.o $(LIB) dgemmtest: dgemmtest.o base.o random.o $(INC) $(FCL) -o dgemmtest $(LINK) dgemmtest.o random.o base.o $(LIB) ffttest: base.o smart_allocate.o mpi.o mgrid.o random.o ffttest.o $(FFT3D) $(INC) $(FCL) -o ffttest $(LINK) ffttest.o mpi.o mgrid.o random.o smart_allocate.o base.o $(FFT3D) $(LIB) kpoints: $(SOURCE) $(FFT3D) makekpoints.o main.F $(INC) $(FCL) -o kpoints $(LINK) makekpoints.o $(SOURCE) $(FFT3D) $(LIB) clean: -rm -f *.g *.f *.o *.L *.mod ; touch *.F main.o: main$(SUFFIX) $(FC) $(FFLAGS)$(DEBUG) $(INCS) -c main$(SUFFIX) xcgrad.o: xcgrad$(SUFFIX) $(FC) $(FFLAGS) $(INLINE) $(INCS) -c xcgrad$(SUFFIX) xcspin.o: xcspin$(SUFFIX) $(FC) $(FFLAGS) $(INLINE) $(INCS) -c xcspin$(SUFFIX) makeparam.o: makeparam$(SUFFIX) $(FC) $(FFLAGS)$(DEBUG) $(INCS) -c makeparam$(SUFFIX) makeparam$(SUFFIX): makeparam.F main.F # # MIND: I do not have a full dependency list for the include # and MODULES: here are only the minimal basic dependencies # if one strucuture is changed then touch_dep must be called # with the corresponding name of the structure # base.o: base.inc base.F mgrid.o: mgrid.inc mgrid.F constant.o: constant.inc constant.F lattice.o: lattice.inc lattice.F setex.o: setexm.inc setex.F pseudo.o: pseudo.inc pseudo.F mkpoints.o: mkpoints.inc mkpoints.F wave.o: wave.F nonl.o: nonl.inc nonl.F nonlr.o: nonlr.inc nonlr.F $(OBJ_HIGH): $(CPP) $(FC) $(FFLAGS) $(OFLAG_HIGH) $(INCS) -c $*$(SUFFIX) $(OBJ_NOOPT): $(CPP) $(FC) $(FFLAGS) $(INCS) -c $*$(SUFFIX) fft3dlib_f77.o: fft3dlib_f77.F $(CPP) $(F77) $(FFLAGS_F77) -c $*$(SUFFIX) .F.o: $(CPP) $(FC) $(FFLAGS) $(OFLAG) $(INCS) -c $*$(SUFFIX) .F$(SUFFIX): $(CPP) $(SUFFIX).o: $(FC) $(FFLAGS) $(OFLAG) $(INCS) -c $*$(SUFFIX) # special rules #----------------------------------------------------------------------- # these special rules have been tested for ifc.11 and ifc.12 only fft3dlib.o : fft3dlib.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) fft3dfurth.o : fft3dfurth.F $(CPP) $(FC) -FR -lowercase -O1 -c $*$(SUFFIX) fftw3d.o : fftw3d.F $(CPP) $(FC) -FR -lowercase -O1 $(INCS) -c $*$(SUFFIX) fftmpi.o : fftmpi.F $(CPP) $(FC) -FR -lowercase -O1 -c $*$(SUFFIX) fftmpiw.o : fftmpiw.F $(CPP) $(FC) -FR -lowercase -O1 $(INCS) -c $*$(SUFFIX) wave_high.o : wave_high.F $(CPP) $(FC) -FR -lowercase -O1 -c $*$(SUFFIX) radial.o : radial.F $(CPP) $(FC) -FR -lowercase -O1 -c $*$(SUFFIX) # the following rules are probably no longer required (-O3 seems to work) wave.o : wave.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) paw.o : paw.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) cl_shift.o : cl_shift.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) us.o : us.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) LDApU.o : LDApU.F $(CPP) $(FC) -FR -lowercase -O2 -c $*$(SUFFIX) |
2楼2014-12-20 12:35:00
magicmonk
至尊木虫 (著名写手)
- 应助: 172 (高中生)
- 金币: 11451.3
- 红花: 24
- 帖子: 1124
- 在线: 3108.8小时
- 虫号: 1191315
- 注册: 2011-01-17
- 专业: 理论和计算化学
4楼2014-12-20 16:11:18
5楼2014-12-20 20:41:51
20