| 查看: 842 | 回复: 0 | |||
jglee646金虫 (小有名气)
|
[交流]
【求助】VASP并行效率相当差，如何解决？
|
|
vasp4.6，编译器是intel fortran 9.0.136,mkl 9.0，并行工具mpich 1.2.7-p1 单机8核并行时，cpu使用率接近100%。双机并行时，cpu只占30-40%，四机并行时，每个cpu使用率降到了个位数。求解决方案！ makefile如下 .SUFFIXES: .inc .f .f90 .F #----------------------------------------------------------------------- # Makefile for Intel Fortran compiler for P4 systems # # The makefile was tested only under Linux on Intel platforms # (Suse 5.3, libc 6 <-> glibc 2.X) # # it might be required to change some of library pathes, since # LINUX installation vary a lot # Hence check ***ALL**** options in this makefile very carefully #----------------------------------------------------------------------- # # Mind that some Linux distributions (Suse 6.1) have a bug in # libm causing small errors in the error-function (total energy # is therefore wrong by about 1meV/atom). The recommended # solution is to update libc. # # BLAS must be installed on the machine # there are several options: # 1) very slow but works: # retrieve the lapackage from ftp.netlib.org # and compile the blas routines (BLAS/SRC directory) # please use g77 or f77 for the compilation. When I tried to # use pgf77 or pgf90 for BLAS, VASP hang up when calling # ZHEEV (however this was with lapack 1.1 now I use lapack 2.0) # 2) most desirable: get an optimized BLAS # for a list of optimized BLAS try # http://www.kachinatech.com/~hjjou/scilib/opt_blas.html # # the two most reliable packages around are presently: # 3a) Intels own optimised BLAS (PIII, P4, Itanium) # http://developer.intel.com/software/products/mkl/ # this is really excellent when you use Intel CPU's # # 3b) or obtain the atlas based BLAS routines # http://math-atlas.sourceforge.net/ # you certainly need atlas on the Athlon, since the mkl # routines are not optimal on the Athlon. # If you want to use atlas based BLAS, check the lines around LIB= # # 3c) brand new and mindblowing fast SSE (4 GFlops on P4, 2.53 GHz) # Kazushige Goto's BLAS # http://www.cs.utexas.edu/users/kgoto/signup_first.html # # #----------------------------------------------------------------------- # all CPP processed fortran files have the extension .f SUFFIX=.f90 #----------------------------------------------------------------------- # fortran compiler and linker #----------------------------------------------------------------------- #FC=ifc # fortran linker #FCL=$(FC) #----------------------------------------------------------------------- # whereis CPP ?? (I need CPP, can't use gcc with proper options) # that's the location of gcc for SUSE 5.3 # # CPP_ = /usr/lib/gcc-lib/i486-linux/2.7.2/cpp -P -C # # that's probably the right line for some Red Hat distribution: # # CPP_ = /usr/lib/gcc-lib/i386-redhat-linux/2.7.2.3/cpp -P -C # # SUSE X.X, maybe some Red Hat distributions: CPP_ = ./preprocess <$*.F | /usr/bin/cpp -P -C -traditional >$*$(SUFFIX) #----------------------------------------------------------------------- # possible options for CPP: # NGXhalf charge density reduced in X direction # wNGXhalf gamma point only reduced in X direction # avoidalloc avoid ALLOCATE if possible # IFC work around some IFC bugs # CACHE_SIZE 1000 for PII,PIII, 5000 for Athlon, 8000-12000 P4 # RPROMU_DGEMV use DGEMV instead of DGEMM in RPRO (usually faster) # RACCMU_DGEMV use DGEMV instead of DGEMM in RACC (faster on P4) #----------------------------------------------------------------------- #CPP = $(CPP_) -DHOST=\"LinuxIFC\" \ # -Dkind8 -DNGXhalf -DCACHE_SIZE=12000 -DPGF90 -Davoidalloc \ # -DRPROMU_DGEMV -DRACCMU_DGEMV #----------------------------------------------------------------------- # general fortran flags (there must a trailing blank on this line) #----------------------------------------------------------------------- FFLAGS = -FR -lowercase #----------------------------------------------------------------------- # optimization # we have tested whether higher optimisation improves performance # -xW SSE2 optimization # -axW SSE2 optimization, but also generate code executable on all mach. # -tpp7 P4 optimization # -prefetch #----------------------------------------------------------------------- OFLAG=-O3 -xW -tpp7 OFLAG_HIGH = $(OFLAG) OBJ_HIGH = OBJ_NOOPT = DEBUG = -FR -O0 INLINE = $(OFLAG) #----------------------------------------------------------------------- # the following lines specify the position of BLAS and LAPACK # on P4, VASP works fastest with Intels mkl performance library # so that's what I recommend #----------------------------------------------------------------------- # Atlas based libraries #ATLASHOME= $(HOME)/archives/BLAS_OPT/ATLAS/lib/Linux_P4SSE2/ #BLAS= -L$(ATLASHOME) -lf77blas -latlas # use specific libraries (default library path points to other libraries) #BLAS= $(ATLASHOME)/libf77blas.a $(ATLASHOME)/libatlas.a # use the mkl Intel libraries for p4 (www.intel.com) # mkl.5.1 # set -DRPROMU_DGEMV -DRACCMU_DGEMV in the CPP lines BLAS=-L/hptc_cluster/intel/mkl/9.0/lib/em64t -lmkl_em64t -lmkl -lpthread # mkl.5.2 requires also to -lguide library # set -DRPROMU_DGEMV -DRACCMU_DGEMV in the CPP lines #BLAS=-L/opt/intel/mkl/lib/32 -lmkl_p4 -lguide -lpthread # even faster Kazushige Goto's BLAS # http://www.cs.utexas.edu/users/kgoto/signup_first.html #BLAS= /opt/libs/libgoto/libgoto_p4_512-r0.6.so # LAPACK, simplest use vasp.4.lib/lapack_double #LAPACK= ../vasp.4.lib/lapack_double.o # use atlas optimized part of lapack #LAPACK= ../vasp.4.lib/lapack_atlas.o -llapack -lcblas # use the mkl Intel lapack LAPACK=-L/hptc_cluster/intel/mkl/9.0/lib/em64t -lmkl_lapack -lguide -lpthread #----------------------------------------------------------------------- LIB = -L../vasp.4.lib -ldmy \ ../vasp.4.lib/linpack_double.o $(LAPACK) \ $(BLAS) # options for linking (for compiler version 6.X) nothing is required #LINK = # compiler version 7.0 generates some vector statments which are located # in the svml library, add the LIBPATH and the library (just in case) #LINK = -L/opt/intel/compiler70/ia32/lib/ -lsvml #----------------------------------------------------------------------- # fft libraries: # VASP.4.6 can use fftw.30 (http://www.fftw.org) # since this version is faster on P4 machines, we recommend to use it #----------------------------------------------------------------------- #FFT3D = fft3dfurth.o fft3dlib.o #FFT3D = fftw3d.o fft3dlib.o /opt/libs/fftw-3.0/lib/libfftw3.a #======================================================================= # MPI section, uncomment the following lines # # one comment for users of mpich or lam: # You must *not* compile mpi with g77/f77, because f77/g77 # appends *two* underscores to symbols that contain already an # underscore (i.e. MPI_SEND becomes mpi_send__). The pgf90 # compiler however appends only one underscore. # Precompiled mpi version will also not work !!! # # We found that mpich.1.2.1 and lam-6.5.X are stable # mpich.1.2.1 was configured with # ./configure -prefix=/usr/local/mpich_nodvdbg -fc="pgf77 -Mx,119,0x200000" \ # -f90="pgf90 -Mx,119,0x200000" \ # --without-romio --without-mpe -opt=-O \ # # lam was configured with the line # ./configure -prefix /usr/local/lam-6.5.X --with-cflags=-O -with-fc=pgf90 \ # --with-f77flags=-O --without-romio # # lam was generally faster and we found an average communication # band with of roughly 160 MBit/s (full duplex) # # please note that you might be able to use a lam or mpich version # compiled with f77/g77, but then you need to add the following # options: -Msecond_underscore (compilation) and -g77libs (linking) # # !!! Please do not send me any queries on how to install MPI, I will # certainly not answer them !!!! #======================================================================= #----------------------------------------------------------------------- # fortran linker for mpi: if you use LAM and compiled it with the options # suggested above, you can use the following line #----------------------------------------------------------------------- FC=/hptc_cluster/mpich-1.2.7p1/bin/mpif90 FCL=$(FC) #----------------------------------------------------------------------- # additional options for CPP in parallel version (see also above): # NGZhalf charge density reduced in Z direction # wNGZhalf gamma point only reduced in Z direction # scaLAPACK use scaLAPACK (usually slower on 100 Mbit Net) #----------------------------------------------------------------------- CPP = $(CPP_) -DMPI -DHOST=\"LinuxIFC\" -DIFC \ -Dkind8 -DNGZhalf -DCACHE_SIZE=4000 -DPGF90 -Davoidalloc \ -DMPI_BLOCK=500 -DPROC_GROUP=8 \ -DRPROMU_DGEMV -DRACCMU_DGEMV #----------------------------------------------------------------------- # location of SCALAPACK # if you do not use SCALAPACK simply uncomment the line SCA #----------------------------------------------------------------------- #BLACS=$(HOME)/archives/SCALAPACK/BLACS/ #SCA_=$(HOME)/archives/SCALAPACK/SCALAPACK #SCA= $(SCA_)/libscalapack.a \ # $(BLACS)/LIB/blacsF77init_MPI-LINUX-0.a $(BLACS)/LIB/blacs_MPI-LINUX-0.a $(BLACS)/LIB/blacsF77init_MPI-LINUX-0.a #SCA= #----------------------------------------------------------------------- # libraries for mpi #----------------------------------------------------------------------- #LIB = -L../vasp.4.lib -ldmy \ # ../vasp.4.lib/linpack_double.o $(LAPACK) \ # $(SCA) $(BLAS) # FFT: fftmpi.o with fft3dlib of Juergen Furthmueller FFT3D = fftmpi.o fftmpi_map.o fft3dlib.o # fftw.3.0 is slighly faster and should be used if available #FFT3D = fftmpiw.o fftmpi_map.o fft3dlib.o /opt/libs/fftw-3.0/lib/libfftw3.a #----------------------------------------------------------------------- # general rules and compile lines #----------------------------------------------------------------------- BASIC= symmetry.o symlib.o lattlib.o random.o SOURCE= base.o mpi.o smart_allocate.o xml.o \ constant.o jacobi.o main_mpi.o scala.o \ asa.o lattice.o poscar.o ini.o setex.o radial.o \ pseudo.o mgrid.o mkpoints.o wave.o wave_mpi.o $(BASIC) \ nonl.o nonlr.o dfast.o choleski2.o \ mix.o charge.o xcgrad.o xcspin.o potex1.o potex2.o \ metagga.o constrmag.o pot.o cl_shift.o force.o dos.o elf.o \ tet.o hamil.o steep.o \ chain.o dyna.o relativistic.o LDApU.o sphpro.o paw.o us.o \ ebs.o wavpre.o wavpre_noio.o broyden.o \ dynbr.o rmm-diis.o reader.o writer.o tutor.o xml_writer.o \ brent.o stufak.o fileio.o opergrid.o stepver.o \ dipol.o xclib.o chgloc.o subrot.o optreal.o davidson.o \ edtest.o electron.o shm.o pardens.o paircorrection.o \ optics.o constr_cell_relax.o stm.o finite_diff.o \ elpol.o setlocalpp.o INC= vasp: $(SOURCE) $(FFT3D) $(INC) main.o rm -f vasp $(FCL) -o vasp $(LINK) main.o $(SOURCE) $(FFT3D) $(LIB) makeparam: $(SOURCE) $(FFT3D) makeparam.o main.F $(INC) $(FCL) -o makeparam $(LINK) makeparam.o $(SOURCE) $(FFT3D) $(LIB) zgemmtest: zgemmtest.o base.o random.o $(INC) $(FCL) -o zgemmtest $(LINK) zgemmtest.o random.o base.o $(LIB) dgemmtest: dgemmtest.o base.o random.o $(INC) $(FCL) -o dgemmtest $(LINK) dgemmtest.o random.o base.o $(LIB) ffttest: base.o smart_allocate.o mpi.o mgrid.o random.o ffttest.o $(FFT3D) $(INC) $(FCL) -o ffttest $(LINK) ffttest.o mpi.o mgrid.o random.o smart_allocate.o base.o $(FFT3D) $(LIB) kpoints: $(SOURCE) $(FFT3D) makekpoints.o main.F $(INC) $(FCL) -o kpoints $(LINK) makekpoints.o $(SOURCE) $(FFT3D) $(LIB) clean: -rm -f *.f *.o *.L ; touch *.F main.o: main$(SUFFIX) $(FC) $(FFLAGS)$(DEBUG) $(INCS) -c main$(SUFFIX) xcgrad.o: xcgrad$(SUFFIX) $(FC) $(FFLAGS) $(INLINE) $(INCS) -c xcgrad$(SUFFIX) xcspin.o: xcspin$(SUFFIX) $(FC) $(FFLAGS) $(INLINE) $(INCS) -c xcspin$(SUFFIX) makeparam.o: makeparam$(SUFFIX) $(FC) $(FFLAGS)$(DEBUG) $(INCS) -c makeparam$(SUFFIX) makeparam$(SUFFIX): makeparam.F main.F # # MIND: I do not have a full dependency list for the include # and MODULES: here are only the minimal basic dependencies # if one strucuture is changed then touch_dep must be called # with the corresponding name of the structure # base.o: base.inc base.F mgrid.o: mgrid.inc mgrid.F constant.o: constant.inc constant.F lattice.o: lattice.inc lattice.F setex.o: setexm.inc setex.F pseudo.o: pseudo.inc pseudo.F poscar.o: poscar.inc poscar.F mkpoints.o: mkpoints.inc mkpoints.F wave.o: wave.inc wave.F nonl.o: nonl.inc nonl.F nonlr.o: nonlr.inc nonlr.F $(OBJ_HIGH): $(CPP) $(FC) $(FFLAGS) $(OFLAG_HIGH) $(INCS) -c $*$(SUFFIX) $(OBJ_NOOPT): $(CPP) $(FC) $(FFLAGS) $(INCS) -c $*$(SUFFIX) fft3dlib_f77.o: fft3dlib_f77.F $(CPP) $(F77) $(FFLAGS_F77) -c $*$(SUFFIX) .F.o: $(CPP) $(FC) $(FFLAGS) $(OFLAG) $(INCS) -c $*$(SUFFIX) .F$(SUFFIX): $(CPP) $(SUFFIX).o: $(FC) $(FFLAGS) $(OFLAG) $(INCS) -c $*$(SUFFIX) # special rules #----------------------------------------------------------------------- # -tpp5|6|7 P, PII-PIII, PIV # -xW use SIMD (does not pay of on PII, since fft3d uses double prec) # all other options do no affect the code performance since -O1 is used fft3dlib.o : fft3dlib.F $(CPP) $(F77) -FR -lowercase -O1 -tpp7 -xW -prefetch- -prev_div -unroll0 -e95 -vec_report3 -c $*$(SUFFIX) fft3dfurth.o : fft3dfurth.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) radial.o : radial.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) symlib.o : symlib.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) symmetry.o : symmetry.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) dynbr.o : dynbr.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) us.o : us.F $(CPP) $(F77) -FR -lowercase -O1 -c $*$(SUFFIX) wave.o : wave.F $(CPP) $(F77) -FR -lowercase -O0 -c $*$(SUFFIX) LDApU.o : LDApU.F $(CPP) $(F77) -FR -lowercase -O2 -c $*$(SUFFIX) |
回复此楼» 猜你喜欢
fileCode有新解读？
已经有13人回复
-
售SCI一区文章，我:8.O.55.1.O.54,科目齐全,可伽急
已经有5人回复
-
售SCI一区T0P文章，我:8.O.55.1.O54,科目全,可伽急
已经有4人回复
-
售SCI-T0P文章，我:8O.5.5.1.O.54,科目齐全,可+急
已经有4人回复
-
售SCI一区文章，我:8.O.55.1.O.54,科目齐全,可伽急
已经有3人回复
-
基金中了
已经有14人回复
-
关于filecode
已经有7人回复
-
娱乐
已经有6人回复
-
化学口download_prp&fileCode的固定段好像这几天一直没变，有变的大神么？
已经有4人回复
-
关于代码变化问题，想知道的进来
已经有12人回复
找到一些相关的精华帖子，希望有用哦~
VASP并行计算时，提示“超过CPU时限”而退出计算。请问如何解决？
已经有11人回复
- VASP并行计算出错
已经有9人回复
- 教教我怎样实现VASP并行计算
已经有3人回复
- Vasp并行编译出错
已经有7人回复
- vasp并行问题及NEB计算出错
已经有5人回复
- vasp并行测试时出错。
已经有7人回复
- vasp 并行运算出错
已经有7人回复
- vasp5.2 并行编译如何修改makefile
已经有5人回复
- 【求助】vasp的编译及运行效率问题
已经有11人回复
- 【求助】单机多核并行VASP，如何编译？
已经有5人回复
- 【求助】vasp无法并行计算
已经有25人回复
- 【求助】请问vasp最大使用的核数？也就说多少核以上vasp的效率就不再增大？
已经有15人回复
- 【求助】并行vasp出了问题（解决）
已经有3人回复
点击这里搜索更多相关资源科研从小木虫开始，人人为我，我为人人
100