24小时热门版块排行榜

返回列表

【奖励】本帖被评价28次，作者wuy069增加金币 22 个

当前只显示满足指定条件的回帖，点击这里查看本话题的所有回帖

wuy069

铁杆木虫 (正式写手)

应助: 20 (小学生)
金币: 7921.9
帖子: 506
在线: 549.1小时
虫号: 760723

[资源] Dalton2011并行编译详记

Dalton 2011 并行编译
      先唠叨几句编译的血泪史吧！这玩意编译起来真是费劲啊，从dalton2.0-cam编译了很多次，从g77,gfortran,pgi,ifort编译器; mpich,mpich2,openmpi,impi等mpi; blas,atals,lapack,mkl等数学库，虽然有些能编译成功，但测试时总有很多例子不能通过。不过我这个人在服务器、系统、软件等方面还是挺能折腾的，不把它弄清楚，始终觉得有什么东西压在心底，很是不舒服。Dalton2011发布后也折腾了一段时间，通过dalton邮件列表也问了些问题，这两天终于算把这并行编译的问题搞定了，测试时串行和并行例子都成功。虽然它很能折腾人，但也让我学到了很多。好了，言归正传，下面就讲Dalton2011的并行编译。
      现在大家所使用的Linux服务器系统几乎都是64位（x84_64或amd64），因而在此讲的几乎都是64位编译器和64位数学库编译的Dalton2011。大家默认编译的mpi都是32位整数的，也能编译成功；但最好用64位整数的mpi。
      下面分别用gfortran/gcc, ifort/icc编译Dalton2011。
1.使用gfortran/gcc, openmpi, atlas with full lapack编译并行的Dalton2011
1）首先说明一点，使用gfortran编译，gfortran版本至少要高于4.3（我用的是RHEL6.1 , gfortran版本为4.4.5），否则你编译时会出现此错误Fatal Error: Can't open module file 'memory_tracker.mod' for reading at (1): No such file or directory。使用gfortran -v  命令查看gfortran的版本。
2）在http://www.open-mpi.org/software/ompi/v1.4/ 下载稳定的openmpi，我使用的是openmpi-1.4.3。使用gfortran/gcc编译64位整数的openmpi：
[wuy@pc2 ~]$ cd tmp/
[wuy@pc2 tmp]$ ls
openmpi-1.4.3.tar.gz
[wuy@pc2 tmp]$ tar -zxvf openmpi-1.4.3.tar.gz
[wuy@pc2 tmp]$ cd openmpi-1.4.3
[wuy@pc2 tmp]$ ./configure --prefix=/home/wuy/mpi/gnu/openmpi-1.4.3 CXX=g++ CC=gcc F77=gfortran FC=gfortran FFLAGS="-m64 -fdefault-integer-8" FCFLAGS="-m64 -fdefault-integer-8" CFLAGS=-m64 CXXFLAGS=-m64
[wuy@pc2 tmp]$ make all install &> make.log &
用gfortran/gcc编译64位整数的openmpi，需要在FFLAGS和FCFLAGS选项中添加-fdefault-integer-8。
在自己用户的根目录下.bashrc中添加openmpi的环境变量
[wuy@pc2 tmp]$ cd $HOME
[wuy@pc2 ~]$ vi .bashrc
在此文件中添加：
#set for openmpi-1.4.3
export PATH=/home/wuy/mpi/gnu/openmpi-1.4.3/bin:$PATH
export LD_LIBRARY_PATH=/home/wuy/mpi/gnu/openmpi-1.4.3:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/home/wuy/mpi/gnu/openmpi-1.4.3/lib:$LD_LIBRARY_PATH
export OMPI_MPIF77="gfortran"
export OMPI_MPIFC="gfortran"
export OMPI_MPICC="gcc"
export OMPI_MPICXX="g++"
#end
保存后退出（:wq），刷新环境变量；
[wuy@pc2 ~]$ source .bashrc
[wuy@pc2 ~]$ which mpif90
~/mpi/gnu/openmpi-1.4.3/bin/mpif90
这时说明就可以使用openmpi了。那如何知道我们编译的是64位整数的openmpi呢？
[wuy@pc2 ~]$ ompi_info -a | grep 'Fort integer size'
   Fort integer size: 8
得到是Fort integer size: 8，说明是64位整数的openmpi；如果得到的是Fort integer size: 4，说明你编译的是32位整数的openmpi。
3）数学库的选择；在此，可以不用自己编译数学库，使用Dalton2011自带的数学库也行。虽然自带的数学库也能编译成功，但是编译后的可执行代码肯定较慢。用mkl数学库也行，但用gfortran编译Dalton2011，我自己不推荐使用mkl数学库。我们可以使用完全支持lapack的atlas库。使用gfortran编译的atlas with full lapack，具体见http://math-atlas.sourceforge.ne ... 00000000|Installing ，见8 Example: Installing ATLAS with full LAPACK on Linux/AMD64。这个虽然是英文的，初步看来比较麻烦，但写的非常详细，按照他的步骤，肯定能安装成功。
我使用的是atlas3.8.2和lapack-3.1.1；安装成功后会有6个库函数，libatlas.a  libcblas.a  libf77blas.a  liblapack.a  libptcblas.a  libptf77blas.a。
在自己用户的根目录下.bashrc中添加，
#set for atlas with full lapack
export INCLUDE=$INCLUDE:/home/wuy/atlas/include
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/wuy/atlas/lib
#end
保存后退出（:wq），刷新环境变量；
[wuy@pc2 ~]$ source .bashrc
4）编译DALTON
[wuy@pc2 ~]$ cd chemsoft/
[wuy@pc2 chemsoft]$ ls
Dalton2011_release_v0.tgz
[wuy@pc2 chemsoft]$ tar -zxvf Dalton2011_release_v0.tgz
[wuy@pc2 chemsoft]$ cd Dalton2011_release
[wuy@pc2 Dalton2011_release]$ cd DALTON/
[wuy@pc2 DALTON]$ ./configure

------------------------------------------------------------------
Configuring the DALTON Makefile.config and "dalton" run script
------------------------------------------------------------------

INFO: Operating system from 'uname -s' : Linux
INFO: Processor type from 'uname -m' : x86_64
No architecture specified, attempting auto-configuration:
This appears to be a -linux architecture. Is this correct? [Y/n] Y

--> Installing DALTON on a -linux computer

Note that 64-bit integers are desirable for Cholesky and very large
scale CI, otherwise the most important effect is that some files will be bigger.

If you choose 64-bit integers, be careful that any system library
routines (incl. MPI) also use 64-bit integers!

Do you want 64-bit integers? [y/N] y
Do you want to install the program in a parallel MPI version? [Y/n] Y

-->WARNING: Makefiles for MPI architecture are difficult to guess
Please compare the generated Makefile.config with local documentation.

Checking for Fortran compiler ...
from this list: mpif90 mpiifort ifort pgf95 pgf90 gfortran g95

Compiler /home/wuy/mpi/gnu/openmpi-1.4.3/bin/mpif90 found, use this compiler? [Y/n] Y

-->Compiler mpif90 found and accepted.
Is backend compiler gfortran ? [Y/n] Y

Checking for C compiler ...
from this list: mpicc  mpiicc icc ecc pgcc gcc

Compiler /home/wuy/mpi/gnu/openmpi-1.4.3/bin/mpicc found, use this compiler? [Y/n] Y

-->Compiler mpicc found and accepted.

Testing existence of libraries in this order:
libacml.a libmkl.so libmkl_p3.a libatlas.a libblas.a
Directory search list for libraries:
  /usr/local/lib/ATLAS /home/wuy/math/atlas3.8.2/stallo-64-i4/lib /home/wuy/atlas/lib /lib /usr/local/lib /usr/lib /usr/local/lib/ATLAS /lib64 /usr/lib64 /usr/local/lib64

Do you want to replace this with your own directory search list? [y/N] N
Found /usr/local/lib/ATLAS/libatlas.a, use it? [Y/n] n
Found /home/wuy/math/atlas3.8.2/stallo-64-i4/lib/libatlas.a, use it? [Y/n] n
Found /home/wuy/atlas/lib/libatlas.a, use it? [Y/n] Y

-->The following mathematical library(ies) will be used:
-L/home/wuy/atlas/lib -llapack -lcblas -lf77blas -latlas

DALTON uses almost 100 Megabytes of static
allocations, in addition to the dynamic allocation.

DALTON has the possibility to reserve an amount of static memory
for storing two-electron integrals in direct and parallel calculations
Storing some or all of the 2-el. integrals in memory will speed up
direct and parallel calculations (and in particular the latter).
NOTE: This will increase the static memory allocation used by DALTON

Would you like to activate the possibility of storing 2-el.int. in memory? [y/N] y
How many MB to use for storing 2-el. integrals? 3072

-->Program will be installed with 3072 MB (402000000 words) used for storing 2-el. integrals

Maximum amount of work memory for dynamic allocations can be changed
at run time with the environment variable WRKMEM (in REAL*8 words = megabytes/8)
or by using the -M option to the run script: "dalton -M mb ..." (in megabytes).
We recommend at least 200 MB work memory,
larger for correlated calculations, but it should for maximum
efficiency NOT exceed available physical memory per CPU in parallel calculations.

How many MB to use as default for work memory (hit return for default of 1000 MB)?

-->Program will be installed with a default work memory of 1000 MB (131000000 words)

-->Current directory is /home/wuy/Dalton2011_release/DALTON

Use default ../bin as installation directory for DALTON binaries and scripts? [Y/n] Y

-->DALTON executable and script will be placed in /home/wuy/Dalton2011_release/bin directory

-->Default basis set directory will be /home/wuy/Dalton2011_release/DALTON/../basis/

Use this directory as default basis set directory? [Y/n] Y

I did not find /work, /scratch, /scr, or /temp. I will use /tmp

-->Job specific directories under $SCRATCH/$USER
-->will be used for temporary files when running DALTON

Use SCRATCH=/tmp as default root scratch space in "dalton" run script? [Y/n] n
Please enter default root scratch directory: /home/wuy/tmp

-->Creating Makefile.config ...
gfortran version 445 prc=x86_64
INFO: Compiling with 64-bit integers.
INFO: Make sure pre-compiled BLAS, MPI etc. libraries are also with 64-bit integers!!!

Proper 64-bit file access detected.

-->Creating the DALTON run-script in /home/wuy/Dalton2011_release/bin

The configuration of DALTON has finished succesfully.
Check compiler flags etc. in Makefile.config and run "make" to get executable.
[wuy@pc2 DALTON]$
这时会产生一个Makefile.config
下面是我的Makefile.config
[wuy@pc2 DALTON]$ cat Makefile.config
ARCH       = linux
#
#
CPPFLAGS    = -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=131000000' -D'INSTALL_MMWORK=1' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=445 -DVAR_INT64 -DIMPLICIT_NONE
F90          = mpif90
CC          = mpicc
LOADER       = mpif90
RM          = rm -f
FFLAGS       = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fbacktrace -fdefault-integer-8
SAFEFFLAGS = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fbacktrace -fdefault-integer-8
CFLAGS       = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1
INCLUDES    = -I../include
MODULES    = -J../modules
LIBS       = -L/home/wuy/atlas/lib -llapack -lcblas -lf77blas -latlas
INSTALLDIR = /home/wuy/chemsoft/Dalton2011_release/bin
PDPACK_EXTRAS = linpack.o eispack.o gp_zlapack.o gp_dlapack.o
GP_EXTRAS    =
AR          = ar
ARFLAGS    = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS  = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet  -noargumants -arguments=number  -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : dalton linuxparallel.x
SAFE_FFLAGS_for_ifort = $(FFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR =
MPI_LIB_PATH =
MPI_LIB       =
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
#             'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .F90 .c .o .i .g .s

.F.o:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F

.F.i:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F > $*.i

.F.g:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F

.F.s:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F

.F90.o:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F90

.F90.i:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F90 > $*.i

.F90.g:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F90

.F90.s:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F90

.c.o:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c

.c.i:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -E $*.c > $*.i

.c.g:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -g -c $*.c

.c.s:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -S -g -c $*.c

从这Makefile.config中就可看出是用64位整数mpi编译，在CPPFLAGS中的-DVR_INT64，以及FFLAGS和SAFEFFLAGS中的-fdefault-integer-8。如果是32位整数mpi编译，需要去掉这些。
还有一点也是需要特别注意的是，在CPPFLAGS中-D'INSTALL_MMWORK=1'的INSTALL_MMWORK一定要小于2G，因为这与一些编译器支持的内存模式有关，要不然最后一步生成可执行文件时链接库会发生问题。这个当时也困惑了我两天。
用gfortran/gcc编译，可以参考我的Makefile.config。
修改完Makefile.config,开始编译。
[wuy@pc2 DALTON]$ make &> make.log &
有什么错误或其他的，可以看make.log中相应部分，也会有提示。
如果编译成功，会在../bin文件夹中生成dalton_mpi.x  dalton.x。
进入DALTON/test中进行测试
[wuy@pc2 test]$ ./TEST parallel
测试所有并行的例子，完成后会产生一个以时间为名的文件夹，里面有个TESTLOG，查看其最后几行，如果出现ALL TESTS ENDED PROPERLY! 恭喜你，编译的并行例子成功。
[wuy@pc2 test]$ ./TEST all
测试所有非并行的例子。同样，如果出现ALL TESTS ENDED PROPERLY!  为正确。
两个都出现ALL TESTS ENDED PROPERLY! 说明编译的非常成功。

5）编译LSDALTON
下面是我的Makefile.config
[wuy@pc2 LSDALTON]$ cat Makefile.config
ARCH       = linux
FMMDIR    = mm
#
#
CPPFLAGS    = -DSYS_LINUX -D_FILE_OFFSET_BITS=64 -D'INSTALL_BASDIR="/home/wuy/chemsoft/Dalton2011_release/basis"' -DGFORTRAN=445 -DVAR_LINSCA -DIMPLICIT_NONE
F77          = gfortran
F90          = gfortran
FLNK          = gfortran
CC          = gcc
RM          = rm -f
FFLAGS       = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store
F90OPTFLAGS = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store -I. -x f95-cpp-input -ffloat-store -fopenmp
SAFEFFLAGS    = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store
CFLAGS       = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1 -ffloat-store -DUSE_UNDERSCORES
INCLUDES    =
LIBS          = -L/home/wuy/atlas/lib -lf77blas -latlas -L/usr/lib/gcc/x86_64-redhat-linux/4.4.4 -lgomp
INSTALLDIR    = /home/wuy/chemsoft/Dalton2011_release/bin
PDPACK_EXTRAS  = linpack.o eispack.o gp_zlapack.o gp_dlapack.o gp_zlapack.o
AR          = ar
ARFLAGS       =  rvs
#
default : linux.x
#
# Suffix rules C
#
.SUFFIXES : .F90 .f90 .F .o .c

.F90.o:
      $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.F90

.f90.o:
      $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.f90

.F.o:
      $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -c $*.F

.c.o:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c
[wuy@pc2 LSDALTON]$
修改完Makefile.config,开始编译。
[wuy@pc2 LSDALTON]$ make &> make.log &
编译成功后，会在../bin中生成lsdalton.x。
同样需要对其进行测试。

2.使用ifort/icc, mkl, openmpi编译并行Dalton2011
1）现在icc也免费了，用intel这套编译器以及数学库编译是大盛其行。去http://software.intel.com/en-us/ ... ftware-development/ 注册获取license和相应的编译器和数学库。如果你是下载的最新ifort12编译器l_fcompxe_intel64_2011.7.256.tgz，就不需用下载mkl了，因为里面已包含最新的mkl数学库了。
   我使用的ifort/icc 11.1, mkl 10.2.5.035；刷新其相应环境变量，在自己用户的根目录下.bashrc中添加（当然也可以直接用命令刷新，使其没有离开登录界面生效），
source /opt/intel/Compiler/11.1/072/bin/ifortvars.sh intel64
source /opt/intel/Compiler/11.1/072/bin/iccvars.sh intel64
source /opt/intel/mkl/10.2.5.035/tools/environment/mklvarsem64t.sh
保存后退出（:wq），刷新环境变量；
[wuy@cluster ~]$ source .bashrc

2）使用ifort/icc编译64位整数的openmpi
[wuy@cluster tmp]$ ls
openmpi-1.4.3.tar.gz
[wuy@cluster tmp]$ tar -zxvf openmpi-1.4.3.tar.gz
[wuy@ cluster tmp]$ cd openmpi-1.4.3
[wuy@ cluster tmp]$ ./configure --prefix=/home/wuy/mpi/intel/openmpi-1.4.3 CXX=icpc CC=icc F77=ifort FC=ifort FFLAGS=-i8 FCFLAGS=-i8
[wuy@pc2 tmp]$ make all install &> make.log &
用ifort/icc编译64位整数的openmpi，需要在FFLAGS和FCFLAGS选项中添加-i8。
在自己用户的根目录下.bashrc中添加openmpi的环境变量
[wuy@ cluster tmp]$ cd $HOME
[wuy@ cluster ~]$ vi .bashrc
在此文件中添加：
#set for openmpi-1.4.3
export PATH=/home/wuy/mpi/intel/openmpi-1.4.3/bin:$PATH
export LD_LIBRARY_PATH=/home/wuy/mpi/intel/openmpi-1.4.3:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/home/wuy/mpi/intel/openmpi-1.4.3/lib:$LD_LIBRARY_PATH
export OMPI_MPIF77="ifort"
export OMPI_MPIFC="ifort"
export OMPI_MPICC="icc"
export OMPI_MPICXX="icc"
#end
保存后退出（:wq），刷新环境变量；
[wuy@ cluster ~]$ source .bashrc
[wuy@ cluster ~]$ which mpif90
~/mpi/intel/openmpi-1.4.3/bin/mpif90
这时说明就可以使用openmpi了。那如何知道我们编译的是64位整数的openmpi呢？
[wuy@ cluster ~]$ ompi_info -a | grep 'Fort integer size'
   Fort integer size: 8
得到是Fort integer size: 8，说明是64位整数的openmpi；

注意：使用impi (intel mpi)时，用ompi_info -a | grep 'Fort integer size'得到的是Fort integer size: 4；并且在impi手册中有这两句话：“Do not use the –i8 option to compile MPI callback functions,” ; “The Fortran module supports 32-bit INTEGER size only.” 因而使用impi编译时，用32位整数编译，即去掉我下面说的CPPFLAGS中的-DVR_INT64，以及FFLAGS和SAFEFFLAGS中的-i8；使用MKL是用LIBS = -L$your_path_to_MKL -lmkl_intel_lp64 -lmkl_sequential -lmkl_core

3）编译DALTON
由于前面比较详细的介绍了gfortran/gcc编译，相应步骤可以参见；
在这里我仅列出我的Makefile.config,以及一些解释；
[wuy@cluster DALTON]$ cat Makefile.config
ARCH       = linux
#
#
CPPFLAGS    = -DVAR_IFORT -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1310000000' -D'INSTALL_MMWORK=1' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DVAR_INT64 -DIMPLICIT_NONE
F90          = mpif90
CC          = mpicc
LOADER       = mpif90
RM          = rm -f
FFLAGS       = -g -O3 -xSSE4.2 -ip -w -fpp -assume byterecl -i8
SAFEFFLAGS = -g -O1 -w -fpp -assume byterecl -i8
CFLAGS       = -g -O3 -xSSE4.2 -ip -restrict -DRESTRICT=restrict
INCLUDES    = -I../include
MODULES    = -module ../modules
LIBS = -L/opt/intel/mkl/10.2.5.035/lib/em64t -Wl,--start-group /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_intel_ilp64.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_sequential.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_core.a -Wl,--end-group -lpthread
INSTALLDIR = /home/wuy/chemsoft/Dalton2011_release/bin
PDPACK_EXTRAS = linpack.o eispack.o gp_dlapack.o gp_zlapack.o
GP_EXTRAS    =
AR          = ar
ARFLAGS    = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS  = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet  -noargumants -arguments=number  -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : dalton linuxparallel.x
SAFE_FFLAGS_for_ifort = $(SAFEFFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR =
MPI_LIB_PATH =
MPI_LIB       =
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
#             'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .F90 .c .o .i .g .s

.F.o:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F

.F.i:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F > $*.i

.F.g:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F

.F.s:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F

.F90.o:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F90

.F90.i:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F90 > $*.i

.F90.g:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F90

.F90.s:
      $(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F90

.c.o:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c

.c.i:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -E $*.c > $*.i

.c.g:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -g -c $*.c

.c.s:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -S -g -c $*.c
这里我用的是至强CPU，因而FFLAGS和CFLAGS使用的是-xSSE4.2；可以根据自己的cpu调整选项。从这Makefile.config也可看出是用64位整数mpi编译，在CPPFLAGS中的-DVR_INT64，以及FFLAGS和SAFEFFLAGS中的-i8。如果是32位整数mpi编译，需要去掉这些。
还有一点也是需要特别注意的是，在CPPFLAGS中-D'INSTALL_MMWORK=1'的INSTALL_MMWORK一定要小于2G，因为这与一些编译器支持的内存模式有关，要不然最后一步生成可执行文件时链接库会发生问题。这个当时也困惑了我两天。

在此，说明下加载mkl库，
use this lib line for MKL (2010/2011 version; 32-bit integers == default; non-threaded MKL):
LIBS = -L$your_path_to_MKL -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
MKL (2010/2011 version; 64-bit integers == non-default; non-threaded MKL)
LIBS = -L$your_path_to_MKL -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core
MKL (2010/2011 version; 64-bit integers == non-default; threaded MKL with gfortran as f90 compiler)
LIBS = -Wl,--start-group -lmkl_gf_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 –lpthread

上面用的是动态链接库，并且动态链接库也有很多优点（编译的执行文件小；可实现进程之间的资源共享；将一些程序升级变得简单，等等）。尽管动态链接有这么多好处，但不知道是我系统问题还是怎么的，一些测试居然说找不到动态链接库（我用ldd明显看到是加载了的，并且刷新了相应变量）。因此，尽管动态链接好处多，最后我还是改为静态链接了。如：LIBS = -L/opt/intel/mkl/10.2.5.035/lib/em64t -Wl,--start-group /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_intel_ilp64.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_sequential.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_core.a -Wl,--end-group –lpthread

测试等参考gfortran编译。

4）编译LSDALTON
我的Makefile.config
ARCH       = linux
FMMDIR    = mm
#
#
CPPFLAGS    = -DSYS_LINUX -D_FILE_OFFSET_BITS=64 -D'INSTALL_BASDIR="/home/wuy/chemsoft/Dalton2011_release/basis"' -DVAR_LINSCA -DIMPLICIT_NONE
F77          = ifort
F90          = ifort
FLNK          = ifort
CC          = icc
RM          = rm -f
FFLAGS       = -O3 -xSSE4.2 -ip -w
F90OPTFLAGS = -O3 -xSSE4.2 -ip -w -fpp1 -openmp
SAFEFFLAGS    = -O2 -w
CFLAGS       = -O3 -xSSE4.2 -ip -restrict -DRESTRICT=restrict -DUSE_UNDERSCORES
INCLUDES    =
LIBS          =  -L/opt/intel/mkl/10.2.5.035/lib/em64t -Wl,--start-group /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_intel_ilp64.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_sequential.a /opt/intel/mkl/10.2.5.035/lib/em64t/libmkl_core.a -Wl,--end-group -lpthread -Vaxlib -Wl,-rpath=/opt/intel/Compiler/11.1/072/lib/intel64 -lguide -lpthread
INSTALLDIR    = /home/wuy/chemsoft/Dalton2011_release/bin
PDPACK_EXTRAS  = linpack.o eispack.o gp_dlapack.o gp_zlapack.o
AR          = ar
ARFLAGS       =  rvs
#
default : linux.x
#
# Suffix rules C
#
.SUFFIXES : .F90 .f90 .F .o .c

.F90.o:
      $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.F90

.f90.o:
      $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.f90

.F.o:
      $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -c $*.F

.c.o:
      $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c

致谢：
首先感谢吴水星师兄在linux服务器以及计算机知识方面的启蒙及教导，感谢温世正师兄在编译器等方面的使用的启蒙以及编译过程中的帮助，也感谢宝德陈齐旺工程师在编译过程中的帮助！


                                                                                                                              吴勇
                                                                                                                  于2011/10/30 19:18 pm

回复此楼