EMPIRE: a highly parallel semiempirical molecular orbital program: 2: periodic boundary conditions
The increasing role of quantum chemical calculations in drug and materials design has led to a demand for methods that can describe the electronic structures of large and complex systems. Semiempirical methods based on the neglect of diatomic differential overlap (NDDO) approximation (e.g., the MNDO [1, 2], MNDO/d , AM1 , AM1* , and PMx [6, 7, 8] methods) are important representatives of such approaches. Many of these methods have been implemented in the massively parallel program EMPIRE , which makes the full quantum-mechanical treatment of systems containing 100,000 atoms or more possible.
Periodic boundary conditions (PBC) enable quantum chemical programs to treat condensed-phase systems, such as proteins in a periodic water box or solids. This allows molecular materials to be studied in their “native” environment, instead of comparing experimental bulk properties with gas-phase monomer calculations. For semiempirical methods, the most practical way of implementing PBC is the cyclic-cluster approach [10, 11, 12] in which the system is approximated by a supercell and by imposing Born–von Karman boundary conditions . Using a large unit cell allows the calculation to be performed entirely in real space. This is easily affordable because of the generally low computational cost of NDDO calculations. The main advantage of this technique is that program features like the calculation of local properties  or excited states are directly transferable from nonperiodic calculations . We have, for example, used periodic EMPIRE calculations to model amorphous carbon .
EMPIRE, which was especially designed for calculations on systems with very many atoms, is also suitable for use on systems with very large unit cells (e.g., disordered and amorphous systems). EMPIRE can, for example, be used in combination with a classical molecular dynamics (MD) code to perform electronic structure calculations on snapshots from an MD run on a periodic system. In the first section of this paper, we discuss the implementation of periodic boundary conditions in EMPIRE. In the second, the program performance is discussed briefly. Finally, some exemplary applications of large-scale periodic NDDO calculations are shown.
Periodic calculations in EMPIRE are performed entirely in real space. Therefore, no major changes to the NDDO SCF algorithm were required. Only small adjustments are necessary in the treatment of two-electron two-center integrals: the exchange energy and electrostatic interactions. These adjustments will be discussed below. For more background information, we refer the reader to [10, 12].
Two-electron two-center integrals
Exchange interaction energy
The next adjustment is required with respect to the two-electron two-center exchange integrals hμν, which appear in the Fock matrix. These terms depend on the density matrix elements Pμν. In a periodic calculation, the exchange interactions for an orbital centered on a given atom are only evaluated within the Wigner–Seitz cell surrounding it. The net result is the neglect of very weak exchange interactions with distant electrons, which causes no loss in accuracy .
MNDO-like NDDO methods describe electrostatic electron–electron and electron–core interactions using multipole–multipole interactions. In a periodic system, small interactions with an infinite number of distant charges lead to unphysical results. This can be alleviated by introducing a simple, one-parameter screening function.
The effect of scaling the multipole size can be evaluated by considering three different scenarios for an interaction between a point-charge and a dipole. The rigorous but unpractical solution is to scale the positions of the constituent point charges of the dipole individually, whereas the simplest approach would be to scale only the center of the dipole. Finally, we can scale the center of the dipole according to (4) and the distances of the distributed point charges to their center by (7).
Setting up periodic calculations
The electrostatic screening parameter can be modified via the keyword ScreeningR, which sets the value of 2α in Å. The default is 30.0 Å. This conservative cutoff corresponds to the MOPAC default. Lower cutoffs result in lower computational cost, but whether the heat of formation is affected by the change should be checked. The energy convergences and computational costs of different values of α are shown for diamond and ZnO in the “Electronic supplementary material” (ESM, Figs. S1 and S2).
Single-node open MP scaling
Wall-clock times for AM1-SCF calculations performed with the single-node OMP version of EMPIRE. These calculations were performed on a node consisting of two quad-core 2.83-GHz Intel® Xenon® E5440 processors with 8 GB of memory. No hyperthreading was used
Number of orbitals
Wall-clock time (s) for N cores
N = 1
N = 2
N = 3
N = 4
N = 5
N = 6
N = 7
N = 8
Wall-clock times for AM1 SCF calculations performed with the multi-node hybrid MPI/OMP version of EMPIRE. Each node was equipped with two six-core Intel® Xeon® 5650 “Westmere” chips; the nodes were connected by an Infiniband interconnect fabric with 40 Gbit/s bandwith per link and direction. We used two MPI tasks per node and six OMP threads for each. No hyperthreading was used
Number of orbitals
Wall-clock time (s) for N nodes
N = 4
N = 8
N = 16
N = 24
N = 32
Multi-node hybrid OMP/MPI scaling
The scaling of the hybrid OMP/MPI multi-node version of EMPIRE was tested on the LiMa cluster at the Regionales Rechenzentrum Erlangen. Here, we used differently sized diamond unit cells from C1,728 to C13,824. Please note that very large unit cells also require large amounts of memory, especially because the integrals are stored, since their calculation is relatively expensive in periodic calculations. Therefore, it is not possible to use the same reference number of nodes when determining the scaling for these systems. The speedup is always relative to the lowest number of nodes feasible for a given system. Optimizing the SCF procedure for periodic calculations may improve the performance of EMPIRE on fewer nodes. As it is, the calculations scale very impressively up to twice the minimum number of nodes. Further increasing the number of nodes leads to a plateau.
The application of NDDO methods to crystalline materials has been thoroughly tested and evaluated by Stewart, and will therefore not be discussed here in any detail . Instead, we would like to focus on two aspects unique to EMPIRE: firstly, the calculation of local properties; secondly, the fact that even unit cells with thousands of atoms can be treated easily.
A local property is any property that can be derived from the wavefunction of a structure and mapped onto a real-space grid, such as the electron density and the molecular electrostatic potential (MEP). These can be calculated with most electronic structure codes. EMPIRE (in combination with an auxiliary program) gives access to several additional local properties derived from molecular orbitals and their energies. These are the local electron affinity (EAL), ionization energy (IEL), electronegativity, and hardness, which have been used for biochemical QSPR studies and to predict the electron-transport properties of nanostructures [19, 20, 21, 22, 23, 24, 25, 26, 27].
Low IEL values (shown in blue) correspond to electron-donating/hole-conducting pathways, whereas high IEL values (shown in red) represent energy barriers. In Fig. 6, the IEL maps look vastly different depending on the orientation of the volume slice. This is in line with experimental reports, which show that the field-effect mobilities in rubrene single crystals depend strongly on the orientation of the contacts [33, 34].
Large unit cells
Note that periodic calculations of this size push double-precision (64-bit) arithmetic to its limit, since many small values are summed to a very large result during the energy summation. To avoid numerical inaccuracies for large systems, this step is performed in quadruple precision (128-bit), and special care is taken in the ordering of the summands.
The resulting HDF5 binary wavefunction file has a size of 21 GB and can be used to calculate local property maps. The molecular electrostatic potential across the membrane is shown in Fig. 7 (right). This clearly visualizes the polar water layer and head groups and the nonpolar lipid bilayer. Such calculations could, for instance, be used to predict the permeability of membranes to different chemicals.
We have implemented periodic boundary conditions in the massively parallel semiempirical molecular orbital theory code EMPIRE. The standard SCF procedure of EMPIRE reliably converges the wavefunctions of a broad range of periodic systems, including covalent, ionic, and molecular crystals and surfaces as well as disordered biological systems such as a lipid bilayer. Like the nonperiodic version of EMPIRE, the program is parallelized in the single-node version via open MP, and in the multi-node version via a hybrid open MP/MPI approach.
The single-node version was shown to perform well for calculations on unit cells containing between 64 and 512 atoms, and to scale very efficiently on up to eight cores. The multi-node version allows systems with tens of thousands of atoms to be treated; the largest system described here consisted of 25,088 atoms. The program scaling is similar to that observed for nonperiodic calculations with EMPIRE .
This work was supported by the Deutsche Forschungsgemeinschaft as part of SFB 953 “Synthetic Carbon Allotropes” and EXC 315 “Engineering of Advanced Materials”, and by the Bavarian Government within the “Solar Technologies go Hybrid” (SolTech) initiative. JTM was supported by a Beilstein Fellowship.
- 1.Dewar MJS, Thiel W (1977) Ground states of molecules, 38. The MNDO method. Approximations and parameters. J Am Chem Soc 99:4899. doi:10.1021/ja00457a004
- 2.Dewar MJS, Thiel W (1977) Ground states of molecules, 39. MNDO results for molecules containing hydrogen, carbon, nitrogen, and oxygen. J Am Chem Soc 99:4907. doi:10.1021/ja00457a005
- 3.Thiel W, Voityuk A (1996) Extension of MNDO to d-orbitals—parameters and results for the second-row elements and for the zinc group. J Phys Chem 100:616. doi:10.1021/jp952148o
- 4.Dewar MJS, Zoebisch EG, Healy EF, Stewart JJP (1985) AM1: a new general purpose quantum mechanical model. J Am Chem Soc 107:3902. doi:10.1021/ja00299a024
- 5.Winget P, Horn AC, Selçuki C, Martin B, Clark T (2003) AM1* parameters for phosphorus, sulfur and chlorine. J Mol Model 9:408. doi:10.1007/s00894-003-0156-7
- 6.Stewart JJP (1989) Optimization of parameters for semi-empirical methods I: method. J Comput Chem 10:209–220. doi:10.1002/jcc.540100208
- 7.Stewart JJP (2007) Optimization of parameters for semiempirical methods V: modification of NDDO approximations and application to 70 elements. J Mol Model 13:1173. doi:10.1007/s00894-007-0233-4
- 8.Stewart JJP (2013) Optimization of parameters for semiempirical methods VI: more modifications to the NDDO approximations and re-optimization of parameters. J Mol Model 19:1–32. doi:10.1007/s00894-012-1667-x
- 9.Hennemann M, Clark T (2014) EMPIRE: a highly parallel semiempirical molecular orbital program: 1: self-consistent field calculations. J Mol Model 20:2331. doi:10.1007/s00894-014-2331-4
- 10.Perkins PG, Stewart JJP (1980) Cluster model for solids. J Chem Soc Faraday Trans 2 76:520. doi: 10.1039/F29807600520
- 11.Bredow T, Geudtner G, Jug K (2001) Development of the cyclic cluster approach for ionic systems. J Comput Chem 22:89–101. doi:10.1002/1096-987X(20010115)22:1<89::AID-JCC9>3.0.CO;2-7
- 13.Born M, Kármán T (1912) Über Schwingungen in Raumgittern. Phys Z 13:297–309Google Scholar
- 15.Gadaczek I, Hintze KJ, Bredow T (2012) Periodic calculations of excited states for solids using a semiempirical approach. Phys Chem Chem Phys 14:741–750. doi:10.1039/c1cp22871d
- 16.Margraf JT, Strauss V, Guldi DM, Clark T (2015) The electronic structure of amorphous carbon nanodots. J Phys Chem B 119:ASAP. doi:10.1021/jp510620j
- 17.Stewart JJP (2015) MOPAC online manual. Stewart Computational Chemistry, Colorado Springs. http://openmopac.net/manual/Solids_cluster.html. Accessed 5 April 2015
- 19.Ehresmann B, de Groot MJ, Alex A, Clark T (2004) New molecular descriptors based on local properties at the molecular surface and a boiling-point model derived from them. J Chem Inf Comput Sci 44:658–668. doi:10.1021/ci034215e
- 20.Güssregen A, Matter H, Hessler G, Müller M, Schmidt F, Clark T (2012) 3D-QSAR based on quantum-chemical molecular fields: towards an improved description of halogen interactions. J Chem Inf Model 52:2441–2453. doi:10.1021/ci300253z
- 21.Ehresmann B, de Groot MJ, Clark T (2005) Surface-integral QSPR models: local energy properties. J Chem Inf Model 45:1053–1060. doi:10.1021/ci050025n
- 22.Clark T (2010) The local electron affinity for non-minimal basis sets. J Mol Model 16:1231–1238. doi:10.1007/s00894-009-0607-x
- 23.ElKerdawy A, Wick CR, Hennemann M, Clark T (2012) Predicting the sites and energies of noncovalent intermolecular interactions using local properties. J Chem Inf Model 52:1061–1071. doi:10.1021/ci300095x
- 24.Clark T, Halik M, Hennemann M, Jäger CM (2013) Simulating “soft” electronic devices. In: Hicks MG, Kettner C (eds) Molecular engineering and control. Logos, Berlin, pp 137–150Google Scholar
- 26.Jäger CM, Schmaltz T, Novak M, Khassanov A, Vorobiev A, Hennemann M, Krause A, Dietrich H, Zahn D, Hirsch A, Halik M, Clark T (2013) Improving the charge transport in self-assembled monolayer field-effect transistors—from theory to devices. J Am Chem Soc 135:4893–4900. doi:10.1021/ja401320n
- 27.Schubert C, Margraf JT, Clark T, Guldi DM (2014) Molecular wires—impact of π-conjugation and implementation of molecular bottlenecks. Chem Soc Rev 44:988–998. doi:10.1039/c4cs00262h
- 29.Kovacik R, Meyer B, Marx D (2007) F centers versus dimer vacancies on ZnO surfaces: characterization by STM and STS calculations. Angew Chem Int Ed 46:4894–4897. doi:10.1002/anie.200604399
- 30.Horn AHC, Lin J-H, Clark T (2005) Multipole electrostatic model for MNDO-like techniques with minimal valence spd-basis sets. Theor Chem Acc 114:159–168; erratum: (2007) 117:461–465Google Scholar
- 31.Atienza C, Martin N, Wielepolski M, Haworth N, Clark T, Guldi DM (2006) Tuning electron transfer through p-phenyleneethylene molecular wires. Chem Commun 30:3202–3204. doi: 10.1039/B603149H
- 32.Bauer T (2015) Multi-Agenten-Simulation organischer Feldeffekttransistoren. Ph.D. thesis. Universität Erlangen-Nürnberg, ErlangenGoogle Scholar
- 33.Ling MM, Reese C, Briseno AL, Bao Z (2007) Non-destructive probing of the anisotropy of field-effect mobility in the rubrene single crystal. Synth Met 157:257–260. doi:10.1016/j.synthmet.2007.02.004
- 34.Reese C, Bao Z (2007) High-resolution measurement of the anisotropy of charge transport in single crystals. Adv Mater 19:4535–4538. doi:10.1002/adma.200701139
- 35.Jämbeck JPM, Lyubartsev AP (2012) Derivation and systematic validation of a refined all-atom force field for phosphatidylcholine lipids. J Phys Chem B 116:3164–3179. doi:10.1021/jp212503e
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.