N.18 Correlation energy

The error in Hartree-Fock is due to the single-determinant approximation only. A term like “Hartree-Fock error“ or single-determinantal error is therefore both precise, and immediately understandable by a general audience.

However, it is called correlation energy, and to justify that term, it would have to be both clearer and equally correct mathematically. It fails both requirements miserably. The term correlation energy is clearly confusing and distracting for nonspecialist. But in addition, there does not seem to be any theorem that proves that an independently defined correlation energy is identical to the Hartree-Fock single determinant error. That would not just make the term correlation energy disingenuous, it would make it wrong.

Instead of finding a rigorous theorem, you are lucky if standard textbooks, e.g,. [33,29,45] and typical web references, offer a vague qualitative story why Hartree-Fock underestimates the repulsions if a pair of electrons gets very close. That is a symptom of the disease of having an incomplete function representation, it is not the disease itself. Low-parameter function representations have general difficulty with representing localized effects, whatever their physical source. If you make up a system where the Coulomb force vanishes both at short and at long distance, such correlations do not exist, and Hartree-Fock would still have a finite error.

The kinetic energy is not correct either; what is the correlation in that? Some sources, like [29] and web sources, seem to suggest that these are indirect results of having the wrong correlation energy, whatever correlation energy may be. The idea is apparently, if you would have the electron-electron repulsions exact, you would compute the correct kinetic energy too. That is just like saying, if you computed the correct kinetic energy term, you would compute the correct potential too, so let’s rename the Hartree-Fock error “kinetic energy interaction.” Even if you computed the potential energy correctly, you would still have to convert the wave function to single-determinantal form before evaluating the kinetic energy, otherwise it is not Hartree-Fock, and that would produce a finite error. Phrased differently, there is absolutely no way to get a general wave function correct with a finite number of single-electron functions, whatever corrections you make to the potential energy.

Szabo and Ostlund [45, p. 51ff,61] state that it is called correlation energy since “the motion of electrons with opposite spins is not correlated within the Hartree-Fock approximation.” That is incomprehensible, for one thing since it seems to suggest that Hartree-Fock is exact for excited states with all electrons in the same spin state, which would be ludicrous. In addition, the electrons do not have motion; a stationary wave function is computed, and they do not have spin; all electrons occupy all the states, spin up and down. It is the orbitals that have spin, and the spin-up and spin-down orbitals are most definitely correlated.

However, the authors do offer a clarification; they take a Slater determinant of two opposite spin orbitals, compute the probability of finding the two electrons at given positions and find that it is correlated. They then explain: that’s OK; the exchange requirements do not allow uncorrelated positions. This really helps an engineer trying to figure out why the motion of the two electrons is uncorrelated!

The unrestricted Hartree-Fock solution of the dissociated hydrogen molecule is of this type. Since if one electron is around the left proton, the other is around the right one, and vice versa, many people would call the positions of the electrons strongly correlated. But now we engineers understand that this does not count, because an uncorrelated state in which electron 1 is around the left proton for sure and electron 2 is around the right one for sure is not allowed.

Having done so well, the authors offer us no further guidance how we are supposed to figure out whether or not electrons 1 and 2 are of opposite spin if there are more than two electrons. It is true that if the wave function

\begin{displaymath}
\Psi({\skew0\vec r}_1,{\textstyle\frac{1}{2}}\hbar,{\skew0...
...-{\textstyle\frac{1}{2}}\hbar,{\skew0\vec r}_3,S_{z3},\ldots)
\end{displaymath}

is represented by a single small determinant, (like for helium or lithium, say), it leads to uncorrelated spatial probability distributions for electrons 1 and 2. However, that stops being true as soon as there are at least two spin-up states and two spin-down states. And of course it is again just a symptom of the single-determinant disease, not the disease itself. Not a sliver of evidence is given that the supposed lack of correlation is an important source of the error in Hartree-Fock, let alone the only error.

Koch and Holthausen, [29, pp.22-23], address the same two electron example as Szabo and Ostlund, but do not have the same problem of finding the electron probabilities correlated. For example, if the spin-independent probability of finding the electrons at positions ${\skew0\vec r}_1$ and ${\skew0\vec r}_2$ in the dissociated hydrogen molecule is

\begin{displaymath}
{\textstyle\frac{1}{2}}\vert\psi_{\rm {l}}({\skew0\vec r}_...
...\vec r}_1)\vert^2\vert\psi_{\rm {l}}({\skew0\vec r}_2)\vert^2
\end{displaymath}

then, Koch and Holthausen explain to us, the second term must be the same as the first. After all, if the two terms were different, the electrons would be distinguishable: electron 1 would be the one that selected $\psi_{\rm {l}}$ in the first term that Koch and Holthausen wrote down in their book. So, the authors conclude, the second term above is the same as the first, making the probability of finding the electrons equal to twice the first term, $\vert\psi_{\rm {l}}({\skew0\vec r}_1)\vert^2\vert\psi_{rm{r}}({\skew0\vec r}_2)\vert^2$. That is an uncorrelated product probability.

However, the assumption that electrons are indistinguishable with respect to mathematical formulae in books is highly controversial. Many respected references, and this book too, only see an empirical requirement that the wave function, not books, be antisymmetric with respect to exchange of any two electrons. And the wave function is antisymmetric even if the two terms above are not the same.

Wikipedia, [[21]], Hartree-Fock entry June 2007, lists electron correlation, (defined here vaguely as effects arising from from the mean-field approximation, i.e. using the same $v^{\rm {HF}}$ operator for all electrons) as an approximation made in addition to using a single Slater determinant. Sorry, but Hartree-Fock gives the best single-determinantal approximation; there is no additional approximation made. The mean field approximation is a consequence of the single determinant, not an additional approximation. Then this reference proceeds to declare this correlation energy the most important of the set, in other words, more important that the single-determinant approximation! And again, even if the potential energy was computed exactly, instead of using the $v^{\rm {HF}}$ operator, and only the kinetic energy was computed using a Slater determinant, there would still be a finite error. It would therefore appear then that the name correlation energy is sufficiently impenetrable and poorly defined that even the experts cannot necessarily figure it out.

Consider for a second the ground state of two electrons around a massive nucleus. Because of the strength of the nucleus, the Coulomb interaction between the two electrons can to first approximation be ignored. A reader of the various vague qualitative stories listed above may then be forgiven for assuming that Hartree-Fock should not have any error. But only the unrestricted Hartree-Fock solution with those nasty, uncorrelated (true in this case), opposite-spin electrons (orbitals) is the one that gets the energy right. A unrestricted solution in terms of those perfect, correlated, aligned-spin electrons gets the energy all wrong, since one orbital will have to be an excited one. In short the correlation energy (error in energy) that, we are told, is due to the motion of electrons of opposite spins not being correlated is in this case 100% due to the motion of aligned-spin orbitals being correlated. Note that both solutions get the spin wrong, but we are talking about energy.

And what happened to the word error in correlation energy error? If you did a finite difference or finite element computation of the ground state, you would not call the error in energy truncation energy; it would be called truncation error or “energy truncation error.” Why does one suspect that the appropriate and informative word error did not sound hot enough to the physicists involved?

Many sources refer to a reference, (Löwdin, P.-E., 1959, Adv. Chem. Phys., 2, 207) instead of providing a solid justification of this widely-used key term themselves. If one takes the trouble to look up the reference, does one find a rigorously defined correlation energy and a proof it is identical in magnitude to the Hartree-Fock error?

Not exactly. One finds a vague qualitative story about some perceived holes whose mathematically rigorous definition remains restricted to the center point of one of them. However, the lack of a defined hole size is not supposed to deter the reader from agreeing wholeheartedly with all sorts of claims about the size of their effects. Terms like main error,, “small error,” large correlation error (qualified by certainly”), “vanish or be very small, (your choice), are bandied around, even though there is no small parameter that would allow any rigorous mathematical definition of small or big.

Then the author, who has already noted earlier that the references cannot agree on what the heck correlation energy is supposed to mean in the first place, states “In order to get at least a formal definition of the problem, ...” and proceeds to redefine the Hartree-Fock error to be the “correlation energy.” In other words, since correlation energy at this time seems to be a pseudo-scientific concept, let’s just cross out the correct name Hartree-Fock error, and write in “correlation energy!”

To this author’s credit, he does keep the word error in correlation error in the wave function instead of using correlation wave function. But somehow, that particular term does not seem to be cited much in literature.