N.18

Correlation energy

The error in Hartree-Fock is due to the single-determinant
approximation only. A term like “Hartree-Fock error“ or
single-determinantal error

is therefore both precise,
and immediately understandable by a general audience.

However, it is called correlation energy,

and to
justify that term, it would have to be both clearer and equally
correct mathematically. It fails both requirements miserably. The
term correlation energy is clearly confusing and distracting for
nonspecialist. But in addition, there does not seem to be any theorem
that proves that an independently defined correlation energy is
identical to the Hartree-Fock single determinant error. That would
not just make the term correlation energy disingenuous, it would make
it wrong.

Instead of finding a rigorous theorem, you are lucky if standard textbooks, e.g,. [33,29,45] and typical web references, offer a vague qualitative story why Hartree-Fock underestimates the repulsions if a pair of electrons gets very close. That is a symptom of the disease of having an incomplete function representation, it is not the disease itself. Low-parameter function representations have general difficulty with representing localized effects, whatever their physical source. If you make up a system where the Coulomb force vanishes both at short and at long distance, such correlations do not exist, and Hartree-Fock would still have a finite error.

The kinetic energy is not correct either; what is the correlation in
that? Some sources, like [29] and web sources, seem
to suggest that these are indirect

results of having
the wrong correlation energy, whatever correlation energy may be. The
idea is apparently, if you would have the electron-electron
repulsions exact, you would compute the correct kinetic energy too.
That is just like saying, if you computed the correct kinetic
energy term, you would compute the correct potential too, so let’s
rename the Hartree-Fock error “kinetic energy
interaction.” Even if you computed the potential
energy correctly, you would still have to convert the wave function to
single-determinantal form before evaluating the kinetic energy, otherwise it is not Hartree-Fock, and that would produce a
finite error. Phrased differently, there is absolutely no way to
get a general wave function correct with a finite number of
single-electron functions, whatever corrections you make to the
potential energy.

Szabo and Ostlund [45, p. 51ff,61] state that it is called correlation energy since “the motion of electrons with opposite spins is not correlated within the Hartree-Fock approximation.” That is incomprehensible, for one thing since it seems to suggest that Hartree-Fock is exact for excited states with all electrons in the same spin state, which would be ludicrous. In addition, the electrons do not have motion; a stationary wave function is computed, and they do not have spin; all electrons occupy all the states, spin up and down. It is the orbitals that have spin, and the spin-up and spin-down orbitals are most definitely correlated.

However, the authors do offer a clarification;

they
take a Slater determinant of two opposite spin orbitals, compute the
probability of finding the two electrons at given positions and find
that it is correlated. They then explain: that’s OK; the exchange
requirements do not allow uncorrelated positions. This really helps an
engineer trying to figure out why the motion

of the
two electrons is uncorrelated!

The unrestricted Hartree-Fock solution of the dissociated hydrogen
molecule is of this type. Since if one electron is around the left
proton, the other is around the right one, and vice versa, many people
would call the positions of the electrons strongly correlated. But
now we engineers understand that this does not count,

because an uncorrelated state in which electron 1 is around the left
proton for sure and electron 2 is around the right one for sure is not
allowed.

Having done so well, the authors offer us no further guidance how we
are supposed to figure out whether or not electrons 1 and 2 are of
opposite spin if there are more than two electrons. It is true that
if the wave function

is represented by a single small determinant, (like for helium or lithium, say), it leads to uncorrelated spatial probability distributions for electrons 1 and 2. However, that stops being true as soon as there are at least two spin-up states and two spin-down states. And of course it is again just a symptom of the single-determinant disease, not the disease itself. Not a sliver of evidence is given that the supposed lack of correlation is an important source of the error in Hartree-Fock, let alone the only error.

Koch and Holthausen, [29, pp.22-23], address the
same two electron example as Szabo and Ostlund, but do not have the
same problem of finding the electron probabilities correlated. For
example, if the spin-independent probability of finding the electrons
at positions and in the dissociated hydrogen molecule
is

then, Koch and Holthausen explain to us, the second term must be the same as the first. After all, if the two terms were different, the electrons would be distinguishable: electron 1 would be the one that selected in the first term that Koch and Holthausen wrote down in their book. So, the authors conclude, the second term above is the same as the first, making the probability of finding the electrons equal to twice the first term, . That is an uncorrelated product probability.

However, the assumption that electrons are indistinguishable with respect to mathematical formulae in books is highly controversial. Many respected references, and this book too, only see an empirical requirement that the wave function, not books, be antisymmetric with respect to exchange of any two electrons. And the wave function is antisymmetric even if the two terms above are not the same.

Wikipedia, [[21]], Hartree-Fock entry June 2007, lists
electron correlation, (defined here vaguely as effects

arising from from the mean-field approximation, i.e. using the same
operator for all electrons) as an approximation made in addition to using a single Slater determinant. Sorry, but
Hartree-Fock gives the best single-determinantal approximation; there
is no additional approximation made. The mean
field

approximation is a consequence of the single
determinant, not an additional approximation. Then this reference
proceeds to declare this correlation energy the most important of the
set, in other words, more important that the single-determinant
approximation! And again, even if the potential energy was
computed exactly, instead of using the operator, and
only the kinetic energy was computed using a Slater determinant, there
would still be a finite error. It would therefore appear then that
the name correlation energy is sufficiently impenetrable and poorly
defined that even the experts cannot necessarily figure it out.

Consider for a second the ground state of two electrons around a
massive nucleus. Because of the strength of the nucleus, the Coulomb
interaction between the two electrons can to first approximation be
ignored. A reader of the various vague qualitative stories listed
above may then be forgiven for assuming that Hartree-Fock should not
have any error. But only the unrestricted Hartree-Fock solution with
those nasty, uncorrelated

(true in this case),
opposite-spin electrons

(orbitals) is the one that
gets the energy right. A unrestricted solution in terms of those
perfect, correlated, aligned-spin electrons

gets the
energy all wrong, since one orbital will have to be an excited one.
In short the correlation energy

(error in energy)
that, we are told, is due to the motion

of electrons
of opposite spins not being correlated

is in this case
100% due to the motion of aligned-spin orbitals being correlated.
Note that both solutions get the spin wrong, but we are talking about
energy.

And what happened to the word error

in
correlation energy error?

If you did a finite
difference or finite element computation of the ground state, you
would not call the error in energy truncation energy;

it would be called truncation error

or “energy
truncation error.” Why does one suspect that the appropriate
and informative word error

did not sound
hot

enough to the physicists involved?

Many sources refer to a reference, (Löwdin, P.-E., 1959, Adv. Chem. Phys., 2, 207) instead of providing a solid justification of this widely-used key term themselves. If one takes the trouble to look up the reference, does one find a rigorously defined correlation energy and a proof it is identical in magnitude to the Hartree-Fock error?

Not exactly. One finds a vague qualitative story about some perceived
holes

whose mathematically rigorous definition remains
restricted to the center point of one of them. However, the lack of a
defined hole size is not supposed to deter the reader from agreeing
wholeheartedly with all sorts of claims about the size of their
effects. Terms like main error,

, “small
error,” large correlation error

(qualified by
certainly”), “vanish or be very small,

(your choice), are bandied around, even though there is no small parameter that would allow any
rigorous mathematical definition of small or big.

Then the author, who has already noted earlier that the references cannot agree on what the heck correlation energy is supposed to mean in the first place, states “In order to get at least a formal definition of the problem, ...” and proceeds to redefine the Hartree-Fock error to be the “correlation energy.” In other words, since correlation energy at this time seems to be a pseudo-scientific concept, let’s just cross out the correct name Hartree-Fock error, and write in “correlation energy!”

To this author’s credit, he does keep the word error in
correlation error in the wave function

instead of
using correlation wave function.

But somehow, that
particular term does not seem to be cited much in literature.