Atomic ground states are usually degenerate. It is demonstrated that the density functionals for the exchange-correlation energy that are commonly used are not invariant over the set of ground state densities. This leads to uncertainties of the order of 3 to 5 kcal/mol in the atomic ground state energy of second and third period main group elements and the first transition series. A much larger spread in energies is obtained for transition elements if symmetry and equivalence restrictions for the Kohn-Sham orbitals are abandoned. It is recommended that atomic ground states that are actually used to calculate heats of atomization are made explicit, and tables with one choice of atomic ground state energies for the first rows of the periodic system are provided for the local density approximation and for a few generalized gradient approximations.