©2015 Optical Society of America. The interest on the conical refraction (CR) phenomenon in biaxial crystals has revived in the last years due to its prospective for generating structured polarized light beams, i.e. vector beams. While the intensity and the polarization structure of the CR beams are well known, an accurate experimental study of their phase structure has not been yet carried out. We investigate the phase structure of the CR rings by means of a Mach-Zehnder interferometer while applying the phase-shifting interferometric technique to measure the phase at the focal plane. In general the two beams interfering correspond to different states of polarization (SOP) which locally vary. To distinguish if there is an additional phase added to the geometrical one we have derived the appropriate theoretical expressions using the Jones matrix formalism. We demonstrate that the phase of the CR rings is equivalent to that one introduced by an azimuthally segmented polarizer with CR-like polarization distribution. Additionally, we obtain direct evidence that the Poggendorff dark ring is an annular singularity, with a π phase change between the inner and outer bright rings.