We study the propagation of light in the presence of a parity-violating coupling between photons and axion-like particles (ALPs). Naively, this interaction could lead to a split of light rays into two separate beams of different polarization chirality and with different refraction angles. However, by using the eikonal method we explicitly show that this is not the case and that ALP clumps do not produce any spatial birefringence. This happens due to non-trivial variations of the photon's frequency and wavevector, which absorb time-derivatives and gradients of the ALP field. We argue that these variations represent a new way to probe the ALP-photon coupling with precision frequency measurements.