Generically the return map of an integrable Hamiltonian system with two degrees of freedom in a Hamiltonian level foliated by invariant tori is a twist map. If we perturb such integrable Hamiltonian system inside the class of Hamiltonian systems with two degrees of freedom, then the Poincaré-Birkhoff theorem allows to determine which periodic orbits of the integrable can be prolonged to the perturbed one, and the KAM theory provides sufficient conditions in order that some invariant tori persist under sufficiently small perturbations. If some power of this return map is the identity, then in general for these degenerate Hamiltonian systems we cannot study which periodic orbits of the integrable can be prolonged to the perturbed one, or if some invariant tori persist. This paper studies the perturbation of integrable Hamiltonian systems with two degrees of freedom having some power of the return map equal to the identity. We show with two different models a way to study the prolongation of periodic orbits and of invariant tori or cylinders filled with periodic orbits for such kind of Hamiltonian systems. The main tool in this study is the averaging theory. © 2010 American Institute of Physics.