This brief studies the cooperative output regulation problems of multi-agent systems with parametric and dynamic uncertainties. By means of robust adaptive dynamic programming, a model-free distributed controller is developed via online input and state data. The cyclic-small-gain theorem is applied to ensure the asymptotic stability of the closed-loop system and thus solves the cooperative output regulation problem. Each follower is able to achieve asymptotic tracking and non-vanishing disturbance rejection. A numerical example is utilized to illustrate the effectiveness of the proposed algorithm.