Signatures of short range anisotropic ferromagnetic correlations and ferromagnetic clustering, manifested as unusually large hysteresis and other anomalies of the low magnetic field dc magnetization and ac magnetic susceptibility, have been observed in both the antiferromagnetic and paramagnetic states of single crystal Gd5Ge4. Ferromagnetic correlations, which are most pronounced in a weak magnetic field applied along the b axis, are readily suppressed by fields exceeding ∼5 kOe and are believed to be related to a Griffiths-like phase that develops in Gd5Ge4 below TG≅240 K.