We used the deep, multiwavelength images obtained by the Great Observatories Origins Deep Survey (GOODS) to identify ~4700 Lyman break galaxies (LBGs) at z > 2.5, and 292 starburst galaxies at z ~ 1.2. We present the results from morphological analysis based on light profile shape and ellipticity for ~1333 of the most luminous LBGs. About 40% of LBGs at z ~ 3 have exponential profiles, ~30% of the galaxies have steep (r1/4-like) profiles, and ~30% of LBGs have multiple cores of disturbed morphologies suggestive of close pairs or mergers. The fraction of spheriod-like LBGs decrease by about 15% from z ~ 5 to 3. A comparison of LBGs with the starburst galaxies at z ~ 1.2 shows that disklike and merger morphologies are dominant, but the fraction of spheroid-like profiles is about 20% higher among LBGs. The ellipticity distribution for LBGs exhibits a pronounced skew toward high ellipticities (epsilon > 0.5), which cannot be explained by morphologies similar to the local disks and spheroids viewed at random orientations. The peak of the distribution evolves toward lower epsilon, from 0.7 at z = 4 to ~0.5 at z = 3. The ellipticity distribution for the z ~ 1.2 galaxies is relatively flat, similar to that seen for present-day galaxies. The dominance of elongated morphologies suggests that in a significant fraction of LBGs we may be witnessing star formation in clumps along gas-rich filaments, or the earliest gas-rich bars that encompass essentially the entire visible galaxy.