We investigate the star formation histories (SFHs) of high redshift (3 . z . 5) star-forming galaxies selected based on their rest-frame ultraviolet (UV) colors in the CANDELS/GOODS-S field. By comparing the results from the spectral-energy- distribution-fitting analysis with two different assumptions about the SFHs — i.e., exponentially declining SFHs as well as increasing ones, we conclude that the SFHs of high-redshift star-forming galaxies increase with time rather than exponentially decline. We also examine the correlations between the star formation rates (SFRs) and the stellar masses. When the galaxies are fit with rising SFRs, we find that the trend seen in the data qualitatively matches the expectations from a semi-analytic model of galaxy formation. The mean specific SFR is shown to increase with redshift, also in agreement with the theoretical prediction. From the derived tight correlation between stellar masses and SFRs, we derive the mean SFH of star-forming galaxies in the redshift range of 3 ≤ z ≤ 5, which shows a steep power-law (with power _ = 5.85) increase with time. We also investigate the formation timescales and the mean stellar population ages of these star-forming galaxies. Our analysis reveals that UV-selected star-forming galaxies have a broad range of the formation redshift. The derived stellar masses and the stellar population ages show positive correlation in a sense that more massive galaxies are on average older, but with significant scatter. This large scatter implies that the galaxies’ mass is not the only factor which affects the growth or star formation of high-redshift galaxies.