We present the deepest far-IR observations obtained with Herschel and examine the 3-500um SEDs of galaxies at 03x10^10 Lsun kpc^-2) and a high specific SFR (i.e., SBs). The rest-frame, UV-2700A size of these distant SBs is typically half that of MS galaxies, supporting the correlation between star formation density and SB activity that is measured for the local sample. Locally, (U)LIRGs are systematically in the SB mode, whereas most distant (U)LIRGs form stars in the "normal" MS mode. This confusion between two modes of star formation is the cause of the so-called "mid-IR excess" population of galaxies found at z>1.5 by previous studies. MS galaxies have strong PAH emission line features, a broad far-IR bump resulting from a combination of dust temperatures (Tdust~15-50 K), and an effective Tdust~31 K, as derived from the peak wavelength of their IR SED. Galaxies in the SB regime instead exhibit weak PAH EW and a sharper far-IR bump with an effective Tdust~40 K. Finally, we present evidence that the mid-to-far IR emission of X-ray AGNs is predominantly produced by star formation and that candidate dusty AGNs with a power-law emission in the mid-IR systematically occur in compact, dusty SBs. After correcting for the effect of SBs on IR8, we identify new candidates for extremely obscured AGNs.