We study the internal response of a galaxy to an unbound encounter and present a survey of orbital parameters covering typical encounters in different galactic environments. Overall, we conclude that relatively weak encounters by low-mass interloping galaxies can cause observable distortions in the primaries. The resulting asymmetries may persist long after the interloper is evident. We focus our attention on the production of structure in dark halos and in cluster ellipticals. Any distortion produced in a dark halo can distort the embedded stellar disk, possibly leading to the formation of lopsided and warped disks. We show that distant encounters with pericenters in the outer regions of a halo can excite strong and persistent features in the inner regions. Features excited in an elliptical are directly observable and we predict that asymmetries in the morphologies of these systems can be produced by relatively small perturbers. For example, a fly-by on an orbit with pericenter equal to the half-mass radius of the primary system and velocity of 200 km/s (a value typical for groups) can result in a significant dipole distortion for perturbers with mass as small as 5% of primary’s mass. We use these detailed results to predict the distribution of the A parameter defined by Abraham et al. (sensitive to lopsidedness) and the shift between the center of mass of the primary system and the position of the peak of density for a range of environments. We find that high-density, low-velocity dispersion environments are more likely to host galaxies with significant asymmetries. Our distribution for the A parameter is in good agreement with the range spanned by the observed values for local galaxy clusters and for distant galaxies in the Medium Deep Survey and in the Hubble Deep Field. Assuming that primordial galaxies were located in dense environments with previrialized low velocity dispersions, our conclusions are consistent with the observational results showing a systematic trend for galaxies at larger redshifts to be more asymmetric than local galaxies. Finally, we propose a generalized asymmetry parameter A(r) which provides detailed information on the radial structure of the asymmetry produced by the mechanism explored in our work.