The destabilisation mechanism of oil-in-water (o/w) emulsions was studied as a function of oil content (20% and 40% o/w), homogenisation conditions and crystallisation temperatures (10, 5, 0, −5 and −10 °C). A mixture of anhydrous milk fat and soya bean oil was used as the lipid phase and whey protein isolate (2 wt%) as emulsifier. Crystallisation and melting behaviours were analysed using differential scanning calorimetry. Physicochemical stability was measured with a vertical scan macroscopic analyser. Emulsions with 20% oil were found to be less stable than those with 40% oil. For 20% o/w emulsions, the crystallisation was delayed and inhibited in emulsions with smaller droplets and promoted in emulsions with larger droplets when compared with 40% o/w emulsions. Depending on the droplet sizes in the emulsion, the formation of lipid crystals (in combination with the emulsifier) either stabilises (small droplets) or destabilises (big droplets) the emulsion.