In this note, we consider the trees (caterpillars) that minimize the number of subtrees among trees with a given degree sequence. This is a question naturally related to the extremal structures of some distance based graph invariants. We first confirm the expected fact that the number of subtrees is minimized by some caterpillar. As with other graph invariants, the specific optimal caterpillar is nearly impossible to characterize and depends on the degree sequence. We provide some simple properties of such caterpillars as well as observations that will help finding the optimal caterpillar.