This paper studies estimation of a smooth function f(x,v) when we are given functional responses of the form f(x, ·) + error, but scientific interest centers on the collection of functions f(·,v) for different v. The motivation comes from studies of human brain development, in which x denotes age whereas v refers to brain locations. Analogously to varying-coefficient models, in which the mean response is linear in x, the “varying-smoother” models that we consider exhibit nonlinear dependence on x that varies smoothly with v. We discuss three approaches to estimating varying-smoother models: (a) methods that employ a tensor product penalty; (b) an approach based on smoothed functional principal component scores; and (c) two-step methods consisting of an initial smooth with respect to x at each v, followed by a postprocessing step. For the first approach, we derive an exact expression for a penalty proposed by Wood, and an adaptive penalty that allows smoothness to vary more flexibly with v. We also develop “pointwise degrees of freedom,” a new tool for studying the complexity of estimates of f(·,v) at each v. The three approaches to varying-smoother models are compared in simulations and with a diffusion tensor imaging data set.