Volatile organic compounds, particulate matter, airborne infectious disease, and harmful chemical or biological agents are examples of gaseous and particulate contaminants affecting human health in indoor environments. Fast and accurate methods are needed for detection, predictive transport, and contaminant source identification. Markov matrices have shown promise for these applications. However, current (Lagrangian and flux based) Markov methods are limited to small time steps and steady-flow fields. We extend the application of Markov matrices by developing a methodology based on Eulerian approaches. This allows construction of Markov matrices with time steps corresponding to very large Courant numbers. We generalize this framework for steady and transient flow fields with constant and time varying contaminant sources. We illustrate this methodology using three published flow fields. The Markov methods show excellent agreement with conventional PDE methods and are up to 100 times faster than the PDE methods. These methods show promise for developing real-time evacuation and containment strategies, demand response control and estimation of contaminant fields of potential harmful particulate or gaseous contaminants in the indoor environment.