A model dependent heuristic dynamic programming (MDHDP) is presented in this paper to regulate the voltage and frequency of a virtual inertia-based grid-connected inverter. To overcome the inertialess and fast-responding characteristics of the conventional direct power inverter, the virtual synchronous generator (VSG) concept has introduced. The negative points of the VSG approach including: (i) weak performance in inductive grids, (ii) disability to perform under uncertainties, and (iii) its dependency to the operating point, motivates us to implement a neurocontrol reinforcement learning technique. The simulation results illustrate that a well-designed and well-trained MDHDP can perform optimally in tracking the active power while facing changes in system parameters.