The numerical solution of partial differential equations (PDEs) with Neumann boundary conditions (BCs) resulted from strong form collocation scheme are typically much poorer in accuracy compared to those with pure Dirichlet BCs. In this paper, we show numerically that the reason of the reduced accuracy is that Neumann BC requires the approximation of the spatial derivatives at Neumann boundaries which are significantly less accurate than approximation of main function. Therefore, we utilize boundary treatment schemes that based upon increasing the accuracy of spatial derivatives at boundaries. Increased accuracy of the spatial derivative approximation can be achieved by h-refmement reducing the spacing between discretization points or by increasing the multiquadric shape parameter, c. Increasing the MQ shape parameter is very computationally cost effective, but leads to increased ill-conditioning. We have implemented an improved version of the truncated singular value decomposition (IT-SVD) originated by Volokh and Vilnay (2000) that projects very small singular values into the null space, producing a well conditioned system of equations. To assess the proposed refinement scheme, elliptic PDEs with different boundary conditions are analyzed. Comparisons that made with analytical solution reveal superior accuracy and computational efficiency of the IT-SVD solutions.