There has been a surge of interest in applying scientific, physics-informed machine learning models to a wide variety of engineering problems. Among them, physics-informed neural networks (PINNs) have attracted significant attention for their flexible capability to tackle both forward and inverse analysis in a unified manner. In particular, they could be effective for a wide range of inverse problems where only sparse or noisy data is available to estimate unobserved physical quantities. Yet, their accuracy is often hindered by the poor treatment of boundary conditions or the inappropriate scaling of multiple objectives. To address these issues, we propose an integrated framework combining the normalized distance field and unbiased adaptive weight tuning to improve both accuracy and efficiency. We present numerical results to demonstrate that the proposed method provides more accurate and efficient solutions to inverse problems. This approach enables a reliable, fast and stable inverse analysis using PINNs, with potential applications to various engineering problems.