Precision X-ray structural studies were carried out for LiNbO3:Znx single crystals with x=0.0, 2.87, 5.20, and 7.60 at. %. It was found that the insertion of the Zn atoms into the Li position was accompanied by a decrease in the concentration of intrinsic NbLi defects. At x=7.6%, the Zn atoms change their locations in the lattice and partially occupy the Nb positions. This clarifies the structural nature of the “threshold” Zn concentration, which manifests itself as singularities in the concentration dependences of various optical properties. The structural origin of the threshold concentration is likely a common feature of all nonphotorefractive impurities (Mg, Zn, In, and Sc) in LiNbO3. A change in the intrinsic defect structure of the LiNbO3 crystals with different Zn concentrations is discussed.