Difficulty of Detecting Minihalos via γ Rays from Dark Matter Annihilation