Prym varieties and moduli of polarized Nikulin surfaces