We study the local geometry of the moduli space of intermediate Jacobians of $(2,2)$-threefolds in ${\mathbb P}^2 \times {\mathbb P}^2$. More precisely, we prove that a composition of the second fundamental form of the Siegel metric in $\mathcal A_9$ restricted to this moduli space, with a natural multiplication map is a nonzero holomorphic section of a vector bundle. We also describe its kernel. We use the two conic bundle structures of these threefolds, Prym theory, gaussian maps and Jacobian ideals.