We study various naturally defined subvarieties of the moduli space ${\mathcal A}_g$ of complex principally polarized abelian varieties (ppav) in a neighborhood of the locus of products of $g$ elliptic curves. In this neighborhood, we obtain a local description for the locus of hyperelliptic curves, reproving the recent result of Shepherd-Barron that the hyperelliptic locus is locally given by tridiagonal matrices. We further reprove and generalize to arbitrary genus the recent result of Agostini and Chua showing that the locus of Jacobians of genus 5 curves with a theta-null is an irreducible component of the locus of ppav with a theta-null such that the singular locus of the theta divisor at the corresponding two-torsion point has tangent cone of rank at most 3. We further show that the locus of ppav such that the gradient vanishes, for some odd theta characteristic, locally has codimension $g$ near the diagonal. Finally, we obtain new results on the locus where the rank of the Hessian of the theta function at a two-torsion point that lies on the theta divisor is equal to 2.