A statistical manifold is a pseudo-Riemannian manifold endowed with a Codazzi structure. This structure plays an important role in Information Geometry and its related fields, e.g., a statistical model admits this structure with the Fisher-Rao metric. In practical application, however, the metric may be degenerate, and then this geometric structure is not fully adapted. In the present paper, for such cases, we introduce the notice of quasi-Codazzi structure which consists of a possibly degenerate metric (i.e., symmetric $(0,2)$-tensor) and a pair of coherent tangent bundles with affine connections. This is thought of as an affine differential geometry of Lagrange subbundles of para-Hermitian vector bundles and also as a submanifold theory of para-Hermitian space-form. As a special case, the quasi-Codazzi structure with flat connections coincides with the quasi-Hessian structure previously studied by Nakajima-Ohmoto. The relation among our quasi-Codazzi structure, quasi-Hessian structure and weak contrast functions generalizes the relation among Codazzi structure, dually flat (i.e., Hessian) structure and contrast functions.