We study the notion of degeneration for affine schemes associated to systems of algebraic differential equations with coefficients in the fraction field of a multivariate formal power series ring. In order to do this, we use an integral structure of this field that arises as the unit ball associated to the tropical valuation, first introduced in the context of tropical differential algebra. This unit ball turns out to be a particular type of integral domain, known as B\'ezout domain. By applying to these systems a translation map along a vector of weights that emulates the one used in classical tropical algebraic geometry, the resulting translated systems will have coefficients in this unit ball. When the resulting quotient module over the unit ball is torsion-free, then it gives rise to integral models of the original system in which every prime ideal of the unit ball defines an initial degeneration, and they can be found as a base-change to the residue field of the prime ideal. In particular, the closed fibres of our integral models can be rightfully called initial degenerations, since we show that the maximal ideals of this unit ball naturally correspond to monomial orders. We use this correspondence to define initial forms of differential polynomials and initial ideals of differential ideals, and we show that they share many features of their classical analogues.