For a smooth scheme $X$ over a perfect field $k$ of positive characteristic, we define (for each $m\in\mathbb{Z}$) a sheaf of rings $\mathcal{\widehat{D}}_{W(X)}^{(m)}$ of differential operators (of level $m$) over the Witt vectors of $X$. If $\mathfrak{X}$ is a lift of $X$ to a smooth formal scheme over $W(k)$, then for $m\geq0$ modules over $\mathcal{\widehat{D}}_{W(X)}^{(m)}$ are closely related to modules over Berthelot's ring $\widehat{\mathcal{D}}_{\mathfrak{X}}^{(m)}$ of differential operators of level $m$ on $\mathfrak{X}$. Our construction therefore gives an description of suitable categories of modules over these algebras, which depends only on the special fibre $X$. There is an embedding of the category of crystals on $X$ (over $W_{r}(k)$) into modules over $\mathcal{\widehat{D}}_{W(X)}^{(0)}/p^{r}$; and so we obtain an alternate description of this category as well. For a map $\varphi:X\to Y$ we develop the formalism of pullback and pushforward of $\mathcal{\widehat{D}}_{W(X)}^{(m)}$-modules and show all of the expected properties. When working mod $p^{r}$, this includes compatibility with the corresponding formalism for crystals, assuming $\varphi$ is smooth. In this case we also show that there is a ``relative de Rham Witt resolution'' (analogous to the usual relative de Rham resolution in $\mathcal{D}$-module theory) and therefore that the pushforward of (a quite general subcategory of) modules over $\mathcal{\widehat{D}}_{W(X)}^{(0)}/p^{r}$ can be computed via the reduction mod $p^{r}$ of Langer-Zink's relative de Rham Witt complex. Finally we explain a generalization of Bloch's theorem relating integrable de Rham-Witt connections to crystals.