Fix an $I$-adically complete Noetherian ring $A$ and suppose $X$ is a proper $A$-scheme. This article concerns the relationship between the Brauer group of $X$ and that of the various $X_n$ where $X_n$ is the fiber over $A/I^{n+1}$. In particular, we answer a question of Grothendieck by showing that, in positive and mixed characteristic, there are examples of $X$ with nontrivial Brauer classes that restrict to zero on all the $X_n$. We characterize such behavior, prove this cannot happen in characteristic zero, and deduce a formal GAGA statement for Brauer classes.