In this article, we investigate the class of Hermitian manifolds whose Bismut connection has parallel torsion ({\rm BTP} for brevity). In particular, we focus on the case where the manifold is (locally) homogeneous with respect to a group of holomorphic isometries and we fully characterize the compact Chern flat {\rm BTP} manifolds. Moreover we show that certain compact flag manifolds are {\rm BTP} if and only if the metric is K\"ahler or induced by the Cartan-Killing form and we then characterize {\rm BTP} invariant metrics on compact semisimple Lie groups which are Hermitian w.r.t. a Samelson structure and are projectable along the Tits fibration. We state a conjecture concerning the question when the Bismut connection of a BTP compact Hermitian locally homogeneous manifold has parallel curvature, giving examples and providing evidence in some special cases.