As in the diagram, let $A$, $B$, $C$, $D$ be the centers of the $4$ balls in the bottom layer, and $A'$, $B'$, $C'$, $D'$ the centers of the $4$ balls in the upper layer. Then $A$, $B$, $C$, $D$ and $A'$, $B'$, $C'$, $D'$ are the $4$ vertices of squares of length $2$, respectively. Now, the circumscribed circles with centers $O$ and $O'$ of the squares constitute the bases of another cylinder, and the projecting point of $A'$ on the bottom base is the middle point $M$ of arc $AB$.

In $\triangle A'A'B$, we have $A'A = A'B = AB = 2$, then $A'N = \sqrt{3}$, where $N$ is the middle point of $AB$. Meanwhile, $OM = OA = \sqrt{2}$, $ON = 1$, so
$$MN = \sqrt{2} - 1, \quad A'M = \sqrt{(A'N)^2 - (MN)^2} = \sqrt{8}.$$
Then the height of the original cylinder is $\sqrt{8} + 2$.