For $p = 2$ we have $\frac{a_{2}}{b_{2}} = \frac{11}{6}$, and therefore $\operatorname{gcd}\left(2, b_{2}\right) = 2$.

For $p$ an odd prime we propose two solutions to this problem:

First solution. We have
$$\frac{a_{p}}{b_{p}} = \frac{\sum_{k=1}^{p^{2}-1} \frac{(p^{2}-1)!}{k}}{(p^{2}-1)!} .$$
Because the only multiples of $p$ appearing in $1 \times 2 \times \cdots \times (p^{2}-1)$ are $p, 2p, \ldots, (p-1)p$, and $p$ is a prime number, $p^{p-1}$ is the largest power of $p$ dividing $(p^{2}-1)!$.
For $1 \leq k \leq p^{2}-1$, if $k$ is not a multiple of $p$ then $p^{p-1}$ divides $\frac{(p^{2}-1)!}{k}$. If $k$ is a multiple of $p$ then $p^{p-2}$ is the largest power of $p$ dividing $\frac{(p^{2}-1)!}{k}$.
Summing up the terms where $k$ is a multiple of $p$ we obtain
$$\sum_{j=1}^{p-1} \frac{(p^{2}-1)!}{jp} = \frac{(p^{2}-1)!}{p!} \sum_{j=1}^{p-1} \frac{(p-1)!}{j}$$
Notice that all these fractions are integers. Using the fact that for each integer $1 \leq j \leq p-1$ there is a unique inverse integer $1 \leq \psi(j) \leq p-1$ modulo $p$ (i.e. $j \psi(j) \equiv 1 \bmod p$) and that this inverse map $j \rightarrow \psi(j)$ is a permutation of $\{1,2, \ldots, p-1\}$, we get
$$\begin{aligned} \sum_{j=1}^{p-1} \frac{(p-1)!}{j} &\equiv \sum_{j=1}^{p-1} \frac{(p-1)!}{j} j \psi(j) \equiv (p-1)! \sum_{j=1}^{p-1} \psi(j) \\ &\equiv (p-1)! \sum_{j=1}^{p-1} j \equiv (p-1)! \frac{p(p-1)}{2} \equiv 0, \quad (\bmod p) \end{aligned}$$
This proves that $p^{p-1}$ divides $\sum_{j=1}^{p-1} \frac{(p^{2}-1)!}{jp}$ and so it divides $\sum_{k=1}^{p^{2}-1} \frac{(p^{2}-1)!}{k}$.
We deduce that $p^{p-1}$ cancels from the denominator and therefore
$$\operatorname{gcd}\left(p, b_{p}\right) = 1 .$$

Second solution. Because $p$ is odd, we have
$$\frac{a_{p}}{b_{p}} = \sum_{k=1}^{\frac{p^{2}-1}{2}} \left( \frac{1}{k} + \frac{1}{p^{2}-k} \right ) = \sum_{k=1}^{\frac{p^{2}-1}{2}} \frac{p^{2}}{k(p^{2}-k)}$$
For $1 \leq k \leq \frac{p^{2}-1}{2}$, if $k$ is not a multiple of $p$ then the denominator $k(p^{2}-k)$ is not divisible by $p$. If $k$ is a multiple of $p$ then
$$\frac{p^{2}}{k(p^{2}-k)} = \frac{1}{j(p-j)},$$
where $k = jp$ for some integer $1 \leq j \leq \frac{p-1}{2}$. But the denominator $j(p-j)$ is not divisible by $p$. Therefore $\frac{a_{p}}{b_{p}}$ is a sum of fractions where none of the denominators is divisible by $p$ and, since $\frac{a_{p}}{b_{p}}$ is irreducible, then $\operatorname{gcd}\left(p, b_{p}\right) = 1$.