Solution:
The condition is symmetric so we can assume that $b \leq a$.

The first case is when $a = b$. In this case, $a! + a = 5^{m}$ for some positive integer $m$. We can rewrite this as $a \cdot ((a-1)! + 1) = 5^{m}$. This means that $a = 5^{k}$ for some integer $k \geq 0$. It is clear that $k$ cannot be $0$. If $k \geq 2$, then $(a-1)! + 1 = 5^{l}$ for some $l \geq 1$, but $a-1 = 5^{k} - 1 > 5$, so $5 \mid (a-1)!$. However, $5 \mid (a-1)! + 1$, which is not possible. This means that $k = 1$ and $a = 5$. In this case, $5! + 5 = 125$, which gives us the solution $(5,5)$.

Let us now assume that $1 \leq b < a$. Let us first assume that $b = 1$. Then $a + 1 = 5^{x}$ and $a! + 1 = 5^{y}$ for integers $x, y \geq 1$. If $x \geq 2$, then $a = 5^{x} - 1 \geq 5^{2} - 1 > 5$, so $5 \mid a!$. However, $5 \mid 5^{y} = a! + 1$, which leads to a contradiction. We conclude that $x = 1$ and $a = 4$. From here $a! + b = 25$ and $b! + a = 5$, so we get two more solutions: $(1,4)$ and $(4,1)$.

Now we focus on the case $1 < b < a$. Then we have $a! + b = 5^{x}$ for $x \geq 2$, so $b \cdot \left(\frac{a!}{b} + 1\right) = 5^{x}$, where $b \mid a!$ because $a > b$. Because $b \mid 5^{x}$ and $b > 1$, we have $b = 5^{z}$ for $z \geq 1$. If $z \geq 2$, then $5 < b < a$, so $5 \mid a!$, which means that $\frac{a!}{b} + 1$ cannot be a power of $5$. We conclude that $z = 1$ and $b = 5$. From here $5! + a$ is a power of $5$, so $5 \mid a$, but $a > b = 5$, which gives us $a \geq 10$. However, this would mean that $25 \mid a!$, $5 \mid b$ and $25 \nmid b$, which is not possible, because $a! + b = 5^{x}$ and $25 \mid 5^{x}$.

We conclude that the only solutions are $(1,4)$, $(4,1)$ and $(5,5)$.