Solution:
$(1,1), (3,3)$
A square must end in the digit $0, 1, 4, 5, 6$, or $9$. If $n \geq 4$, then $1! + 2! + \cdots + n!$ ends in the digit $3$, so cannot be a square. A simple check for the remaining cases reveals that the only solutions are $(1,1)$ and $(3,3)$.