For $n \geq 10$, $n!$ is divisible by $10$, so $(n!)^2$ is divisible by $100$. Therefore, only the terms $(1!)^2, (2!)^2, \dots, (9!)^2$ contribute to the last two digits.

Compute each term:

$(1!)^2 = 1^2 = 1$

$(2!)^2 = 2^2 = 4$

$(3!)^2 = 6^2 = 36$

$(4!)^2 = 24^2 = 576$

$(5!)^2 = 120^2 = 14400$

$(6!)^2 = 720^2 = 518400$

$(7!)^2 = 5040^2 = 25401600$

$(8!)^2 = 40320^2 = 1625702400$

$(9!)^2 = 362880^2 = 131681894400$

Now, take the last two digits of each term:

$(1!)^2 = 1$ (last two digits: $01$)

$(2!)^2 = 4$ (last two digits: $04$)

$(3!)^2 = 36$ (last two digits: $36$)

$(4!)^2 = 576$ (last two digits: $76$)

$(5!)^2 = 14400$ (last two digits: $00$)

$(6!)^2 = 518400$ (last two digits: $00$)

$(7!)^2 = 25401600$ (last two digits: $00$)

$(8!)^2 = 1625702400$ (last two digits: $00$)

$(9!)^2 = 131681894400$ (last two digits: $00$)

Sum the last two digits:
$01 + 04 + 36 + 76 + 00 + 00 + 00 + 00 + 00 = 117$

The last two digits of $117$ are $17$.

Answer: $17$