When the condition is satisfied, there exists a non-negative integer $n$ such that $ap^4 + 2p^3 + 2p^2 + 1 = n^2$ and then
$$(a - p^2 - 2p - 1)p^4 = n^2 - (p^3 + p^2 + 1)^2 = (n + p^3 + p^2 + 1)(n - p^3 - p^2 - 1) \quad (*)$$
holds. If we assume that both $n + p^3 + p^2 + 1$ and $n - p^3 - p^2 - 1$ are divisible by $p$, then $n + 1 \equiv 0 \equiv n - 1 \pmod p$, leading to $2 \equiv 0 \pmod p$, which contradicts the assumption that $p$ is an odd prime. Therefore, at least one of $n + p^3 + p^2 + 1, n - p^3 - p^2 - 1$ is not divisible by $p$. Since the left hand side of (*) is divisible by $p^4$, one of $n + p^3 + p^2 + 1, n - p^3 - p^2 - 1$ is divisible by $p^4$.
In the case $n + p^3 + p^2 + 1$ is divisible by $p^4$, there exists a positive integer $k$ such that $n = kp^4 - p^3 - p^2 - 1$. By substituting this into (*), we obtain $(a - p^2 - 2p - 1)p^4 = kp^4(kp^4 - 2p^3 - 2p^2 - 2)$, hence $a = p^2 + 2p + 1 + k(kp^4 - 2p^3 - 2p^2 - 2)$. If $k \ge 2$, we have
$$a > 4(p^4 - p^3 - p^2 - 1) = 4p^4 \left(1 - \frac{1}{p} - \frac{1}{p^2} - \frac{1}{p^4}\right) \ge 4p^4 \left(1 - \frac{1}{3} - \frac{1}{9} - \frac{1}{81}\right) > p^4$$
which contradicts $a < p^4$. If $k = 1$, then $a = p^4 - 2p^3 - p^2 + 2p - 1 = (p^2 - p - 1)^2 - 2$. In this case, by $p \ge 3$ we get $2 \le p^2 - p - 1 < p^2$ and then $1 \le a < p^4$. Therefore, the number of pairs $(p, a)$ satisfying the condition is equal to the number of odd prime $p$ such that $(p^2 - p - 1)^2 - 2 \le 2024$, and there exist three such primes $p = 3, 5, 7$.
In the case $n - p^3 - p^2 - 1$ is divisible by $p^4$, there exists a non-negative integer $k$ such that $n = kp^4 + p^3 + p^2 + 1$. By substituting this to (*), we obtain $(a - p^2 - 2p - 1)p^4 = kp^4(kp^4 + 2p^3 + 2p^2 + 2)$, hence $a = p^2 + 2p + 1 + k(kp^4 + 2p^3 + 2p^2 + 2)$. If $k \ge 1$, then we have $a > p^4$ which contradicts $a < p^4$. If $k = 0$, then we have $a = p^2 + 2p + 1 = (p+1)^2$. In this case, by $p \ge 3$ we get $1 < (p+1)^2 < (p^2)^2 = p^4$ and then $1 \le a < p^4$. Therefore, the number of pairs $(p, a)$ satisfying the condition is equal to the number of odd prime $p$ such that $(p+1)^2 \le 2024$, and there exist 13 such primes $p = 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43$.
We have proved that the total number of pairs $(p, a)$ satisfying the condition is $3 + 13 = 16$.