The given system is equivalent to
$$\left| \begin{array}{l} x^2 - 2x + 2b - 3a = 0 \\ x + a = y + b \end{array} \right.$$
It has a unique solution $(x_0, y_0)$ if the quadratic equation $x^2 - 2x + 2b - 3a = 0$ has a unique root $x_0$. This means that $D = 1 - 2b + 3a = 0$ and $x_0 = 1$. The condition $x_0^2 + y_0^2 = 1025$ gives $y_0^2 = 1024$, i.e. $y_0 = \pm 2$. For $y_0 = -2$ we get $a = x_0^2 - 2y_0 = 5$ and $b = x_0 + a - y_0 = 8$, and for $y_0 = 2$ we get $a = x_0^2 - 2y_0 = -3$ and $b = x_0 + a - y_0 = -4$.