Let d=gcd(a,b) and x,y∈Z+ such that a=dx,b=dy. Obviously, (x,y)=1. The equation is equivalent to d4x2y2+208=4d2(xy+1)2. Hence d2∣208 or d2∣13⋅42, so d∈{1,2,4}. Take t=xy with t∈Z+.
Case I. If d=1, then (xy)2+208=4(xy+1)2 or 3t2+8t−204=0, without solutions.
Case II. If d=2, then 16x2y2+208=16(xy+1)2 or t2+13=t2+2t+1⇒t=6, so (x,y)∈{(1,6);(2,3);(3,2);(6,1)}⇒(a,b)∈{(2,12);(4,6);(6,4);(12;2)}.
Case III. If d=4, then 162x2y2+208=4⋅16(xy+1)2 or 16t2+13=4(t+1)2 and if t∈Z, then 13 must be even, contradiction!
Finally, the solutions are (a,b)∈{(2,12);(4,6);(6,4);(12;2)}.