National Math Olympiad in Slovenia

The answer is $2$. If $a = 4$ and $b = 3$, then $3a^2 - ab^2 - 2b - 4 = 2$.

It therefore suffices to show that the equation $3a^2 - ab^2 - 2b - 4 = 1$ has no solutions in positive integers. We can rewrite the equation as $3a^2 - ab^2 = 2b + 5$. The right-hand side is odd, so the left-hand side must be odd as well. So, $a$ must be odd and $b$ must be even, which implies that $b^2$ is divisible by $4$, and $a^2$ gives a remainder of $1$ when divided by $4$. It follows that the left-hand side gives the remainder of $3$, and the right-hand side gives the remainder of $1$ because $2b$ is divisible by $4$. This contradiction shows that the equation has no solution in positive integers.