Consider P(x,y,z)=(y−x)2+(z−y)2. Then, P(n,n+m,n+2m)=2m2. Clearly, r>2 does not work. We show that r=2.
If there exist (a1,b1,c1), (a2,b2,c2) such that P(a1,b1,c1)>0 and P(a2,b2,c2)<0, draw a continuous path from (a1,b1,c1) to (a2,b2,c2), not passing through the line x=y=z.
Since P is continuous, there exists (a3,b3,c3) on the path such that P(a3,b3,c3)=0. This is a contradiction as the condition that a3=b3=c3 is not satisfied. Hence, either all P≤0 or P≥0.
Let u=y−x, v=z−y. P(x,y,z)=Q(x,u,v)=uvQ1(x,u,v)+u2Q2(x,u,v)+v2Q3(x,u,v)+uR1(x)+vR2(x)+R3(x). Substituting u,v=0, x=a, we have R3(a)=0 for all real a⇒R3(x)=0. Substituting v=0, x=a, we have Q(a,u,0)=u2Q2(a,u,0)+uR1(a). However, since P≥0 or P≤0, for any fixed real a, there must be a double root at u=0. Hence, R1(a)=0 for all a⇒R1(x)=0. Similarly, R2(x)=0.
Since m∣u,v, we have that m2 divides P(n,n+m,n+2m)=Q(n,m,m)=m2(Q1(n,m,m)+Q2(n,m,m)+Q3(n,m,m)).