Solution 1. In the first two steps, we deal with any polynomial P(x,y,z) satisfying P(x,y,z)=P(x,y,xy−z). Call such a polynomial weakly symmetric, and call a polynomial satisfying the full conditions in the problem symmetric.
Step 1. We start with the description of weakly symmetric polynomials. We claim that they are exactly the polynomials in x,y, and z(xy−z). Clearly, all such polynomials are weakly symmetric. For the converse statement, consider P1(x,y,z):=P(x,y,z+21xy), which satisfies P1(x,y,z)=P1(x,y,−z) and is therefore a polynomial in x,y, and z2. This means that P is a polynomial in x,y, and (z−21xy)2=−z(xy−z)+41x2y2, and therefore a polynomial in x,y, and z(xy−z).
Step 2. Suppose that P is weakly symmetric. Consider the monomials in P(x,y,z) of highest total degree. Our aim is to show that in each such monomial μxaybzc we have a,b⩾c. Consider the expansion
P(x,y,z)=i,j,k∑μijkxiyj(z(xy−z))k(1.1)
The maximal total degree of a summand in (1.1) is m=maxi,j,k:μijk=0(i+j+3k). Now, for any i,j,k satisfying i+j+3k=m the summand μi,j,kxiyj(z(xy−z))k has leading term of the form μxi+kyj+kzk. No other nonzero summand in (1.1) may have a term of this form in its expansion, hence this term does not cancel in the whole sum. Therefore, degP=m, and the leading component of P is exactly
i+j+3k=m∑μi,j,kxi+kyj+kzk
and each summand in this sum satisfies the condition claimed above.
Step 3. We now prove the problem statement by induction on m=degP. For m=0 the claim is trivial. Consider now a symmetric polynomial P with degP>0. By Step 2, each of its monomials μxaybzc of the highest total degree satisfies a,b⩾c. Applying other weak symmetries, we obtain a,c⩾b and b,c⩾a; therefore, P has a unique leading monomial of the form μ(xyz)c. The polynomial P0(x,y,z)=P(x,y,z)−μ(xyz−x2−y2−z2)c has smaller total degree. Since P0 is symmetric, it is representable as a polynomial function of xyz−x2−y2−z2. Then P is also of this form, completing the inductive step.
Solution 2. We will rely on the well-known identity
cos2u+cos2v+cos2w−2cosucosvcosw−1=0whenever u+v+w=0.(2.1)
Claim 1. The polynomial P(x,y,z) is constant on the surface
S={(2cosu,2cosv,2cosw):u+v+w=0}
Proof. Notice that for x=2cosu,y=2cosv,z=2cosw, the Vieta jumps x↦yz−x, y↦zx−y,z↦xy−z in ( ∗ ) replace ( u,v,w ) by ( v−w,−v,w ), ( u,w−u,−w ) and ( −u,v,u−v ), respectively. For example, for the first type of jump we have
yz−x=4cosvcosw−2cosu=2cos(v+w)+2cos(v−w)−2cosu=2cos(v−w).
Define G(u,v,w)=P(2cosu,2cosv,2cosw). For u+v+w=0, the jumps give
G(u,v,w)=G(v−w,−v,w)=G(w−v,−v,(v−w)−(−v))=G(−u−2v,−v,2v−w)=G(u+2v,v,w−2v)
By induction,
G(u,v,w)=G(u+2kv,v,w−2kv)(k∈Z).(2.2)
Similarly,
G(u,v,w)=G(u,v−2ℓu,w+2ℓu)(ℓ∈Z).(2.3)
And, of course, we have
G(u,v,w)=G(u+2pπ,v+2qπ,w−2(p+q)π)(p,q∈Z)(2.4)
Take two nonzero real numbers u,v such that u,v and π are linearly independent over Q. By combining (2.2-2.4), we can see that G is constant on a dense subset of the plane u+v+w=0. By continuity, G is constant on the entire plane and therefore P is constant on S.
Claim 2. The polynomial T(x,y,z)=x2+y2+z2−xyz−4 divides P(x,y,z)−P(2,2,2).
Proof. By dividing P by T with remainders, there exist some polynomials R(x,y,z),A(y,z) and B(y,z) such that
P(x,y,z)−P(2,2,2)=T(x,y,z)⋅R(x,y,z)+A(y,z)x+B(y,z).(2.5)
On the surface S the LHS of (2.5) is zero by Claim 1 (since (2,2,2)∈S ) and T=0 by (2.1). Hence, A(y,z)x+B(y,z) vanishes on S.
Notice that for every y=2cosv and z=2cosw with 3π<v,w<32π, there are two distinct values of x such that (x,y,z)∈S, namely x1=2cos(v+w) (which is negative), and x2=2cos(v−w) (which is positive). This can happen only if A(y,z)=B(y,z)=0. Hence, A(y,z)=B(y,z)=0 for ∣y∣<1,∣z∣<1. The polynomials A and B vanish on an open set, so A and B are both the zero polynomial.
The quotient (P(x,y,z)−P(2,2,2))/T(x,y,z) is a polynomial of lower degree than P and it also satisfies (*). The problem statement can now be proven by induction on the degree of P.
Solution 3 (using algebraic geometry, just for interest). Let Q=x2+y2+z2−xyz and let t∈C. Checking where Q−t,∂x∂Q,∂y∂Q and ∂z∂Q vanish simultaneously, we find that the surface Q=t is smooth except for the cases t=0, when the only singular point is (0,0,0), and t=4, when the four points (±2,±2,±2) that satisfy xyz=8 are the only singular points. The singular points are the fixed points of the group Γ of polynomial automorphisms of C3 generated by the three Vieta involutions
ι1:(x,y,z)↦(x,y,xy−z),ι2:(x,y,z)↦(x,xz−y,z),ι3:(x,y,z)↦(yz−x,y,z).
Γ acts on each surface Vt:Q−t=0. If Q−t were reducible then the surface Q=t would contain a curve of singular points. Therefore Q−t is irreducible in C[x,y,z]. (One can also prove algebraically that Q−t is irreducible, for example by checking that its discriminant as a quadratic polynomial in x is not a square in C[y,z], and likewise for the other two variables.) In the following solution we will only use the algebraic surface V0.
Let U be the Γ-orbit of (3,3,3). Consider ι3∘ι2, which leaves z invariant. For each fixed value of z, ι3∘ι2 acts linearly on (x,y) by the matrix
Mz:=(z2−1z−z−1)
The reverse composition ι2∘ι3 acts by Mz−1=Mzadj. Note detMz=1 and trMz=z2−2. When z does not lie in the real interval [−2,2], the eigenvalues of Mz do not have absolute value 1, so every orbit of the group generated by Mz on C2∖{(0,0)} is unbounded. For example, fixing z=3 we find (3F2k+1,3F2k−1,3)∈U for every k∈Z, where (Fn)n∈Z is the Fibonacci sequence with F0=0,F1=1.
Now we may start at any point (3F2k+1,3F2k−1,3) and iteratively apply ι1∘ι2 to generate another infinite sequence of distinct points of U, Zariski dense in the hyperbola cut out of V0 by the plane x−3F2k+1=0. (The plane x=a cuts out an irreducible conic when a∈/{−2,0,2}.) Thus the Zariski closure Uˉ of U contains infinitely many distinct algebraic curves in V0. Since V0 is an irreducible surface this implies that Uˉ=V0.
For any polynomial P satisfying (*), we have P−P(3,3,3)=0 at each point of U. Since Uˉ=V0, P−P(3,3,3) vanishes on V0. Then Hilbert's Nullstellensatz and the irreducibility of Q imply that P−P(3,3,3) is divisible by Q. Now (P−P(3,3,3))/Q is a polynomial also satisfying (*), so we may complete the proof by an induction on the total degree, as in the other solutions.