a.
We have
P∗(z)=aˉ0zd+aˉ1zd−1+⋯+aˉd=zd(aˉ0+aˉ1z1+⋯+aˉd(z1)d)
so
P∗(z)=zd(a0+a1z1+⋯+ad(z1)d)=zdP(z1)
b.
Let z1,z2,…,zn be roots (not necessarily distinct) of q. So
q(z)=(z−z1)⋯(z−zn)
By assumptions of the problem for every 1≤i≤n we know that ∣zi∣≤1. Using the first part of the problem we have
q∗(z)=znq(z1)=zn(z1−z1)⋯(z1−zn)
so we have
q∗(z)=(1−zz1)⋯(1−zzn)
Now assume that r is a root of polynomial Q. Then
Q(r)=rmq(r)+q∗(r)=0
so
rm(r−z1)⋯(r−zn)=−(1−rzˉ1)⋯(1−rzˉn)
which implies
∣r∣m∣r−z1∣⋯∣r−zn∣=∣1−rzˉ1∣⋯∣1−rzˉn∣
Case 1: ∣r∣>1
We claim that for every 1≤i≤n, we have ∣r−zi∣≥∣1−rzˉi∣. We have
∣r−zi∣2=(r−zi)((r−zi))=∣r∣2−rzˉi+rˉzi+∣zi∣2∣1−rzˉi∣2=(1−rzˉi)((1−rzˉi))=1−rzˉi−rˉzi+(∣r∣∣zi∣)2
So it is enough to prove that
∣r∣2+∣zi∣2≥1+(∣r∣∣zi∣)2
Or equivalently
(∣r∣2−1)(1−∣zi∣2)≥0
Which is true. So our claim is proved. So according to the inequality we must have
∣r∣m∣r−z1∣⋯∣r−zn∣>∣r−z1∣⋯∣r−zn∣≥∣1−rzˉ1∣⋯∣1−rzˉn∣
Which is a contradiction. So ∣r∣≤1.
Case 2: ∣r∣<1
Same as the previous part we would have ∣r−zi∣≤∣1−rzˉi∣ which also leads to a contradiction and implies ∣r∣≥1. Thus we can conclude that ∣r∣=1.