We show first that if A is a matrix from Mn(R) such that A2+In is singular, then there exists a matrix B in Mn(R), such that AB=−BA.
As A2+In is singular, i is a proper value of A, and −i is a proper value of the transpose Aτ. There exist two proper vectors x and y in Mn,1(C), so that Ax=ix and Aτy=−iy. Because x and y are nonzero, B=xyτ is a nonzero matrix in Mn(C).
The following relations show that A and B anticommute:
AB=Axyτ=ixyτ=−x(−iy)τ=−x(Aτy)τ=−xyτA=−BA.
Taking conjugates and using that A is in Mn(R), we get that the conjugate Bˉ of B anticommutes with A. This proves that any linear combination with complex coefficients of the matrices B and Bˉ anticommutes with A. Consequently, B or i(B−Bˉ) is a nonzero matrix in Mn(R) that anticommutes with A. This proves the first part of the problem.
We shall prove the converse. Let B be a nonzero matrix in M4(R) that anticommutes with A. Then
AkB=(−1)kBAk,k∈N.(∗)
Consider the characteristic polynomial f of A,
f=λ4−(trA)λ3+aλ2−(trA∗)λ+detA=λ4−(trA)λ3+aλ2−(trA)λ+detA, where a is a real number (the latter form of f is a consequence of trA=trA∗).
By Hamilton-Cayley, f(A)=O4. Taking into account (∗), we successively obtain:
O4=f(A)B=B(A4+(trA)A3+aA2+(trA)A+(detA)I4)=B(f(A)+2(trA)(A2+I4)A)=2(trA)B(A2+I4)A.
Because trA=0 and A is invertible, we obtain that B(A2+I4)=O4. As B is nonzero we conclude that A2+I4 is singular.