(1) Since point F(1,0) is inside the circle M:(x+1)2+y2=36, circle N is internally tangent to circle M. Therefore, ∣NM∣+∣NF∣=6>∣FM∣. According to the definition of an ellipse, the trajectory of the center N is an ellipse, with 2a=6 and c=1. Thus, a2=9 and b2=8. Therefore, the equation of the trajectory of the center N of the moving circle is 9x2+8y2=1.
(2) Let P(x0,y0), A(x1,y1), S(xS,0), and T(xT,0). Then B(x1,−y1). Given that x0=±x1, we have kAP=x1−x0y1−y0, and the equation of line AP is y−y1=kAP(x−x1). Setting y=0, we get xS=y1−y0x0y1−x1y0, and similarly xT=(−y1)−y0x0(−y1)−x1y0=y1+y0x0y1+x1y0.
Thus, ∣OS∣⋅∣OT∣=∣xSxT∣=∣y1−y0x0y1−x1y0⋅y1+y0x0y1+x1y0∣=∣y12−y02x02y12−x12y02∣.
Since P(x0,y0) and A(x1,y1) are on the ellipse 9x2+8y2=1, we have y02=8(1−9x02),y12=8(1−9x12), thus y12−y02=98(x02−x12),x02y12−x12y02=8x02(1−9x12)−8x12(1−9x02)=8(x02−x12).
Therefore, ∣OS∣⋅∣OT∣=∣y12−y02x02y12−x12y02∣=∣98(x02−x12)8(x02−x12)∣=9.