The probability that a set of three distinct vertices chosen at random from among the vertices of a regular n-gon determine an obtuse triangle is . Find the sum of all possible values of .
Solution
Inscribe the regular polygon inside a circle. A triangle inside this circle will be obtuse if and only if its three vertices lie on one side of a diameter of the circle. (This is because if an inscribed angle on a circle is obtuse, the arc it spans must be 180 degrees or greater).
Break up the problem into two cases: an even number of sides , or an odd number of sides . For polygons with sides, the circumdiameter has endpoints on vertices. There are points on one side of a diameter, plus of the endpoints of the diameter for a total of points. For polygons with points, the circumdiameter has endpoint on a vertex and endpoint on the midpoint of the opposite side. There are also points on one side of the diameter, plus the vertex for a total of points on one side of the diameter.
Case 1: -sided polygon. There are clearly different triangles total. To find triangles that meet the criteria, choose the left-most point. There are obviously choices for this point. From there, the other two points must be within the points remaining on the same side of the diameter. So our desired probability is
so
.
and so the polygon has sides.
Case 2: -sided polygon. Similarly, total triangles. Again choose the leftmost point, with choices. For the other two points, there are again possibilities.
The probability is
so
and our polygon has sides.
Adding,