The answer is $n+1$.

For a harmonic set, we consider a graph $G$ with $V_1, V_2, \dots, V_n$ as its vertices and with the segments in the harmonic set as its edges.

First, we show that there are at least $n$ edges in $G$. Note that $G$ must be connected. Also note that each vertex must have degree at least $2$, because when a chess piece is moved from $V_i$ to $V_j$ there is another piece moved from $V_k$ (with $k \neq j$) to $V_i$. Hence, the total degree is at least $2n$, from which it follows that there are at least $2n = 2 = n$ edges.

Second, we show that there are at least $n+1$ edges. Assume that there are only $n$ edges. In this connected graph, each vertex has exactly degree $2$, and hence it must be a complete cycle. Without loss of generality, we may assume that the cycle $V_1 \to V_2 \to \dots \to V_n \to V_1$ consists of all the edges. In this case, if $C_1$ and $C_2$ are placed at $V_2$ and $V_1$ initially, we cannot put them back to $V_1$ and $V_2$ simultaneously. This is because we can only rotate all the pieces along the cycle and cannot change their relative positions along the cycle.

Third, we show that $n+1$ edges is enough. We consider the graph $G$ with the cycle $C_1: V_1 \to V_2 \to \dots \to V_n \to V_1$ and one additional edge, $V_2V_n$. (This graph $G$ now has the second cycle $C_2: V_2 \to V_3 \to \dots \to V_n \to V_2$.) With this additional edge, we can switch the relative positions of the chess pieces along the cycle $C_1$. Indeed, without loss of generality, we may assume that $C_i$ is at $V_1$ and $C_j$ is at $V_2$ initially. Applying rotations on the cycle $C_2$, we can place $C_j$ at $V_n$, i.e. the relative positions of $C_i$ and $C_j$, along $C_1$, are switched. Because we can switch the positions of any two neighboring pieces in a finite amount of moves, we can place $C_1, C_2, \dots, C_n$ in that order on the cycle $C_1$. We can then move each $C_i$ to $V_i$ by applying rotations along the cycle $C_1$.