Let $P$ be the set of all prime divisors of members of $S'$. Suppose, to the contrary, that $P$ is finite and $P = \{p_1, p_2, \dots, p_n\}$. Define
$$N = 4 \prod_{i=1}^{n} p_i(p_i - 1).$$
$S$ is infinite, hence there exists an infinite subset $S_1$ of $S$ such that every two members of $S_1$ are congruent to each other modulo $N$. Let $x, y > 1$ be members of $S_1$ such that $y$ is large enough to satisfy $\frac{y}{\log_2 y} > x$. It's easy to see that
$$x < \frac{y}{\log_2 y} \le \frac{y}{\log_x y} \implies y > x \log_x y \implies x^y > y^x$$
Since $x^y + y^x \in S'$, it can be written in the form $x^y + y^x = q_1^{\alpha_1} q_2^{\alpha_2} \dots q_k^{\alpha_k}$ where $q_1, q_2, \dots, q_k \in \mathbb{P}$ and $\alpha_1, \alpha_2, \dots, \alpha_k \in \mathbb{Z}^+$. For every $1 \le i \le k$ we have
$$\begin{aligned} q_i \mid N, x \equiv y \pmod{N} &\implies x \equiv y \pmod{q_i} \\ &\implies x^y \equiv y^y \pmod{q_i} \end{aligned} \qquad (1)$$
$$\begin{aligned} q_i - 1 \mid N, x \equiv y \pmod{N} &\implies x \equiv y \pmod{q_i - 1} \\ &\implies y^x \equiv y^{x} \pmod{q_i} \end{aligned} \qquad (2)$$
According to (1), (2)
$$\left. \begin{array}{l} x^y \equiv y^x \pmod{q_i} \\ x^y + y^x \equiv 0 \pmod{q_i} \end{array} \right\} \implies 2x^y \equiv 0 \pmod{q_i}$$
So for every odd $q_i$ we have $q_i \mid x$. For $q_i = 2$ if at least one of $x$ and $y$ is even, then the other one is even too; And if they both are odd we have
$$\left. \begin{array}{l} 2 \mid x-y \implies y^y \equiv y^x \pmod{4} \\ y \equiv x \pmod{N} \\ 4 \mid N \end{array} \right\} \implies x^y \equiv y^y \equiv y^x \pmod{4} \\ \implies x^y + y^x \equiv 2x^y \equiv 2 \pmod{4} \implies \alpha_i = 1 \qquad (3)$$
Now let
$$x = q_1^{\beta_1} q_2^{\beta_2} \dots q_k^{\beta_k} x', \quad y = q_1^{\gamma_1} q_2^{\gamma_2} \dots q_k^{\gamma_k} y'$$
so that
$$\beta_i, \gamma_i \in \mathbb{Z}^+ \cup \{0\}, \text{gcd}(x', q_1, q_2, \dots, q_k) = \text{gcd}(y', q_1, q_2, \dots, q_k) = 1$$
For every $1 \le i \le k$ if $q_i \ne 2$, it was proven that $\beta_i, \gamma_i > 0$. Furthermore, if $q_i = 2$ and one of $x, y$ is even, then $\beta_i, \gamma_i > 0$. So if these conditions hold, we have $\beta_i y \ge y$. From the definition of $y$ we have
$$\beta_i y \ge y > x \log_2 y \ge x \log_{q_i} y \ge x \gamma_i.$$
Also if $x, y$ are odd and $q_i = 2$, the inequality $0 = \beta_i y \ge \gamma_i x = 0$ still holds. Now we have
$$\begin{aligned} q_1^{\alpha_1} q_2^{\alpha_2} \dots q_k^{\alpha_k} &= x^y + y^x \\ &= x'^y \prod_{1 \le i \le k} q_i^{y\beta_i} + y'^x \prod_{1 \le i \le k} q_i^{x\gamma_i} \\ &= \prod_{1 \le i \le k} q_i^{x\gamma_i} \left( y'^x + x'^y \prod_{1 \le i \le k} q_i^{y\beta_i - x\gamma_i} \right) \end{aligned}$$
If $q_i = 2$ and $x, y$ are odd, from (3) we have $\alpha_i = 1$. Otherwise we have
$$y\beta_i - x\gamma_i > 0 \implies \left( y'^x + x'^y \prod_{1\le i\le k} q_i^{y\beta_i - x\gamma_i}, q_i \right) = (y'^x, q_i) = 1$$
But we also have
y'^x + x'^y \prod_{1\le i\le k} q_i^{y\beta_i - x\gamma_i} \quad \left| \quad x^y + y^x = q_1^{\alpha_1} q_2^{\alpha_2} \dots q_k^{\alpha_k}
So
$$\begin{align*} & y'^x + x'^y \prod_{1\le i\le k} q_i^{y\beta_i - x\gamma_i} \quad \Big| \quad 2 \\ \implies \quad & y'^x + x'^y \prod_{1\le i\le k} q_i^{y\beta_i - x\gamma_i} \le 2 \\ \implies \quad & y'^x \le 1 \implies y'^x = 1 \implies y^x = \prod_{1\le i\le k} q_i^{x\gamma_i} \\ \implies \quad & x^y + y^x = q_1^{\alpha_1} q_2^{\alpha_2} \dots q_k^{\alpha_k} = y^x \left( y'^x + x'^y \prod_{1\le i\le k} q_i^{y\beta_i - x\gamma_i} \right) \le 2y^x \\ \implies \quad & x^y \le y^x \end{align*}$$
Which contradicts the way $y$ was chosen, and completes our proof. ■