Observation 1. If the situation is possible for a certain number $m$ of tokens, then it is also possible for a number $m' > m$: we can add $m' - m$ tokens with different sizes and colors to a suitable configuration with $m$ tokens. This shows that it is sufficient to find the minimum number of tokens for which we can obtain the desired configuration.

Observation 2. If each of the numbers $0, 1, \ldots, n - 1$ appears at least once on the tokens, then the table has rows (or columns) with exactly $1, 2, \ldots, n$ tokens, respectively.

Property. We will show that if $n \equiv 2 \pmod{3}$, then the table has at least $\frac{1}{3} n(n+1)$ tokens.

Let $n = 3k - 1$, and consider $2k$ rows filled with $k, k + 1, \ldots, 3k - 1$ tokens. Counting, possibly some tokens multiple times, these rows contain a total of
$$N = k + (k + 1) + (k + 2) + \dots + (3k - 1) = k(4k - 1)$$
tokens. Each token can be counted at most twice. If we denote by $x$ the number of rows and by $y$ the number of columns participating in obtaining $N$, there are at most $xy$ tokens counted twice. Since $x + y = 2k$, we have $xy \le k^2$, so the rows participating in obtaining $N$ have at least $N - k^2 = k(3k - 1) = \frac{1}{3} n(n + 1)$ tokens.

Example. A configuration with exactly $k(3k-1)$ tokens can be obtained from a $(2k-1) \times (3k-1)$ matrix where, starting from the first column, we place $1, 2, \dots, k-1$ tokens on the first $k-1$ rows, and $2k, 2k+1, \dots, 3k-1$ tokens on the remaining rows. The number of tokens is $(1+2+\dots+(k-1)) + (2k+(2k+1)+\dots+(3k-1)) = \frac{k(k-1)}{2} + k(5k-1) = k(3k-1) = \frac{1}{3} n(n+1)$, the rows represent rows with $1, 2, \dots, k-1, 2k, 2k+1, \dots, 3k-1$ tokens, and the first $k$ columns represent rows with $2k-1, 2k-2, \dots, k$ tokens.