Consider the tuples $\underbrace{11\dots1}_{1011 \text{ times}} \underbrace{00\dots0}_{1010 \text{ times}}$ and $\underbrace{11\dots1}_{1010 \text{ times}} \underbrace{00\dots0}_{1011 \text{ times}}$. Concatenating infinitely many instances of these tuples in any order produces a sequence that satisfies the conditions of the problem. Indeed, consider any segment of $2021$ consecutive terms of such sequence. As any two consecutive full blocks of zeros or ones contain at least $2020$ terms in total, the segment under consideration contains terms of at most three such blocks. If it contains terms of three blocks then it contains the middle block fully. If the segment contained only two partial blocks of zeros or ones, it could contain at most $2020$ terms in total. Hence the segment under consideration must contain one full block even if it contains only terms of two consecutive blocks. In any case, the digit of the full block occurs either $1010$ or $1011$ times and the other digit must occur either $1011$ or $1010$ times, respectively. Combining the tuples defined at the beginning of the solution according to some non-periodic pattern (e.g., $xyxxyxxxyxxxxy\dots$), the resulting sequence is not periodic from any place on. Indeed, suppose the contrary; let the length of the period be $p$. We can find a pattern of length $\text{lcm}(p, 2021)$ starting from the place where periodicity starts, containing the tuples defined at the beginning of the solution a full number of times. But the middle term of these tuples is not repeating periodically, implying that the entire pattern also cannot be periodic. Consequently, there exist non-periodic sequences that satisfy the conditions of the problem.