In essence, your question is about convergence of infinite series. The mathematical discipline that studies such series is hundreds (if not thousands) years old an has nothing to do with "hardware architecture".
A basic example of an infinite series is the geometric series:
$$ S = 1 + \gamma + \gamma^2 + \gamma^3 + \dots$$
Note that the series is infinite - no one says that $\gamma^k$ "becomes zero at sufficiently large $k$". To compute the infinite sum we represent it as a limit of partial sums:
$$ S = \lim_{k\to\infty} ( 1 + \gamma + \gamma^2 + \gamma^3 + \dots + \gamma^k)$$
The partial sum is called the geometric progression and has a well-known expression for it:
$$ 1 + \gamma + \gamma^2 + \gamma^3 + \dots + \gamma^k = \frac{1-\gamma^{k-1}}{1-\gamma}$$
As a result, the sum of our geometric series is:
$$ S = \lim_{k\to\infty} \frac{1-\gamma^{k-1}}{1-\gamma} = \frac{1}{1-\gamma}\quad\text{if}\;|\gamma| < 1$$
In Reinforcement Learning we are dealing with similar infinite series. In partcular, when we are talking about returns:
$$R_t = r_{t+1} + \gamma r_{t+2} + \gamma^2 r_{t+3} + \dots = \sum_{k=0}^\infty\gamma^kr_{t+k+1}$$
The convergence of this is acutally discussed in the Sutton and Barto's book:
If $\gamma < 1$, the infinite sum has a finite value as long as the reward sequence $\{r_k\}$ is bounded.
I guess, the simplest way to prove that would be to use the direct comparison test. Having $\{r_k\}$ bounded means that there exist such number $C$ so that $|r_k| \leq C$ for all $k$. So we've got the dominating series for the return :
$$ \left|\gamma^kr_{t+k+1}\right| \leq C |\gamma^k|$$
And, since the $\gamma^k$ series absolutely converges for $0 \leq \gamma < 1$, so does $\gamma^kr_{t+k+1}$ series.
