6
$\begingroup$

How important is true (non-pseudo) randomness in Artificial Intelligence designs? Is there any chance that pseudo-randomness could be a barrier to more successful designs?

$\endgroup$
5
$\begingroup$

Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right.

For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under investigation.

But randomness only works for this in expectation. If we actually knew what the confounders were, we could do a paired assignment (or a similar scheme) that ensured the various groups were matched as well as possible, instead of us just not knowing ahead of time which way the bias went.


There are some cases where pseudorandomness, rather than full randomness, will impair training AI designs. A simple example would be a case where you want to randomly initialize weights in a network where the number of parameters exceeds the periodicity of the RNG; this means that while you have as many possible networks as there are possible unique seeds, you can't actually visit the entire weight space that you wanted to sample over.

I don't think any of those cases are limiting factors, however. Having truly random stochastic gradient descent instead of pseudorandom stochastic gradient descent doesn't seem like it would make a serious difference in the trajectory of AI designs.

$\endgroup$
3
  • 1
    $\begingroup$ Is the number of parameters an issue, given that a modern RNG like Mersenne Twister has a period of (2^19968) - 1? $\endgroup$ Aug 4 '16 at 19:41
  • $\begingroup$ @user217281728 I'm starting to step outside of my area of expertise, but I'd guess it wouldn't for something like the Mersenne Twister. (That would require a truly huge AI model to exhaust!) $\endgroup$ Aug 5 '16 at 13:39
  • $\begingroup$ @NietzscheanAI I think that's a really good question. I suggest you post that as a proper question in ai.stackexchange $\endgroup$ Mar 12 '18 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.