@reiver

A Good Example For Understanding The Use Of Randomness To Solve Problems: Tangled Keys

Here is (I think) a good physical world example of using randomness to solve problems, that (I think) provides a nice analogy useful for understanding using randomness to solve algorithmic problems, including in machine learning.

Tangled Keys

In fact, this just happened to me.

I was at the gym. I pulled my keys out of my pocket. But they were tangled.

Somehow several keys somehow got themselves awkwardly lodged through the key ring and against each other.

Untangling My Keys, Try 1

I started trying to untangle them.

I tried to pull the keys out of the key ring. But that didn't work.

I tried turning the keys. But that failed. In fact, that seemed to make it worse.

I tried pushing the keys out of the key ring. But that failed too.

Each deliberate attempt I made to untangle my keys failed.

Untangling My Keys, Try 2

Then I tried another approach that worked.

I vigorously yet gently shook my keys for a second or two, and they became untangled! :-)

None of my deliberate attempts to untangle my keys worked. Sure I could have kept on trying. But how long would that have taken‽

Shaking them worked. And it worked pretty fast!

Randomness

Think about what happened when I shook my keys.

All sorts of different twists and turns, pushes and pulls happened!

When I deliberately tried to untangle my keys, I also tried a number of twists, turns, pushes and pulls.

But when I shook my keys, two things happened.

#1: the number of different twists, turns, pushes and pulls increased a lot.

#2: different twists, turns, pushes and pulls that I didn't try and perhaps wouldn't have considered trying happened.

By shaking my keys, I effectively randomly tried different sequences of twists, turns, pushes and pulls, to untangle them.

And it worked. Even more, it worked quickly.

Randomness In Silico

There are equivalents of "shaking your keys to untangle them" for solving algorithmic problems, including in machine learning.

They often gets called "randomness" and often involved random number generators.

Randomness By Any Other Name …

Note that the use of randomness (to solve problems) sometimes gets hidden behind other fancy sounding names.

Names like: "stochastic", "non-deterministic", "Monte Carlo simulation", "drawing from a distribution", etc ….

Caveats

And there are important details to having a deep understanding of the different randomness techniques. But that's not the point. The point is, did this analogy help with yours or someone else's understanding.

--