A friend pointed me at this paper, “The Overfitted Brain: Dreams evolved to assist generalization”, which made me think of some machine learning concepts that still stand out five years after I learned them.

I used to talk a lot with two engineers building GAN (Generative Adversarial Network) frameworks. One day walking back from lunch a few things came up that have stood out since. Context here is that the output of a Neural Network, the probability distribution function encoded in the network, is a manifold over the search space. In other words, if the real terrain is the problem we’re looking at then the network we trained is a map. Maybe a map in crayon, maybe a high res printout but in either case a representation.

Ok so the two interesting things were:

1. what we think of as Intelligence is ‘how well does the map represent reality?’
2. what we think of as Creativity is ‘how diverse are the generated manifolds?’
   “Oh cool, you made a crayon map that lets me understand the high level layout of the city” “This black and white map makes the major highways really stand out”.

The latter relates to that paper on dreaming. By generating all kinds of different (often crazy) representations of the world we (or algorithms) are able to develop a more complex understanding of how it all really works and fits together. Dreaming is a great way to generate those representations. Turns out so is lunch with friends!

So remember, “Lunch is the most important meeting of the day“. And creative pursuits are key to improving our understanding of the world. Keep dreaming.

Photo by Trinity Kubassek: https://www.pexels.com/photo/sheep-288621/
Photo by Nishant Meena: https://www.pexels.com/photo/lighting-strikes-over-city-during-night-time-12238683/