AI to enhance Human Imagination – what about Reality?

Posted by Peter Rudin on 17. July 2020 in Essay

Two surprised Owls         Picture Credit: Wikimedia


Human imagination occurs as a mental state and is a prerequisite for creativity and innovation. It implies our ability to produce and simulate novel objects, without any immediate input of the senses. It is also described as the forming of experiences in one’s mind, which can be re-creations of past experiences or be completely invented with no relationship to the past. Imagination helps make knowledge applicable in solving problems and is fundamental for integrating experience and learning. Human active imagination is a conscious method of experimentation. In the 1970s, active imagination was most strongly associated with C. G. Jung’s experiments with clients emphasizing its therapeutic value. Beginning in the 1990s, active imagination began to be used in commercial contexts, for example in architecture or molecular biology based on a mental process also dubbed as ‘Design Thinking’. Imagination can also be expressed through stories such as fairy tales or fantasies. Children often use such narratives and pretend-play to exercise their imaginations. In contrast, artificial imagination, also called synthetic imagination or machine imagination, is defined as the simulation of human imagination by computer hardware and artificial neural network software. Some of the traits that researchers hope to simulate include creativity, vision, digital art, humour or satire. Artificial imagination research applies tools and insights from many fields, including computer science, psychology, creative arts, philosophy, neuroscience and linguistics.

Differentiating between imagination and reality 

Understanding the distinction between what is real and what is imagined is an important step in the development of children’s imaginative abilities. Current evidence suggests that this distinction is in place by around age three, enabling children to discriminate between real and pretend-identities of objects and also to explicitly separate real from imagined events. Children also organize multiple fictional worlds beyond this binary distinction and by the time they enter grade school, understand the more subtle issues of probability and transfer of fictional information into reality. Despite this positive learning process, children do demonstrate some imagination-reality confusions, as measured by their implicit reactions to certain kinds of pretend-scenarios and their occasionally incorrect reports about fictional experiences. One might conclude that these confusions reflect a fundamentally immature capacity to distinguish imagination and reality. However, on balance it seems more accurate to say that children’s imagination–reality distinction is well developed but can be influenced by various features of the task measuring it. Strong negative emotions can lead children to act as if a pretended entity had become real. Likewise, children may differ in the intensity of their division between imagination and reality. Individual differences in fantasy scenarios, imaginary companions and natural levels of scepticism can oscillate children’s perception and reactions to imagined situations. However, cases where children make errors in properly separating imagination from reality do not reveal particularly strong differences to adult behaviour. According to Jacqueline D. Woolley, Professor of Psychology at the University of Texas at Austin, the difference between children and adults likely lies in children’s relatively lesser experience and knowledge, rather than in any radically different cognitive processing of imagined information. A different possibility is that infants from the very earliest ages are equipped with the cognitive machinery to understand that some actions take place in a fictional rather than the real world. In this view, what changes as children develop is the ability to verbally express their understanding of this distinction, rather than the understanding itself.

Generative Adversarial Networks (GANs) as source for artificial imagination

One of the computational processes that provides AI with imaginative powers is referred to as Generative Adversarial Networks. GANs consist of two neural networks which compete against – and learn from – each other. An essential element of GANs is that they can assess their own work. For example to generate art as an expression of an artist’s imagination, one network  (called the generator) generates a flow of images out of nothing while the other (called the judge), trained on specific images representing a desired style of painting, accepts or rejects them. In this way, the generator learns and creates better and better work from scratch. GANs produce art based on the way we think and how we emotionally value art. But ultimately, we may build machines that can go beyond our comprehension and are intelligent enough to create art of their own, without human intervention. Likewise, advances in GAN-research could open new scientific avenues for us to explore and support medical research or suggesting ways of dealing with our failing climate and growing ecological crisis. GANs imaginary ability to create is not limited to image generation. Imagination is, in a broader sense, a tool for discovery. Modern cybersecurity tools feature GANs that can detect threats by looking at their characteristics. Researchers have designed GANs which learn to generate pieces of malignant code, bypassing these cybersecurity detectors. That may sound scary, but we can use the results to perfect detectors of malignant code. GANs are a turning point in the development of AI as they will help us to augment our mental capacities. They are a tool with which we can study the mechanisms of human imagination to better understand its potential role in domains such as behaviour or personal development.

A neuroscientific hypothesis about the source of imagination

Visualizing new, never-before-seen images in your mind, involves a synthesis of several familiar mental images tucked away in your brain. Andrey Vyshedskiy, Neuroscientist and Professor at Boston University calls this process mental synthesis. Mental synthesis is a key component of what we commonly call ‘imagination,’ along with other components such as dreaming or hallucinating. It is responsible for many uniquely human traits, such as mental planning, modelling or engineering. Humans can purposefully and deliberately create and visualize a seemingly endless array of novel

images in the mind, but the neurological process underlying this essential human skill is not well understood. In a 2017 TED animation video, Vyshedskiy provides an easy-to-understand demonstration of his mental synthesis model. Imagine a dolphin balancing a pineapple on his nose – you probably have not actually seen this, but you could imagine it instantly. How does your brain produce an image of something you have never seen? That may not seem hard, but it turns out that this is a complex problem that requires sophisticated coordination inside your brain. When you look at an object, thousands of neurons in your prefrontal cortex brain region fire-up. These neurons encode various characteristics of an object, in case of a pineapple: spiky, fruit, brown, green, and yellow. This synchronous firing strengthens the connections between the corresponding set of neurons, linking them together into what is known as a neuronal ensemble. In neuroscience, this is called the Hebbian principle: neurons that fire together wire together. If you try to imagine a pineapple later, the whole neuronal ensemble will light up, assembling a complete mental image. Dolphins are encoded by a different neuronal ensemble but the neuronal ensemble for a dolphin balancing a pineapple does not exist. So how come you can imagine it anyway? The hypothesis of the Mental Synthesis Theory says that if the neuronal ensembles for the dolphin and the pineapple are activated at the same time, we can perceive the two separate objects as a single image. If the neuronal ensembles are turned on at the same time, you experience the composite image just as if you had actually seen it. However, for mental synthesis to work, imagination-triggered signals must arrive at both neuronal ensembles at the same time. The problem is that some neurons are much farther away from the prefrontal cortex than others. If the signals of both neural ensembles travel along their corresponding neural fibers at the same speed, they will most likely arrive out of sync. You cannot change the length of the connections but your brain, especially as it develops in childhood, does have a way to change the conduction velocity. Neural fibers are wrapped in a fatty substance called myelin. Myelin is an insulator and speeds up the electrical signals zipping down the nerve fiber. Some neural fibers have as many as 100 layers of myelin, others only have a few. Fibers with thicker layers of myelin can conduct signals 100 times faster or more than those with thinner ones. Some scientists now think that this difference in myelination could be the key to uniform conduction time in the brain and consequently to our ability for mental synthesis. A lot of this myelination process happens in childhood. Hence our capacity to imagine may have a lot to do with the way we actively use our brains starting at an early age.


The information that stipulates imagination in a human is typically found in two places: in our genes and in our brains. The information in our genes is digital and coded in the four-level alphabet of DNA. How information is stored and processed in our brains is still a great mystery. Nobody yet knows how the human memory works. It seems likely that memories are recorded in variations of the strengths of synapses connecting the billions of neurons in the brain with one another. Yet there are smells around that we physically cannot smell, there are sights to witness beyond the wavelength that our eyes operate in and there are sounds to be heard that we will never fully hear. Humans are programmed for survival. The environments that we have adapted to do not need us to be able to see like a snake does or to smell to the extent that a dog can. Everything we take for granted as reality is, in fact, an illusion of sorts.  Yet the global problems we face today are real and our capacity for imagination is very much needed to create innovative solutions to solve these problems.

Leave a Reply

Your email address will not be published. Required fields are marked *