Automated machine learning through evolutionary computation – interview with Professor Risto Miikkulainen at SRA 2019

With our CEO as one of the co-organizers, we were happy to be a part of the 5th Systemic Risk Analytics conference at the Bank of Finland, together with RiskLab at Arcada, Bank of Finland and European Systemic Risk Board. At the event we interviewed one of the keynote speakers, the leading academic of evolutionary computation Risto Miikkulainen, who is a Professor of Computer Science at the University of Texas at Austin and AVP of Evolutionary AI at Cognizant.

A Finn who moved to the US to pursue his PhD in Computer Science at UCLA in the mid 1980s, Risto has led an impressive academic career in computer science and artificial intelligence, stepping into startups and entrepreneurship as CTO of San Francisco-based Sentient Technologies in the mid 2010s. Risto is most renowned for his award-winning research on neuroevolution and evolutionary computation, a type of evolution based automated machine learning.

At the Systemic Risk Analytics 2019 you talked about creative AI, by which you mean evolutionary computation. Evolutionary computation simulates evolution on a computer, leveraging learning evolution models that use evolutionary principles for automated problem solving. What makes the process creative?

By creativity I mean the ability to design engineering solutions that are innovative, very complex and perform better than any existing solution. In other words, the automated evolutionary system is able to come up with a solution that can be something completely new with superior performance, often surprising even the experts in the field.

At Sentient we built two applications that leveraged these technologies, one for trading stock options and the other for website optimization. Even though the solutions were for different industries and quite different from one another, the technology made both the solutions able to come up with strategies that no human would be able to invent, for the most part because of the biases we humans have.

How long have you been excited about evolutionary computation?

The seed was planted at the 1988 summer school, where evolution of neural networks first came up as a potential future research topic in the discussions among students. I worked on other topics for decades, including natural language processing and computational modeling of the visual cortex. But starting from 1991, I always had a few students who were interested in evolution, and it gradually grew into a bigger and bigger part of my lab. By about 2004, computational resources had increased sufficiently so that these ideas started to scale up, resulting in some exciting demonstrations such as the NERO machine learning video game. And now it has matured into a technology that can take AI to the realm of creativity.

What makes evolutionary computation more interesting to you than deep learning?

Evolutionary computation is different than what everyone else is doing. When comparing with the mainstream AI which is deep learning (mostly supervised training on an existing data set), evolutionary computation has ways to create something completely new, which makes it really interesting to me. This is possible thanks to evolutionary computation learning mechanism, where the models learn based on the interaction with the environment, somewhat similarly as in reinforcement learning. However, in evolution there is a whole population of models, which means there are more opportunities to explore and discover surprising solutions.

How has the new wave of research in the evolutionary computation developed?

The development of evolutionary computation has gone very much in hand with the development of deep neural networks (DNNs). For a long time, the theory was already there, but because of the lack of computational power, it was hard to realize many of the ideas. 

Most importantly, it has recently become possible to use evolution to design better deep learning architectures. This requires a lot of computational resources, and is still limited by them. However, we already see that the exploration that goes on in the evolving population results in surprising deep learning architectures that perform better than those that are designed by humans.

How can businesses leverage creative AI?

We’re still in the very early days of evolutionary computation. Deep neural networks are already mainstream, but with creative AI, the challenge is to get people to think more creatively, that is, to let evolution to explore and come up with novel solutions. You can’t leverage evolutionary computation for all the problems, but for some it will be very interesting, something we've never seen before. 

There’s a great collection of examples of how evolutionary computation can result in surprising solutions, summarised in this article. Most use cases, including the ways how evolution has been leveraged to explore new models, are still quite abstract and theoretical, but it is only a matter of time before we start seeing more and more business cases.

As with natural evolution, you have to be careful as sometimes the evolution doesn’t work the way you intend it to, which in these simulated environments can result in discovering bugs and cheats. But if you do it right, you can avoid such solutions and find truly creative and useful ones.

Evolutionary computation is a form of automated machine learning (AutoML). You’ve said that AutoML is a way of democratizing AI. How do you envision this happening?

Services are already available and in a year or two they can be useful more broadly. The pace of development is really fast. Already today, thanks to the tools like Tensorflow and Keras, you can use AI without fully understanding what you’re doing. But you still need tremendous expertise to take full advantage of it. In a couple of years automated machine learning will become highly popular and we will be able to automate many of the machine learning processes. We don’t have to build the models ourselves, but they will create themselves, through evolution. Thus, in the future, AI will not just imitate what we do, but help us be more creative.

Thank you, Risto for the interview! To stay up-to-date on the world's top AI research and real-world business applications, sign-up for the Silo.AI monthly newsletter.