Artificial intelligence (AI), a term buzzing from its immense potential to transform industries and redefine how we live, work, and think. But, as you may have noticed, we are still far from the hyperbolized visions of the super AI that can undertake complex analytical jobs and form genuine relationships with human beings.
So where are the limitations of AI today, and could those limitations be overcome? This is an article about the current edge of AI capabilities, the efforts to push beyond those boundaries, and the obstacles ahead.
Narrow and shallow
Headline AI triumphs like AlphaGo and Alexa are undoubtedly monumental achievements, but they can also be misleading about the scale and scope of what AI can do: while they excel at solving very specific, well-defined problems, AI systems struggle in the face of scenarios which differ from the examples of their training sets. That is, they have trouble generalizing and abstracting from the information they learn.
A useful way to think about what an AI capable of general abstraction would be like is the ‘coffee test’ proposed by Steve Wozniak. In this generalizability test, a robot would be able to walk into any given household on a random street and make a cup of coffee. This would require the system to quickly interpret an unknown environment, know what a house is, determine where the necessary materials are kept (knowing that they’ll likely be in the kitchen and not in a bathroom, and also distinguishing the kitchen from the bathroom), find the coffee (which might be kept in a jar, or in an unmarked container), and then coordinate the processes and equipment to make a cup of coffee.
A field of study within computer science, called transfer learning, is dedicated to this particular challenge. For instance, a technique called ‘one shot learning’ is being pursued which could train systems using only a few real world examples. This is possible by first pre-training the model in a simulated virtual environment, showing it a few (or just one) real world variations, then getting the model to apply what it learned in the virtual world to similar, but previously unseen scenarios.
A longstanding issue for AI systems, and something which has become more complex with machine learning, is the explainability of internal processes. A machine learning system could have millions, or even billions of parameters, but they don’t appear to the programmer in terms easily understood by human perception. Instead of a label like ‘year_founded’ whose function can be readily discerned, data in an AI system would look more like a numbered value indicating their weight within a network of nodes. In other words, it can be very difficult to explain how a system arrived at its conclusions.
Because of the difficulty in seeing how the system arrives at a decision, the black box problem could be a big liability in domains such as medical diagnoses, financial trades, or in the legal system — as regulatory authorities and users would likely want know how and why the decisions were made.
There are initiatives in development which try to address this issue. For instance, LIME (or the tongue twister version: local interpretable model-agnostic explanations) looks at certain sections of data individually and tries to see which inputs are relied on most by the model to make a conclusion. If we were talking about recognizing bikes, a LIME model might exclude the wheels, then the handles in the calculation process, to see which feature carries what weight in the recognition process.
Currently, AI systems often require colossal amounts of well-labeled training data to perform at or above human levels. Most children could learn to recognize bicycles with fairly reliable accuracy after being shown a few examples, but an AI may need hundreds of thousands, or even millions of annotated images before it can reach a similar level of recognition.
This creates some obvious logistics issues. It can be difficult to train a system for problems where data is limited; data on bikes may be plenty, but scarce when it comes to, say, treatment outcomes for a small clinical-trial. And even in cases where data does exist, it can be time consuming and costly to collect and label enough data to be fed into the algorithm.
Researchers are looking into different methods to reduce the need for heavily annotated datasets. One method is Generative Adversarial Networks (GANs), a semisupervised method, where two deep learning networks compete against one another to refine their results (where one network provides real and generated pictures of bikes, for example, and the other network tries to distinguish the real from the fake). Another avenue is reinforcement learning, an unsupervised method where algorithms improve through repetition, where they are left to explore on their own, and then ‘rewarded’ with high scores or ‘punished’ with low scores depending on the success of their performance.
Another, more prickly, limit arises from the potential biases in data which get fed into the systems. Because data collection methodologies can be inconsistent between different groups, and the data itself may contain human biases, the resulting AI could be ‘misinformed’ and therefore give out unrepresentative outputs.
Consider Microsoft’s infamous AI Twitterbot, Tay, which made replies based on interactions with Twitter users, and which had to be taken down after a mere 16 hours because it began posting racist and genocidal messages. Admittedly, that was a result of deliberate attempts by the public to sabotage Tay. Yet, this weakness of biased data could translate into grave consequences if human biases and prejudices are either intentionally or unconsciously injected into the output of a system — for instance in lending decisions or in criminal justice judgements.
This problem poses a particularly difficult challenge, because the issue jumps beyond pure technical complexity and into the meta-zone of social values. While ongoing research exists to identify best practices to filter out biases and adjust weightings to data points to better reflect reality, this problem is also compounded by the black box obstacle, which makes it harder to discern the internal processes of an AI.
So those are the some of the biggest limitations to the AI of today, and some of the most promising efforts to address them. But, promising as they are, it is important to note that they are largely very early attempts to address immensely difficult challenges, meaning there are still many hurdles ahead.
The first broad hurdle is how to accurately represent the world and goals to machines. For a system to solve something, it first needs to understand the context of the problem, and secondly its goal.
For simpler tasks like recognizing a bicycle, there are a manageable number of variables to be reproduced virtually, and the goal is simple — to recognize a bike. But beyond that, it is difficult to package our world into neatly defined inputs and clear goals for AIs to interpret. For example, taking a bike from A to B would mean: get to point B promptly, while adapting to potentially unpredictable terrain and weather, obeying all relevant laws, and appropriately determine the value of human life in dangerous situations — a far more nuanced and difficult task to represent in a machine’s ‘mind’.
The second challenge is deciphering human cognition. In cases where the goals and rules can be clearly distilled, like chess or long division, machines have already surpassed what we are capable of. But human cognition is still far superior in interpreting and solving poorly defined problems, such as making a cup of coffee in any given household.
The issue is, we don’t yet fully understand the mechanics of our own cognition. We can distinguish certain features used in cognition, such as perception, reasoning, and abstraction, but we haven’t a clear set of rules or frameworks that lead to intelligent problem solving. For intelligent machines to solve problems as we do, or better than we can, we first need to dissect how we go about solving problems, and apply those principles to an AI system.
With that said, the intention of this post is not to promote a pessimistic view of AI. Instead, we want to give a sober account of what AI can and cannot (yet) do. The limits to today’s AI are numerous, as are the corresponding initiatives that try to tackle those challenges; entirely new approaches and techniques could also emerge to navigate these present difficulties. Ultimately, though, it seems the biggest limitations to artificial intelligence have less to do with the systems themselves, but with our ability to convey to these systems the nuances of our world and mechanics of our own cognition.