Taxonomy and Types of AI Algorithms

What is an AI Algorithm?

Before understanding what is an AI Algorithm, it is necessary to understand what is Artificial Intelligence. It is concerned with not just understanding but also building intelligent entities i.e. machines that can compute how to act effectively and safely in a wide variety of novel situations. Typically, the Turing test approach is used to build and evaluate these systems. Alan Turing, the great computer scientist, formulated a thought experiment, in which a computer passes the test if a human interrogator, after posing some questions, cannot tell whether the responses came from a person or from a computer.

Accordingly, AI algorithms need the following capabilities to pass the Turing test:

• Natural Language Processing
• Knowledge Representation
• Automated Reasoning
• Machine Learning
• Computer Vision
• Robotics

In compute science, an algorithm is a finite set of instructions to solve a problem or perform a computation. AI algorithm(s), are the set of algorithms to realise intelligent machines that can perform the aforementioned tasks.

How do AI Algorithms work?

To understand how AI algorithms work, it is important to understand the concept of an agent.  An agent can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.   

A software agent takes keystrokes, file contents, and network packets as sensory inputs and acts on the environment by displaying on the screen, writing files, and sending network packets.  

AI algorithms are specialized software agents called rational agents. A rational agent is a computer program that acts in the best possible way to achieve its goals based on what it knows and what it sees. To understand how a rational agent works, we need to consider four factors: 

• The performance measure that defines the criterion of success. This is how the agent evaluates how well it is doing and what it wants to achieve. 
• The agent’s prior knowledge of the environment. This is what the agent knows about the world before it starts acting, such as the rules of the game, business domain knowledge, or the laws of physics. 
• The actions that the agent can perform. This is what the agent can do to change the state of the world, such as moving, speaking, or buying. The agent’s percept sequence to date. The agent has observed this so far, such as the images, sounds, or data it receives from its sensors. 

 Based on these four factors, we can define a rational agent as follows: 

A rational agent is a computer program that, for each possible percept sequence, selects an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. 

This means that a rational agent always chooses the best action according to its current situation and its goal. It does not act randomly or irrationally. It learns from its experience and adapts to changing circumstances.  

Types of AI algorithms: Supervised Learning vs Unsupervised Learning

Nevertheless, let’s see how this applies to Machine Learning, which is one type of Artificial Intelligence algorithms. A machine learning model or a learning agent can be defined as follows: It learns from experience E about some class of tasks T, and performance measures P if its performance in tasks in T, as measured by P, gets better with experience E. With this in mind, let’s review two key types of ML/AI Algorithms:

• Supervised ML/AI
• Unsupervised ML/AI

What is supervised learning? 

An Supervised Learning Agent has the following characteristics:

Task T: Predicting, Classifying, Forecasting, etc.

Performance Measure P: Predictive Error, Accuracy, etc.

Experience E: Historical, Labeled Data in a Database.

Supervised Learning algorithms/agents are trained using labeled data i.e. every instance has the ground truth- a category, label. The algorithm learns to associate the input features with the target or label variable. Mathematically, it can be represented as:

y = f(x) + ε where:

• y is the output variable
• x is the input variable
• f () is the model function
• ε is the error term

Supervised Learning can further be classified as Regression and Classification. In regression, the target variable y is continuous, numerical in nature whereas, in classification it is categorical. An example of Regression is stock price prediction, where the target value i.e. stock prices, can take any value. On the other hand, an example of classification could be image classification i.e. to classify various images amongst cat, dog, etc.

What is unsupervised learning? 

An Unsupervised Learning Agent has the following characteristics:

Task T: Identify Patterns

Performance Measure P: Similarity, Separation, Compactness of clusters

Experience E: Historical, Unlabeled Data in a Database.

Note the key difference here. Unlike Supervised Learning, Unsupervised learning does not consist of labeled data. The algorithm is supposed to identify patterns in data and cluster them in groups on its own. Hence, performance evaluation of the algorithm is ambiguous and subjective. Moreover, it cannot be expressed in a close, compact mathematical form easily.

Examples of Unsupervised Learning include Clustering, Anomaly Detection, PCA, etc.

Types of AI Algorithms: Generative AI vs Discriminative AI

AI algorithms can also be categorized into discriminative AI and generative AI. 

• Discriminative AI algorithms give accurate answers for decision-making. They use ML algorithms like Logistic regression, Support Vector machines, Neural networks, etc. Some examples are: 

• • Predictive maintenance: AI algorithms can detect machine failures before they happen and help organizations take preventive measures. This saves money and time. 
  • Spam Filtering: AI algorithms can sort emails into spam or non-spam and protect users from harmful messages. 

• Generative AI algorithms create new content instead of giving exact answers. They use models like Generative Adversarial Networks, Variational Autoencoders, and Transformers like the GPT-n series. Some examples are: 

• • Content Generation: This is creating new text in different formats based on specific topics, rules, or limits. For instance, writing a script for a space adventure movie is a content-generation task. 
  • Content Editing: This is the improvement of existing text by checking and correcting its quality, meaning, and structure. It can involve fixing grammar, ensuring coherence, and rearranging for better flow. For instance, editing a rough draft of the Space Adventures script is a content editing task. 
  • Content Synthesis: This is the combination of information from various sources and contexts to create a clear and concise output. It requires both content generation and editing skills. It also aims to present the data in a new or original way. For instance, combining scripts from Ramayana and Mahabharata to create a new story for the millennial generation is a content synthesis task. 

Types of AI Algorithms: AI for Analytics vs AI for Automation

Another way of classifying AI is by business case. AI can be applied to perform analytics to augment human decision making, by uncovering insights from massive amounts of data, which may be virtually impossible for naked human eye. Take, for instance, descriptive analytics. It is hard to visualize high-dimensional data for humans. Algorithms like t-SNE/PCA can help with dimensionality reduction, thus enabling data visualization in lower dimensions. Besides, application of Machine Learning in predictive analytics is well known.

However, AI finds maximum use cases in automation, especially in Natural Language Processing and Computer Vision. This is especially true with the outburst of Generative AI.

Lastly, here is a detailed article of mine on AI for Analytics vs AI for Automation.

P.S.

AI Agent Image credit: Artificial Intelligence: A Modern Approach, 4th US ed.