4 Types of Neural Network Architecture

A neural network is a tool for deep learning inspired by the biology of our human brains, allowing computers to make connections with data and learn to improve from experience over time. Neural network architecture is the structure of a neural network, a map of the neural layers and processes.

Use this article to learn about different types of neural network architectures, including feedforward neural networks, recurrent networks, convolutional networks, and generative adversarial networks.

What is a neural network?

A neural network is an artificial intelligence (AI) algorithm that allows computers to “think” similarly to our human way of thinking. While processing data, the artificial intelligence can make mistakes and then create improvements, calculating its errors and adjusting the weights of its nodes or neurons to compensate. Through this series of making errors and learning from them, neural networks are a powerful vehicle for machine learning and artificial intelligence.

Every neural network has a structure or architecture that starts with an input and ends with an output. In between, a number of hidden layers exist that the data passes through. Each layer contains nodes or neurons that react to the data in various ways, and each node has a weight that affects how it reacts to the input.

For example, an algorithm for computer vision might contain nodes that detect different properties or objects in images. The arrangement of these nodes and layers makes up the neural network's architecture. In the case of social media, if the AI’s goal was to moderate content on a social media page, it may have a series of nodes assigned to look for inappropriate images in order to manage and screen content.

Neural network architecture types

Neural network architecture refers to the structure of the neural network or the number and types of layers. Let's learn more about these four types of neural networks and their architectures: feedforward neural networks, recurrent neural networks, convolutional neural networks, and generative adversarial networks.

Feedforward neural networks 

A feedforward is one of the more basic forms of neural networks, and you can often use the architecture of a feedforward neural network to create more specialized networks. As the name suggests, feedforward neural networks feed data forward from input to output with no loops or circles. Although it’s one of the simplest structures for neural networks, the hidden layers between input and output can still be complex. You can use this type of neural network for various tasks, such as pattern and image recognition, regression analysis, and classification.

How does a feedforward neural network architecture work? 

A feedforward neural network has an input layer, followed by a series of hidden layers, and ends with an output layer. Data flows into the algorithm through the input and passes through the nodes in the first layer. The first layer of nodes computes the data based on the node’s weights and passes the calculation to the next layer of nodes. Each node in each layer connects to each node in the next layer, but the data can only flow towards the output. 

Recurrent neural network 

A recurrent neural network is a model used for sequential data or time series prediction. For example, a recurrent neural network can make stock market predictions by calculating what is likely to happen in the future based on what happened in the past. You can also use a recurrent neural network for tasks like translation, where the sequence of words changes based on the language, such as a noun before or after an adjective. 

How does a recurrent neural network architecture work? 

In addition to the architecture found in the feedforward neural network, a recurrent network uses loops to circle the data back through the hidden layers before returning an output. Sometimes, recurrent neural networks include specialized hidden layers called context layers, which provide feedback to the neural network and help it become more accurate. 

Convolutional neural networks

Convolutional neural networks are particularly skilled at recognizing patterns and images, which makes them important for AI technology like computer vision, among other uses. For example, the US Postal Services uses neural networks to recognize handwritten zip codes. Convolutional neural networks are different from other networks because of their architecture and because the CNN nodes have shared weights and bias values, unlike feedforward or recurrent neural networks. They have shared weight because each node does the same job in a different input area, such as detecting the edge of an image.

How does a convolutional neural network architecture work? 

In addition to input and output layers, convolutional neural networks contain two main types of hidden layers: convolutional and pooling. Convolutional layers filter the input, typically an image, to extract various features. This data then feeds into a pooling layer, simplifying the parameters but keeping important information. The process repeats many times, sometimes including other layers, such as a multilayer perceptron or a rectified linear unit for activation.

Generative adversarial networks

A generative adversarial network differs from the models above because it is actually two separate networks. Working as a team, these two algorithms generate new content based on training data.

One of these neural networks, the generator, creates a novel image or text based on training data. The second neural network, the discriminator, judges the generator’s work to determine whether it looks real or fake. These two models go back and forth until the discriminator can’t tell the difference between the real training data and the generator's fake work.

Generative adversarial networks can create 3D models from 2D images, generate images, or create training data sets for other neural networks that are similar but different from existing data sets.

How does a generative adversarial network architecture work? 

The basic architecture of a generative adversarial network is two distinct neural networks working in tandem to produce an output from the input. Within this category of neural networks are subtypes that have unique architectures, such as:

• Vanilla GAN: This is the basic version of a generative adversarial neural network that needs to be adapted for many specific real-world applications.

• CycleGAN: The cycle-consistent generative adversarial network, or CycleGAN, is useful for image-to-image translation, moving an image from one domain to another.

• DCGAN: Deep convolutional generative adversarial network, or DCGAN, leverages convolutional neural networks for more powerful image generation.

• Text-2-image: A text-2-image generative adversarial neural network can create novel images from text-based descriptions, such as adding specific eye color to a generated face.

What are neural networks used for?

Although we have been studying and implementing neural networks since at least the 1940s, advancements in deep learning have guided us to work with the algorithms in new and advanced ways. Today, researchers and scientists can use neural networks for real-world applications in various fields, including the automotive industry, finance, national defense, insurance, health care, and utilities.

• Automotive: Self-driving cars use neural networks to make decisions based on the data they receive from their surroundings. Neural networks can also optimize vehicle parts and functions or estimate how many vehicles you need to make to meet demand.

• Finance: Neural networks have many uses in the finance industry, from predicting the performance of the stock market or exchange rates between monetary denominations to determining credit scores and default risks.

• National defense: The Department of Defense uses neural networks to simulate situational training, such as combat readiness. Other neural network applications in national defense include the ability to develop unmanned aircraft.

• Insurance: Insurance providers can use neural networks to model how often customers file insurance claims and the size of those claims.

• Health care: In a health care setting, doctors, health care administrators, and researchers use neural networks to make informed decisions about patient care, organizational decisions, and developing new medications.

• Utilities: Utility companies can use neural networks to forecast energy demand. Other uses include stabilizing electrical voltage or modeling oil recovery from residential areas.

How to get started in neural network architecture

If you are interested in a career in neural network architecture, three potential careers to consider are test engineer, research scientist, or applied scientist. Let’s take a closer look at these career paths below. 

Test engineer

Average annual US salary (Glassdoor): $90,183 [1]

Job outlook (projected growth from 2022 to 2032): 5 percent [2]

Requirements: Test engineers typically require a bachelor’s degree, usually in mechanical, industrial, or software engineering.

As a test engineer, you can work with a product development team in the testing stage of developing a system, machine, or piece of software. You’ll likely work as a software testing engineer specializing in neural networks. Alternatively, you could specialize in building automated testing software using neural network technology. Other potential responsibilities in this role include configuring testing scenarios, providing feedback to designers, and looking for bugs in the software. 

Research scientist

Average annual US salary (Glassdoor): $116,021 [3]

Job outlook (projected growth from 2022 to 2032): 23 percent [4]

Requirements: Typically, a research scientist needs to earn a master’s degree, although it is possible to become a research scientist with a bachelor’s degree, and in some cases, you may need to earn your doctorate.

As a research scientist, you can design and conduct experiments to gain insight into problems or questions in your field. To become a research scientist focusing on neural networks, you can either work with neural networks to collect data or conduct research on neural networks themselves. Other potential responsibilities as a research scientist include reporting on the results of your work, presenting at conferences or to stakeholders, and applying for new or continued research funding.

Deep learning engineer

Average annual US salary (Glassdoor): $145,254 [5]

Job outlook (projected growth from 2022 to 2032): 23 percent [4]

Requirements: You'll likely need to earn a bachelor’s degree to become a deep learning engineer, typically in computer science or a related field.

As a deep learning engineer, you can create and operate neural networks and other forms of artificial intelligence. This role is closely related to a career in machine learning. You may work to improve artificial neural networks, build new networks to solve problems, perform analysis, or look for errors in the program.

Learn more with Coursera.

To take the next step and learn more about neural network architecture, consider earning a Deep Learning Specialization offered by DeepLearning.AI on Coursera. This five-course series takes approximately three months to complete and can help you learn more about artificial neural networks, recurrent neural networks, convolutional neural networks, TensorFlow, and more.