What is Deep Learning?

Deep learning is a subfield of machine learning that employs artificial neural networks to process and analyze vast amounts of complex data. In architecture, deep learning can be used for a variety of applications.

For example, generative design algorithms that employ deep learning can explore design space and find optimal solutions based on specific performance criteria such as energy efficiency, structural integrity, and aesthetic preferences. Deep learning can also be used for image recognition tasks, such as automatically creating 3D building models from 2D floor plans or predicting the behavior of buildings in response to various environmental conditions such as wind, heat, and earthquakes. Additionally, deep learning can be used for material classification, object recognition, and generative design.

Contents

> Deep learning model architecture

We create an artificial neural net where we have nodes for input values (what we already know) and output values (our predictions), and in between those, we have a hidden layer (or layers) where the information travels before it hits the output. This is analogous to how the information you see through your eyes is filtered into your understanding rather than being shot straight into your brain.

Deep learning. (2022, July 9). In Wikipedia https://simple.wikipedia.org/wiki/Deep_learning .

Each neuron receives input, performs a dot product with weights, applies an activation function, and passes the result to the next layer. The final output is a prediction made by the network based on the input and the weights, which are adjusted during the training process to minimize the error between the predictions and the actual target outputs.

Activation function. In Wikipedia https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions .

The training process involves multiple forward and backward passes through the network, where the gradients of the error with respect to the weights are calculated and used to update the weights through optimization algorithms such as gradient descent.

> What is an Artificial neural network (ANN)?

Artificial Neural Networks (ANNs) are a class of machine learning algorithms loosely inspired by the structure and function of the human brain. ANNs can be used for various applications in architecture, such as classification, regression, and clustering problems. For example, ANNs can be used to predict building energy consumption based on various factors such as weather, occupancy, and building characteristics. They can also be used for material classification, object recognition, and generative design. ANNs can be trained using supervised, unsupervised, or reinforcement learning and applied to numerical and non-numerical data. ANNs are a powerful tool in architecture and continue to be an active research area.

The output of an Artificial Neural Network (ANN) is a value that represents the prediction of a variable. An essential part of an ANN is its weights, which are used as input multipliers that enter a neuron. Each neuron connected to another will have its own weight.

Further reading about MLP

> What is Convolutional Neural Network (CNN)?

A Convolutional Neural Network (CNN) is an artificial neural network designed for image and visual recognition tasks. It uses convolutional layers to process and analyze local patterns and correlations in the input data. It gradually builds a high-level understanding of the visual content through multiple layers. The main advantage of CNNs over other types of neural networks is their ability to reduce the number of parameters and computation required while maintaining high accuracy for image classification and recognition tasks.

Convolutional neural network. Wikipedia. https://en.wikipedia.org/wiki/Convolutional_neural_network .

Check out this great distill.pub article, which shows the checkerboard artifacts that occur when transposed convolutions and compares them against upsampling plus convolutions.

> What is GAN?

A GAN (Generative Adversarial Network) is a type of deep learning architecture that involves two neural networks: a generator and a discriminator. The generator network uses a random noise vector as input and generates synthetic data, while the discriminator network tries to differentiate between real and synthetic data. These two networks are trained against each other in an adversarial manner, with the generator trying to produce synthetic data indistinguishable from real data and the discriminator trying to correctly identify which data is real and which is synthetic. The competition between the two networks results in the generator learning to produce more realistic data over time.

GANs have been used for various applications, such as generating realistic images, creating music, and even generating text. They are a powerful tool in deep learning and continue to be an active research area.

A fascinating project using Generative Adversarial Networks (GANs) to create realistic house layouts can be found at the following URL. The project utilizes GANs to generate house layouts while considering graph-constrained relationships. Click on the image to learn more.

Generator

The generator network is an essential component of the Generative Adversarial Network (GAN) architecture. Its goal is to create new data that resembles the real data used to train the network. The generator network takes a random noise vector as input and generates synthetic data, which is then evaluated by the discriminator network to determine whether it is real or fake. The generator network is trained against the discriminator network in an adversarial manner to produce synthetic data that is indistinguishable from real data. Over time, the competition between the generator and discriminator networks leads to the generator network producing more realistic and diverse data.

Discriminator

The discriminator network is one of the two neural networks in a Generative Adversarial Network (GAN). Its purpose is to differentiate between real and synthetic data generated by the generator network. It is trained using a binary classification task, where it is presented with both real and synthetic data and learns to identify which is which. As the generator network improves at creating realistic data, the discriminator network is challenged to improve its ability to differentiate between real and synthetic data. This competition between the two networks eventually leads to the generator network creating more realistic and diverse data.

Overview of GAN Structure.