ENGN4536/6536 - Wireless Communications

AI Mini Project: Deep Learning for PAPR Reduction in OFDM

Goal

In this project, you are expected to adopt a deep learning tool to reduce the peak-to-average power ratio (PAPR) in  an  emerging  wireless  technology,  orthogonal  frequency  division  multiplexing  (OFDM) .  This  helps  you  to understand and practice how to use artificial intelligence to solve wireless communication problems, aligning with the new trend in the telecommunications industry. You are expected to use the knowledge gained in this course and conduct offline research to complete this project. This is an individual project.

Assessment submission

•    You are required to submit two files: (i) One written Project Report in Wattle as a single PDF file and (ii) One ‘ .ipynb ’ file containing all the codes and the required outputs for completing Project Deliverables.

•    Please  include the coversheet in front of the written report where your name, university ID, and other information are entered.

Due time and date

•    Due Date: 11:55pm, Sunday, Week 11, 20 October 2024

•    Cut-off Date: N/A, i.e. late submission is not allowed

Mark and weight

•    Full mark: 100.

•    This assessment is worth 10% and 9% of the total mark for undergraduate students and postgraduate students, respectively.

Note

•    Sections 1.2, 1.4, and 2.1 provide information similar to the Deep Learning Project from ENGN3226/6626 (i.e. the pre-requisite course for ENGN4536/6536). It is expected that you have already familiarised with these preliminary materials.

•    In this document, these materials are kept for the sake of consistency and completeness. Nevertheless, you are recommended to review these materials again to recall the important knowledge of deep learning.

Section 1: Introduction

1.1 Project objective

The objective of this project is to use a deep learning (DL) tool to reduce peak-to-average power ratio (PAPR) in a wireless communication system where orthogonal frequency division multiplexing (OFDM) is used. Specifically, a DL model will be designed to achieve lower PAPR as compared to the classical OFDM scheme. After completing this project, you should be able to

•    Develop a deep understanding of the design and optimisation of OFDM-aided wireless communication systems.

•    Design an artificial neural network based autoencoder to solve a specific problem in wireless communication systems.

1.2 Motivation of using DL for designing communication systems

Designing a wireless communication system involves (i) modelling different types of channels and hardware characteristics and (ii) designing optimal signalling, transmission, reception, and detection methods. Generally, the designs of a communication system are based on models that are simple and can effectively approximate the major effects in the system. For example, these models are typically linear and stationary, with known statistics. These models allow mathematically tractable designs and performance evaluation, as we have seen in ENGN3226/6626 and ENGN4536/6536. The resultant designs are often reasonably good. However, if we want to achieve better performance by allowing more design flexibility and incorporating more detailed and complex system features, the conventional approach which relies on mathematically tractable designs is often not feasible anymore. To this end, recent research and development endeavours in the telecommunications industry have started to useDL-based design. In general, a DL-based system does not mandate mathematical tractability or linearity. Therein, the DL-based design can optimise the system for particular channel behaviour and hardware settings.

One often-mentioned drawback of the DL-based system or design is its computational complexity. Nevertheless, the DL computation can be highly parallelised on distributed architectures and thus are executed much efficiently. Moreover, specific parallel processing architectures are now available,i.e. graphical processing units (GPUs), which greatly reduce the processing time and energy cost for distributive algorithms. Due to these reasons, the DL-based communication systems and designs have picked up momentum in recent years. It is, therefore, important for communication engineers to have exposure to this relatively new approach.

1.3 Project scope

In this project, we will use the DL-based autoencoder model for constellation mapping and demapping ofsymbols on each orthogonal subcarrier to jointly minimise the PAPR at the transmitter and the bit error rate (BER) at the receiver in an OFDM-aided wireless communication system (referred to in this document as “the OFDM system”). Particularly, the  constellation mapper model will minimise the PAPR at the transmitter  and the  constellation demapper will minimise the BERat the receiver. Taking the data sequence in bits as per the given constellation order, the constellation mapper will generate encoded OFDM symbols to be transmitted over a fading channel and the constellation demapper will reconstruct the transmitted data sequence at the receiver. Lastly, we will do a statistical PAPR performance comparison between the classical OFDM system and the DL-based OFDM system with the autoencoder constellation mapper/demapper.

The scope of this project is limited to 2-dimentional QAM modulation scheme with Gray coding in a point-to- point communication system where one transmitter communicates with one receiver. We model the communication channel with multipath Rayleigh fading and additive white Gaussian noise (AWGN). Moreover, we create a model for a single OFDM symbol transmission over a given number of orthogonal subcarriers; therefore, we do not consider cyclic prefix to simplify the implementation, although the cyclic prefix can be added straightforwardly.

1.4 Implementation platform. Google Colab (Option 1)

The first option for your use is Colab, a web-based platform. from Google to implement the system in Python. In Colab, we will use Tensorflow with keras libraries. Please gotohttps://colab.research.google.com/and then sign in using your preferred email account (such as Gmail). The Google drive associated with a google account will be used as a directory to store, save, and load the user’s notebook (coding) files.

After signing in, please select File>New notebook from drop-down menu. You can rename the notebook by double clicking on the file name at the top. To save the changes made to the notebook, please select File>Save (or hit ‘Ctrl+S’). Please remember togo to your Google drive and check if you can find the newly created notebook file.

We write our code in the cell as indicated in the screenshot. You can add new cell by clicking on the “+ Code” button below drop-down menu or hit ‘Ctrl+M+B’ .

Let us run a simple code first, which can be seen from the screenshot below. Here, we are using 3 cells to write our code. In the first cell, we are importing the numpy library. In the second cell, we initialise two parameters (M and N). In the third cell, we use the numpy library to randomly generate N integers between 0 and M. We put the generated integers in an array named label and print this array. These 3 cells can be executed at once or individually from the Runtime tab as indicated in the screenshot or using hotkeys ‘Ctrl+F9’ and ‘Ctrl+Enter’, respectively.

1.5 Implementation platform. Jupyter Notebook in Anaconda (Option 2)

The second option for your use is Jupyter Notebook in Anaconda to implement the system in Python. First, please download Anaconda fromhttps://www.anaconda.com/downloadand install the software on your computer. You will see the interface of Anaconda shown by the left-hand-side picture below if it is installed successfully.

Second, please create a customised environment in Anaconda for the project by following the procedures in the  right-hand-side picture below. You may name the created environment as “AIMiniProject” or another name you want. We can choose Python 3.11.9 for the AIMiniProject environment.

Third, please  install  Jupter  Notebook  on  the  newly  created  AIMiniProject  environment  by  following  the procedures in the picture below. After that, you can launch Jupyter Notebook under the AIMiniProject environment and open the provided ‘ .ipynb’ file. By running the first cell of the provided code, an “Module Not Found” error will occur, telling you that the libraries to be imported are not exist, as shown in the right-hand-side picture below.

To fix this error, we should manually install the required libraries of “numpy”, “tensorflow”, “tensorflow-  probability” and “matplotlib” for the AIMiniProject environment. The installation procedures for “tensorflow” library are shown in the picture below. Please repeat the same procedure for installing all the required libraries.

After installing all the required libraries for the AIMiniProject environment, you can re-run the first cell of the provided code, and it is expected that the required libraries are imported successfully without reporting any errors. Although a warning message may appear as shown by the right-hand-side picture below, you can ignore this message and proceed with programming for the tasks specified in Section 4.

Section 2: Preliminaries

2.1 Basics of deep learning

A neural network (NN) is a general function approximator which learns the parameters of a given function that best predicts the output from the input data. Training is performed to identify the NN parameters which will minimise the prediction error. Typically, an NN consists of three main layers, including the input layer, hidden layer(s), and output layer. Each layer has one or multiple (potentially a large number of) neurons (as illustrated in the figure on the right- hand side, borrowed from Wikipedia). A neuron in one layer is connected to one or multiple neurons in its adjacent layers (indicated by the arrows in the figure on the right-hand side). Each neuron in a hidden layer applies a non-linear function on its input (i.e. the values carried by incoming arrows) and produces its output (which becomes the input to some neurons in the next layer as indicated by outgoing arrows).

The non-linear function performed by each neuron in the hidden layer is illustrated in Figure 1. Specifically, the particular neuron takes n input values,

applies a weight on each input value, adds the weighted values together, applies another additive bias value on the summation, and applies another non-linear function (i.e. the activation function) on the resultant value, in order to produce a single output. Let us denote this neuron with index j. Thus, the output hj  of a neuron j in a hidden layer is given by

ℎj  = σ 

where xi  is the ith input value, wij  is the ith weight, and bj  is the bias. The parameters to learn for the NN are the weights and biases of all the neurons. In addition, σ(·) is the activation function which is often anon-learning function, e.g. a chosen and fixed function.

Figure 1: A single neuron in the hidden layer taking inputs, processing the inputs, and producing an output.

In general, the activation function maps the input into a desired interval. Different types of activation functions are available,  such  as  sigmoid, hyperbolic tangent,  and  ReLU,  each performing  specific  data mapping.  Some activation functions have special roles or need to complete certain tasks, e.g. the softmax function is suitable for classification tasks. Lastly, a specific loss function is used during the training and evaluation processes to quantify prediction errors, e.g. mean squared error and categorical cross entropy. The choice of the loss function depends on the type of data.

In simple terms, the purpose of NN is to approximate a desired function we want, such that when we feed input values into the NN, it should produce the desired output values, i.e. it behaves the same way as the desired function. The process to ensure theNN behaving as what we want is through training, which will modify the values of theNN parameters (including weights and biases) in an iterative manner.

The training process of NN involves two operations, namely, the forward pass and backward pass. The forward pass involves producing an output for a given input, i.e. making prediction. The backward pass involves tuning the NN parameters as per the correctness of prediction. This is conducted by using a loss function and anoptimiser. The loss function is used to quantify the prediction error and the optimiser is used to control the manner in which the gradient of the loss function with respect to the NN parameters is calculated. The backward pass iterates the parameters from the output of the NN back to its input. Since the partial derivative with respect to each parameter is dependent on later layers in the NN, the error in the final prediction is propagated backward through the NN, which is called backpropagation. Different optimisers employ different methods to update the NN parameters which typically depend on the learning rate, the change observed in the gradient, and the batch size of input samples. Some popular optimisers are gradient descent, stochastic gradient descent (SGD), Adagrad, and Adam. For example, the most popular optimiser, SGD, updates its parameters iteratively according to

θi+1 = θi - η ▽L (θi),

where θ =  {w, b} is the set of NN parameters, η is the learning rate, L(·) is the average loss function over the considered batch size, and ▽ represents the gradient operation. The output of a hidden layer can be regularised to improve network generality and avoid overfitting, by scaling the outputs of the hidden layer. We can use this technique to enforce the energy constraints for our specific problem in this project.

Simply speaking, the training process iteratively checks how good the current NN (with the current parameter values) is by using the loss function and then updates the NN parameter values based on the optimiser. In each iteration, the optimiser finds out how to change the values of the NN parameters in order for the NN to get closer to the desired behaviour and then applies the change.

2.2 Autoencoder

The autoencoder is a type of the unsupervised learning method,i.e. learning without labelled data. It is similar to the feedforward NN structure we discussed earlier. However, they differ in that the feedforward technique attempts to predict a desired output value for a given input value, while on the contrary, the autoencoder attempts to reconstruct the original input at the output layer with the minimal differences between input and output. Specifically, we will use denoising autoencoder (DAE) in this project to reconstruct the transmitted data from the received noisy data. The DAE has a symmetrical network structure such that the input dimension is equal to the output dimension. Between the input and the output, an autoencoder has three parts: Encoder, code, and decoder. Because these terminologies can be confused with similar terms used in communication systems with different meanings, we will rename the three parts as: Constellation mapper, channel, and constellation demapper, as shown in Figure 2, for this project only. The functions of these three parts are as follows:

•    The constellation mapper maps the input data sequence to some specific data representation.

•    The channel model introduces some modification to the data representation to the transmitted data.

•    The  constellation demapper maps the data representation from the channel model to an estimate of the original input data sequence.

Figure 2: An OFDM system with autoencoder constellation mapper and demapper.

In this project, we use the autoencoder to introduce two additional blocks into an OFDM  system. At the transmitter, a constellation mapper, which takes a data sequence of symbols modulated using M-QAM, outputs a data representation for the IFFT block which generates OFDM symbols to be transmitted over the wireless channel using orthogonal subcarriers. The multipath Rayleigh fading channel with AWGN introduces corruption to the transmitted OFDM symbols. At the receiver, the output from the FFT block is passed to the constellation demapper block which generates an estimate of the original modulated data sequence.

In this project, the designed autoencoder model has two loss functions to minimise. At the transmitter, the constellation mapper’s  loss  function  tries to minimise  the  PAPR  of the  OFDM  symbol.  At  the  receiver,  the constellation demapper tries to minimise the BER. Thus, we introduce loss weights associate with each loss function to assist the training process in the direction we want our model to put more efforts on. For example, giving lesser weight to one loss function (e.g. BER reduction) will lead to better performance for the other minimisation problem (e.g. PAPR reduction) and viceversa. Nevertheless, both loss functions are jointly minimised in the training phase.

Section 3: Project Description and Deliverables

In this project, you are required to design and implement a DL-based autoencoder composed of a constellation mapper, a channel, and a constellation demapper for an OFDM system. The design shall be trained to transmit an OFDM symbol with a lower PAPR over a multipath Rayleigh fading channel with AWGN and reconstruct the transmit data sequence at the receiver with the optimal accuracy. Moreover, you are required to develop a framework to evaluate the PAPR for the classical OFDM system and compare the PAPR performance with the DL-based OFDM system. This comparison is performed through statistical analysis. In this analysis, first, a large number of OFDM symbols are generated at the transmitter and the PAPR value for each such symbol is recorded. Second, these symbols are transmitted over the fading channel and recovered at the receiver. Third, the BER performance is recorded. The overall design will mainly address the modulation order, the number of orthogonal subcarriers, and the loss weight for the PAPR loss function. As an outcome, the design produces the statistical PAPR performance and decoding accuracy. The project implementation must be based on the system model given in the following subsection.

3.1 System model

Let us consider a communication system based on OFDM, where a transmitter applies M-ary modulation on the bit sequence to be transmitted over N orthogonal subcarriers. Let M = 2k; thus, for each k-bit message, and the bit sequence to be transmitted is of length N×k, i.e. one symbol per subcarrier, is sent over a Rayleigh fading channel, and the receiver decides on the most likely k-bit message based on the received symbol. The procedure of the autoencoder implementation is as follows:

•    Based on the modulation order and number of subcarriers, a bit sequence is generated.

•    The bit sequence is given as the input to the constellation mapper model which outputs an encoded data sequence.

•    The IFFT operation is performed over the encoded bit sequence to generate an OFDM symbol of length N, which is sent using N orthogonal subcarriers.

•    The transmitted OFDM symbol is corrupted by the channel due to multipath Rayleigh fading and AWGN.

•    Equalisation is performed at the receiver and the equalised OFDM symbol undergoes the FFT operation.

•    The corrupted bit  sequence is given as the input to the constellation demapper model which outputs an estimate of the encoded data sequence.

The Python implementation for this project can be very sequential. This means that you do not need to use much programming logic except for loops. A significant part of the coding process is the neural network (NN) model architecture design, e.g. adding different types of layers using predefined single line instructions and their syntax is generic. That is, once you can define an input layer, hidden layer, and an output layer for an NN model, you can easily expand this model to design much more complicated architectures.

3.2 Project deliverables

You are required to complete each of the following tasks to fulfil all requirements of the project.  When completing the project, you are required to submit a project report by answering the question(s) under each task. The report will be marked based on the following three pieces of information for each task.

1)   A commentary on source codes:

    Please use the commentary to explain how the source codes work. Specifically, you are required to answer the questions specified in the commentary frame. (such as the conceptual understanding and programming logic,the objective achieved by a piece of code, and the functional structure of a program) in a direct and concise manner.

2)   The source codes which implement the task:

    You are required to provide the source codes which consist of the set of instructions to be executed, in order to achieve the objective of the task.

3)   The output of the source code:

    You are required to provide the output of source codes, except for Task 3.

    Important: The output of the source code should be displayed in your final submitted ‘ .ipynb’ file.

Please note that the tasks below are given in sequential manner, i.e. the source code of a given task can only work if all prior tasks have been executed. Thus, it is recommended to write the code for each task in a separate Colab cell and executed individually. In this way, you can troubleshoot with ease and keep track of progress.

In Wattle, a ‘ .ipynb’ file is provided which includes a partial code required to complete this project. This code specifies the important libraries to import, a modulator class to perform. QAM modulation for classical OFDM, etc. Most importantly, the provided ‘ .ipynb’ file indicates the missing code which you need to write in order to complete the project task. You are recommended to not change the values provided for the SNR, modulation order, and number of subcarriers in the ‘.ipynb’ file, for achieving uniformity across different student reports. As mentioned earlier, you should ensure that all the required outputs of the source code can be found in your submitted ‘.ipynb’ file.

Section 4: Instructions on Task Requirements

Task 1: Define system parameters and generate dataset (5 marks: 2.5 for commentary + 2.5 for source code and output)

•    Run the provided modulator class defined in a separate cell. It will be used to execute modulation operation for the classical OFDM design later. You do not need to provide any comments on this modulator class.

•    Define the system parameters that maybe sourced by the user. Please consider the modulation order M = 4, SNR = 25 dB, and the number of subcarriers N = 64. The rest of the hyper-parameters can be changed. Specifically, try different values for the weight of the loss function for the PAPR minimisation model, i.e. weight_papr, and observe its impact on bit sequence prediction accuracy.

•    Generate the dataset. Each data sample is a bit sequence of length N×k.

•    In the commentary, write the reason why each system parameter is required for the simulation.

•    Print the shape of the training dataset and include it in the output box.

•    Print the training dataset in a concise manner and include it in the output box.

Task 2: Define a constellation mapper model (15 marks: 7.5 for commentary + 7.5 for source code and output)

•    Let us define a constellation mapper model according to the following procedure:

o Taking abit sequence at the input layer.

o Followed by a set of hidden layers composed of a Dense layer with 2048 neurons, Batch Normalization, Activation layer. Use ‘relu’ as the activation function.

o Followed by multiple (e.g. 3) such set of layers to make the model more powerful.

o This is followed by a Dense layer to generate the encoded bit sequence. Please use ‘tanh’ as the activation function. Please state what is the shape of this layer and why the ‘tanh’ activation function is appropriate for this layer?

o Next, we take the IFFT of the encoded bit  sequence to generate the encoded OFDM symbol to be transmitted on the channel.

•    Please create a constellation mapper model to visualise the model and print it using the following statements. Here, please note that we will not train this model separately.

Task 3: Channel modelling and equalisation (15 marks: 7.5 for commentary + 7.5 for source code)

•    Let us pass the encoded OFDM symbol to a Lambda layer and perform. three core operations as follows:

o First, generate noise as per the given SNR value and generate fading gains.

    Compute the average symbol energy (or average symbol power).

    Compute the average noise energy (or average noise power).

    Compute the real component and imaginary component of noise.

    Generate the fading gain.

o Second, directly use the noise and fading gains to generate the equalised received signal.

    Generate the corruption component using the real and imaginary components of noise and fading gain.

    Generate the equalised signal using the corruption component.

o Third, apply FFT on the equalised received signal to generate the corrupted bit sequence.

Task 4: Define a constellation demapper model (10 marks: 5 for commentary + 5 for source code and output).

•    Next let us define a constellation demapper model by performing the following tasks.

o Take a corrupted bit sequence.

o Pass this bit sequence to a set of hidden layers composed of a Dense layer with 2048 neurons, Batch Normalization, ‘relu’ Activation layer.

o Followed by multiple (e.g. 3) such set of layers to make the model more powerful, as what we did before.

o This is followed by a Dense layer to generate the prediction for the original bit sequence. Please use ‘sigmoid’ as the activation function.

•    Please create a constellation demapper model to visualise the model and print using the following statements. Here, please note that we will not train this model separately.

Task 5: Define and compile the autoencoder model. Test trained autoencoder model’s performance (30

marks: 15 for commentary + 15 for source code and output).

•    Please define the autoencoder model by performing the following task:

o Put all the layers defined earlier together and create the autoencoder model with appropriate input and output.

•    Please compile the autoencoder model by performing the following tasks:

o Define and select Adam optimiser for given learning rate.

o Select appropriate loss functions,  e.g. use the provided  symbol_papr  function for the constellation mapper  and  the  binary   cross  entropy  function  for  the  constellation  demapper.  Please  provide justifications for the selected loss functions.

o Set loss weights to indicate dominance of one loss function over the other one.

o Print the accuracy of successfully decoding transmitted bit sequence during training.

o Compile the model to create it. Here, please note that we will use this model for training and testing.

o Print the autoencoder model summary.

•    Generate test dataset containing ‘test_dataset_size’ samples. Print test_dataset and its shape.

•    Pass  the  dataset to the trained autoencoder model to produce the optimal  symbol constellations. Print autoencoder output encoded symbols and its shape.

•    Compute the average PAPR across all these OFDM symbols produced by the autoencoder model, which will produce ‘test_dataset_size’ number of PAPR values. Convert these values to the dB scale. Write the steps needed to perform. this task. Print these PAPR values and shape.

Task  6:  Compute  classical  OFDM  PAPR  performance.  Plot  autoencoder  vs  classical  OFDM  PAPR

performance (20 marks: 10 for commentary + 10 for source code and output).

•    Please use the provided Modulation function which adopts modulator class that takes test_dataset and outputs a modulated dataset containing OFDM symbols, each having 64 complex modulated symbols. Print the modulated dataset and its shape.

•    Please create a function which takes IFFT of these modulated  symbols and calculates the PAPR for the classical OFDM system. Convert these PAPR values to the dB scale. Write the steps needed to perform. this task. Print the OFDM PAPR values and its shape.

•    Please plot the probability that the average PAPR of a single OFDM symbol for an OFDM scheme (e.g. the autoencoder-based OFDM scheme and classical OFDM scheme) is greater than a PAPR threshold. In order to do this, please follow the steps below:

o Compute a histogram for the PAPR values of the autoencoder-based OFDM scheme from Task 5. This reveals the histogram values and bin edge values. Then repeat this computation for the classical OFDM scheme.

o Use the output bin edge values and the corresponding histogram values to compute the probability that the given bin edge value is greater than the average OFDM symbol PAPR value for an OFDM scheme. Perform. this computation for both OFDM schemes and record the values.

o Plot the probability values on the log-scaled y-axis versus the PAPR values, i.e. bin edge values, on the x-axis for both OFDM schemes.