Energy prediction using evolutionary lean neural networks

Foo, Yong Wee (2022) Energy prediction using evolutionary lean neural networks. PhD thesis, University of Glasgow.

Abstract

The demand for data center services, driven by the surge in online applications and services, has propelled energy consumption to unprecedented levels. While renewable energy provides an attractive and more environmentally friendly alternative to existing energy resources, renewable intermittency is a major issue for grid operators. Accurate energy predictions are thus paramount to maintaining optimal services and energy provisions amidst a shift towards greener energy for more sustainable data centers.

Artificial Neural Networks (ANN) are powerful learning machines adopted for several decades for prediction problems. Recent years have seen increased interest in ANN, led by advancements in AI and computing hardware. Despite the significant progress, ANNs are notoriously hard to train and extremely difficult to interpret as the relationships between the input variables and the output responses are often hard to tease apart. The structure of ANN can considerably impact its performance as it has a direct dependency on the model architecture and parameters. Achieving high performance accuracy and the ability to generalize across different problem sets remains a big challenge for ANNs. For example, an over-trained model becomes too large and complex, is more prone to overfitting, and cannot make accurate predictions as it does not generalize well to new data. Additionally, the more complex a network, the more difficult it is to explain the relationships learned by the model.

Traditionally, most research focus on model parameter learning, where gradient-based methods are frequently applied to optimize connection weights and biases. In contrast, model architecture learning is manually set based on experience or trial-and-error experimentation. However, this approach suffers several constraints, including limiting the search space of candidate solutions with a predefined number of neurons and connections.

To address these limitations, a novel ANN called the Evolutionary Lean Neural Network (EVLNN) is developed in this thesis. EVLNN uses an improved Genetic Algorithm (GA) to optimize ANN architecture and parameters, offering greater training flexibility than traditional approaches. The proposed approach has the advantage of simplifying energy prediction tasks by allowing one to specify parameters such as the minimum and maximum network size, the transfer functions, feedforward architecture, or architecture with feedback for time series forecasting. In this approach, structural optimality properties of the problem are formulated and solved with an implementation of an improved GA that includes species parallelism, intra-and-inter species crossover, and a two-stage mutation. EVLNN serves as a global search algorithm by locating a parsimonious ANN that can provide a more generalized solution. Sensitivity analysis mechanisms are designed into the algorithm to help with interpretability and understanding of the model.

In developing the EVLNN algorithm, a set of benchmark functions was used to empirically evaluate and compare the algorithm’s performance with other well-established algorithms - Particle Swarm Optimization (PSO), Differential Evolution (DE), and the standard Genetic Algorithm (GA). The results showed EVLNN’s ability to generalize well by locating the peaks in all the test functions, whereas the other algorithms have located the peaks in all but one test function.

The EVLNN algorithm was applied to two energy prediction problems in this thesis. The first application is in predicting the energy consumption of a Hadoop testbed. Using variables related to energy consumption from the Hadoop system, EVLNN accurately predicted its energy consumption and helped identify key energy influencing factors. It also performed more favorably than networks trained by PSO-NN, DE-NN, and GA-NN. The second application is in the forecasting of solar irradiance. EVLNN showed accurate forecasts in different settings of time resolutions (sample size) and using a different number of input variables. In most of those settings, EVLNN outperformed PSO-NN, DE-NN, GA-NN, and the fully-connected Time Delay Backpropagation neural network (TD-BPNN).

Accurate energy predictions underpin the essential improvements required in energy resource management for both data center owners and grid operators. Furthermore, the ability to explain and interpret the model behavior provides a basis for understanding the dynamics of energy consumption. This work has provided a simplified and flexible approach to ANN architecture design and parameter optimization to achieve interpretable models with high accuracy and good generalization properties for energy prediction problems. The findings demonstrated that EVLNN could create parsimonious models for accurate energy prediction, which are also capable of discovering the relationships between key determinants of energy consumption.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Colleges/Schools: College of Science and Engineering > School of Engineering
• Supervisor's Name: Goh, Dr. Cindy
• Date of Award: 2022
• Depositing User: Theses Team
• Unique ID: glathesis:2022-83020
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 07 Jul 2022 11:34
• Last Modified: 07 Jul 2022 11:35
• Thesis DOI: 10.5525/gla.thesis.83020
• URI: https://theses.gla.ac.uk/id/eprint/83020

Related URLs

• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI