Transformer neural network is one of the most state- of-the-art designs in the field of deep learning. In the architecture of such deep neural networks, there are several hyperparameters which are highly effective in the performance of the model and tuning hyperparameters is a challenging problem. In this work, we aim to suggest a method to choose optimum hyperparameters automatically, namely Hybrid Grey Wolf Optimization (HGWO) algorithm. The proposed method is evaluated on the HAR dataset from the UCI Machine Learning repository and the result shows the ability of HGWO in determining the optimal values for the network hyperparameters.

Keywords