Neural networks and decision tree algorithms are essential tools in machine learning and data science. They deal with patterns among inputs and provide predictions for targets. In this article, we use a hybrid approach in regression trees by incorporating possible dependencies among inputs and apply neural networks in terminal nodes. The proposed approach implements neural networks on the basis of dependency structures among inputs. We allow that the weights in training neural networks differ in various terminal nodes. In both regression and classification problems, the performance of the new approach is assessed by analyzing various real datasets and by conducting a Monte--Carlo simulation study. We show that the proposed approach provides more flexibility for neural networks when associations among inputs are observed.

Keywords