The authors will examine prediction of temperature daily profile using various modifications of BPTT （backpropagation through time algorithm） done by stochastic update in the artificial RCNN （recurrent neural netwo...The authors will examine prediction of temperature daily profile using various modifications of BPTT （backpropagation through time algorithm） done by stochastic update in the artificial RCNN （recurrent neural networks）. The general introduction was provided by Salvetti and Wilamowski in 1994 in order to improve probability of convergence and speed of convergence. This update method has also one another quality, its implementation is simple for arbitrary network topology. In stochastic update scenario, constant number of weights/neurons is randomly selected and updated. This is in contrast to classical ordered update, where always all weights/neurons are updated. Stochastic update is suitable to replace classical ordered update without any penalty on implementation complexity and with good chance without penalty on quality of convergence. They have provided first experiments with stochastic modification on BP （backpropagation algorithm） used for artificial FFNN （feed-forward neural network） in detail described in the article ＂Stochastic Weight Update in the Backpropagation Algorithm on Feed-Forward Neural Networks＂ presented on the conference IJCNN （International Joint Conference of Neural Networks） 2010 in Barcelona. The BPTT on RCNN uses the history of previous steps stored inside of the NN that can be used for prediction. They will describe exact implementation on the RCNN, and present experiment results on temperature prediction with recurrent neural network topology. The dataset used for temperature prediction consists of the measured temperature from the year 2000 till the end of February 2011. Dataset is split into two groups： training dataset, which is provided to network in learning phase, and testing dataset, which is unknown part of dataset to NN and used to test the ability of NN to predict the temperature and the ability of NN to generalize the model hidden in the temperature profile. The results show promising properties of stochastic weight update with toy-task data, and the higher complexity of the temperature daily profile prediction.展开更多
文摘The authors will examine prediction of temperature daily profile using various modifications of BPTT （backpropagation through time algorithm） done by stochastic update in the artificial RCNN （recurrent neural networks）. The general introduction was provided by Salvetti and Wilamowski in 1994 in order to improve probability of convergence and speed of convergence. This update method has also one another quality, its implementation is simple for arbitrary network topology. In stochastic update scenario, constant number of weights/neurons is randomly selected and updated. This is in contrast to classical ordered update, where always all weights/neurons are updated. Stochastic update is suitable to replace classical ordered update without any penalty on implementation complexity and with good chance without penalty on quality of convergence. They have provided first experiments with stochastic modification on BP （backpropagation algorithm） used for artificial FFNN （feed-forward neural network） in detail described in the article ＂Stochastic Weight Update in the Backpropagation Algorithm on Feed-Forward Neural Networks＂ presented on the conference IJCNN （International Joint Conference of Neural Networks） 2010 in Barcelona. The BPTT on RCNN uses the history of previous steps stored inside of the NN that can be used for prediction. They will describe exact implementation on the RCNN, and present experiment results on temperature prediction with recurrent neural network topology. The dataset used for temperature prediction consists of the measured temperature from the year 2000 till the end of February 2011. Dataset is split into two groups： training dataset, which is provided to network in learning phase, and testing dataset, which is unknown part of dataset to NN and used to test the ability of NN to predict the temperature and the ability of NN to generalize the model hidden in the temperature profile. The results show promising properties of stochastic weight update with toy-task data, and the higher complexity of the temperature daily profile prediction.
