A New Reward System Based on Human Demonstrations for Hard Exploration Games

The main idea of reinforcement learning is evaluating the chosen action depending on the current reward.According to this concept,many algorithms achieved proper performance on classic Atari 2600 games.The main challenge is when the reward is sparse or missing.Such environments are complex exploration environments likeMontezuma’s Revenge,Pitfall,and Private Eye games.Approaches built to deal with such challenges were very demanding.This work introduced a different reward system that enables the simple classical algorithm to learn fast and achieve high performance in hard exploration environments.Moreover,we added some simple enhancements to several hyperparameters,such as the number of actions and the sampling ratio that helped improve performance.We include the extra reward within the human demonstrations.After that,we used Prioritized Double Deep Q-Networks(Prioritized DDQN)to learning from these demonstrations.Our approach enabled the Prioritized DDQNwith a short learning time to finish the first level of Montezuma’s Revenge game and to perform well in both Pitfall and Private Eye.We used the same games to compare our results with several baselines,such as the Rainbow and Deep Q-learning from demonstrations(DQfD)algorithm.The results showed that the new rewards system enabled Prioritized DDQN to out-perform the baselines in the hard exploration games with short learning time.

A robust optimization method for label noisy datasets based on adaptive threshold: Adaptive-k

The use of all samples in the optimization process does not produce robust results in datasets with label noise.Because the gradients calculated according to the losses of the noisy samples cause the optimization process to go in the wrong direction.In this paper,we recommend using samples with loss less than a threshold determined during the optimization,instead of using all samples in the mini-batch.Our proposed method,Adaptive-k,aims to exclude label noise samples from the optimization process and make the process robust.On noisy datasets,we found that using a threshold-based approach,such as Adaptive-k,produces better results than using all samples or a fixed number of low-loss samples in the mini-batch.On the basis of our theoretical analysis and experimental results,we show that the Adaptive-k method is closest to the performance of the Oracle,in which noisy samples are entirely removed from the dataset.Adaptive-k is a simple but effective method.It does not require prior knowledge of the noise ratio of the dataset,does not require additional model training,and does not increase training time significantly.In the experiments,we also show that Adaptive-k is compatible with different optimizers such as SGD,SGDM,and Adam.The code for Adaptive-k is available at GitHub.

A comparative analysis of text representation, classification and clustering methods over real project proposals

Purpose-When a large number of project proposals are evaluated to alocate available funds,grouping them based on their simiarites is benefciaL.Current approaches to group proposals are primarily based on manual matching of similar topics,discipline areas and keywordls declared by project applicants.When the number of proposals increases,this task becomes complex and requires excessive time.This paper aims to demonstrate how to ffctively use the rich information in the titles and abstracts of Turkish project propsals to group them atmaially.Design/methodology/approach-This study proposes a model that effectively groups Turkish project proposals by combining word embedding,clustering and classification technigues.The proposed model uses FastText,BERT and term frequency/inverse document frequency(TF/IDF)word-embedding techniques to extract terms from the titles and abstracts of project proposals in Turkish.The extracted terms were grouped using both the clustering and classification techniques.Natural groups contained within the corpus were discovered using k-means,k-means++,k-medoids and agglomerative clustering algorithms,Additionally,this study employs classification approaches to predict the target class for each document in the corpus.To classify project proposals,var ious classifiers,including k nearest neighbors(KNN),support vector machines(SVM),artificial neural networks(ANN),cassftcation and regression trees(CART)and random forest(RF),are used.Empirical experiments were conducted to validate the effectiveness of the proposed method by using real data from the Istanbul Development Agency.Findings-The results show that the generated word embeddings an fftvely represent proposal texts as vectors,and can be used as inputs for dustering or casificatiomn algorithms.Using clustering algorithms,the document corpus is divided into five groups.In adition,the results demonstrate that the proposals can easily be categoried into predefmned categories using cassifiation algorithms.SVM-Linear achieved the highest prediction accuracy(89.2%)with the FastText word embedding.method.A comparison of mamual grouping with automatic casification and clutering results revealed that both classification and custering techniques have a high sucess rate.Research limitations/implications-The propsed mdelatomatically benefits fromthe rich information in project proposals and significantly reduces numerous time consuming tasks that managers must perform manually.Thus,it eliminates the drawbacks of the curent manual methods and yields significantly more acurate results.In the future,additional experiments should be conducted to validate the proposed method using data from other funding organizations.Originality/value-This study presents the application of word embedding methods to eftively use the rich information in the titles and abstracts of Turkish project proposals.Existing research studies focus on the automatice grouping of proposals;traditional frequency-based word embedding methods are used for feature extraction methods to represent project proposals.Unlike previous research,this study employs two outperforming neural network-based textual feature extraction techniques to obtain termns representing the proposals:BERT as a contextual word embedding method and F astText as a static word embedding method.Moreover,to the best of our knowledge,there has been no research conducted on the grouping of project proposals in Turkish.