Multi-task Learning of Semantic Segmentation and Height Estimation for Multi-modal Remote Sensing Images

Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.

Height estimation from single aerial imagery using contrastive learning based multi-scale refinement network

Height map estimation from a single aerial image plays a crucial role in localization,mapping,and 3D object detection.Deep convolutional neural networks have been used to predict height information from single-view remote sensing images,but these methods rely on large volumes of training data and often overlook geometric features present in orthographic images.To address these issues,this study proposes a gradient-based self-supervised learning network with momentum contrastive loss to extract geometric information from non-labeled images in the pretraining stage.Additionally,novel local implicit constraint layers are used at multiple decoding stages in the proposed supervised network to refine high-resolution features in height estimation.The structural-aware loss is also applied to improve the robustness of the network to positional shift and minor structural changes along the boundary area.Experimental evaluation on the ISPRS benchmark datasets shows that the proposed method outperforms other baseline networks,with minimum MAE and RMSE of 0.116 and 0.289 for the Vaihingen dataset and 0.077 and 0.481 for the Potsdam dataset,respectively.The proposed method also shows around threefold data efficiency improvements on the Potsdam dataset and domain generalization on the Enschede datasets.These results demonstrate the effectiveness of the proposed method in height map estimation from single-view remote sensing images.

Reconstructing the size of individual trees using log data from cut-to-length harvesters in Pinus radiata plantations: a case study in NSW, Australia

With their widespread utilization, cut-to-length harvesters have become a major source of ‘‘big data’’ for forest management as they constantly capture, and provide a daily flow of, information on log production and assortment over large operational areas. Harvester data afford the calculation of the total log length between the stump and the last cut but not the total height of trees. They also contain the length and end diameters of individual logs but not always the diameter at breast height overbark（DBHOB） of harvested stems largely because of time lapse, operating and processing issues and other system deficiencies. Even when DBHOB is extracted from harvester data, errors and/or bias of the machine measurements due to the variation in the stump height of harvested stems from that specified for the harvester head prior to harvesting and diameter measurement errors may need to be corrected. This study developed（1） a system of equations for estimating DBHOB of trees from diameter overbark（DOB） measured by a harvester head at any height up to 3 m above ground level and（2） an equation to predict the total height of harvested stems in P. radiata plantations from harvester data. To generate the data required for this purpose, cut-to-length simulations of more than 3000 trees with detailed taper measurements were carried out in the computer using the cutting patterns extracted from the harvester data and stump height survey data from clearfall operations. The equation predicted total tree height from DBHOB, total log length and the small end diameter of the top log. Prediction accuracy for total tree height was evaluated both globally over the entire data space and locally within partitioned subspaces through benchmarking statistics. These statistics were better than that of the conventional height-diameter equations for P. radiata found in the literature, even when they incorporated stand age and the average height and diameter of dominant trees in the stand as predictors. So this equation when used with harvester data would outperform the conventional equations in tree height prediction. Tree and stand reconstructions of the harvested forest is the necessary first step to provide the essential link of harvester data to conventional inventory, remote sensing imagery and Li DAR data. The equations developed in this study will provide such a linkage for the most effective combined use of harvester data in predicting the attributes of individual trees, stands and forests, and product recovery for the management and planning of P. radiata plantations in New South Wales, Australia.

Target height and multipath attenuation joint estimation with complex scenarios for very high frequency radar

Low-angle estimation for very high frequency(VHF)radar is a difficult problem due to the multipath effect in the radar field,especially in complex scenarios where the reflection condition is unknown.To deal with this problem,we propose an algorithm of target height and multipath attenuation joint estimation.The amplitude of the surface reflection coefficient is estimated by the characteristic of the data itself,and it is assumed that there is no reflected signal when the amplitude is very small.The phase of the surface reflection coefficient and the phase difference between the direct and reflected signals are searched as the same part,and this represents the multipath phase attenuation.The Cramer-Rao bound of the proposed algorithm is also derived.Finally,computer simulations and real data processing results show that the proposed algorithm has good estimation performance under complex scenarios and works well with only one snapshot.

On the potential to predetermine dominant tree species based on sparse-density airborne laser scanning data for improving subsequent predictions of species-specific timber volumes

Background： Tree species recognition is the main bottleneck in remote sensing based inventories aiming to produce an input for species-specific growth and yield models. We hypothesized that a stratification of the target data according to the dominant species could improve the subsequent predictions of species-specific attributes in particular in study areas strongly dominated by certain species. Methods： We tested this hypothesis and an operational potential to improve the predictions of timber volumes, stratified to Scots pine, Norway spruce and deciduous trees, in a conifer forest dominated by the pine species. We derived predictor features from airborne laser scanning （ALS） data and used Most Similar Neighbor （MSN） and Seemingly Unrelated Regression （SUR） as examples of non-parametric and parametric prediction methods, respectively Results： The relationships between the ALS features and the volumes of the aforementioned species were considerably different depending on the dominant species. Incorporating the observed dominant species inthe predictions improved the root mean squared errors by 13.3-16.4 % and 12.6-28.9 % based on MSN and SUR, respectively, depending on the species. Predicting the dominant species based on a linear discriminant analysis had an overall accuracy of only 76 % at best, which degraded the accuracies of the predicted volumes. Consequently, the predictions that did not consider the dominant species were more accurate than those refined with the predicted species. The MSN method gave slightly better results than models fitted with SUR. Conclusions： According to our results, incorporating information on the dominant species has a clear potential to improve the subsequent predictions of species-specific forest attributes. Determining the dominant species based solely on ALS data is deemed challenging, but important in particular in areas where the species composition is otherwise seemingly homogeneous except being dominated by certain species.