Mutual Information Maximization via Joint Power Allocation in Integrated Sensing and Communications System

In this paper, we focus on the power allocation of Integrated Sensing and Communication(ISAC) with orthogonal frequency division multiplexing(OFDM) waveform. In order to improve the spectrum utilization efficiency in ISAC, we propose a design scheme based on spectrum sharing, that is,to maximize the mutual information(MI) of radar sensing while ensuring certain communication rate and transmission power constraints. In the proposed scheme, three cases are considered for the scattering off the target due to the communication signals,as negligible signal, beneficial signal, and interference signal to radar sensing, respectively, thus requiring three power allocation schemes. However,the corresponding power allocation schemes are nonconvex and their closed-form solutions are unavailable as a consequence. Motivated by this, alternating optimization(AO), sequence convex programming(SCP) and Lagrange multiplier are individually combined for three suboptimal solutions corresponding with three power allocation schemes. By combining the three algorithms, we transform the non-convex problem which is difficult to deal with into a convex problem which is easy to solve and obtain the suboptimal solution of the corresponding optimization problem. Numerical results show that, compared with the allocation results of the existing algorithms, the proposed joint design algorithm significantly improves the radar performance.

Robust and Discriminative Feature Learning via Mutual Information Maximization for Object Detection in Aerial Images

Object detection in unmanned aerial vehicle(UAV)aerial images has become increasingly important in military and civil applications.General object detection models are not robust enough against interclass similarity and intraclass variability of small objects,and UAV-specific nuisances such as uncontrolledweather conditions.Unlike previous approaches focusing on high-level semantic information,we report the importance of underlying features to improve detection accuracy and robustness fromthe information-theoretic perspective.Specifically,we propose a robust and discriminative feature learning approach through mutual information maximization(RD-MIM),which can be integrated into numerous object detection methods for aerial images.Firstly,we present the rank sample mining method to reduce underlying feature differences between the natural image domain and the aerial image domain.Then,we design a momentum contrast learning strategy to make object features similar to the same category and dissimilar to different categories.Finally,we construct a transformer-based global attention mechanism to boost object location semantics by leveraging the high interrelation of different receptive fields.We conduct extensive experiments on the VisDrone and Unmanned Aerial Vehicle Benchmark Object Detection and Tracking(UAVDT)datasets to prove the effectiveness of the proposed method.The experimental results show that our approach brings considerable robustness gains to basic detectors and advanced detection methods,achieving relative growth rates of 51.0%and 39.4%in corruption robustness,respectively.Our code is available at https://github.com/cq100/RD-MIM(accessed on 2 August 2024).

Quantized Decoders that Maximize Mutual Information for Polar Codes

In this paper,we innovatively associate the mutual information with the frame error rate(FER)performance and propose novel quantized decoders for polar codes.Based on the optimal quantizer of binary-input discrete memoryless channels(BDMCs),the proposed decoders quantize the virtual subchannels of polar codes to maximize mutual information(MMI)between source bits and quantized symbols.The nested structure of polar codes ensures that the MMI quantization can be implemented stage by stage.Simulation results show that the proposed MMI decoders with 4 quantization bits outperform the existing nonuniform quantized decoders that minimize mean-squared error(MMSE)with 4 quantization bits,and yield even better performance than uniform MMI quantized decoders with 5 quantization bits.Furthermore,the proposed 5-bit quantized MMI decoders approach the floating-point decoders with negligible performance loss.

A method based on mutual information and gradient information for medical image registration

Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.

Fuzzy entropy design for non convex fuzzy set and application to mutual information

Fuzzy entropy was designed for non convex fuzzy membership function using well known Hamming distance measure.The proposed fuzzy entropy had the same structure as that of convex fuzzy membership case.Design procedure of fuzzy entropy was proposed by considering fuzzy membership through distance measure,and the obtained results contained more flexibility than the general fuzzy membership function.Furthermore,characteristic analyses for non convex function were also illustrated.Analyses on the mutual information were carried out through the proposed fuzzy entropy and similarity measure,which was also dual structure of fuzzy entropy.By the illustrative example,mutual information was discussed.

Feature selection based on mutual information and redundancy-synergy coefficient

Mutual information is an important information measure for feature subset. In this paper, a hashing mechanism is proposed to calculate the mutual information on the feature subset. Redundancy-synergy coefficient, a novel redundancy and synergy measure of features to express the class feature, is defined by mutual information. The information maximization rule was applied to derive the heuristic feature subset selection method based on mutual information and redundancy-synergy coefficient. Our experiment results showed the good performance of the new feature selection method.

ANALYSIS OF THE MUTUAL INFORMATION BETWEEN INPUT AND OUTPUT OF A CLASS OF CLOCK-CONTROLLED SEQUENCES

In this paper, the mutual information between clock-controlled input and output sequences is discussed. It is proved that the mutual information is a strictly monotone increasing function of the length of output sequence, and its divergent rate is gaven.

A novel SINR and mutual information based radar jamming technique

The improvements of anti-jamming performance of modem radar seeker are great threats to military targets. To protect the target from detection and estimation, the novel signal-to-interference-plus-noise ratio （SINR）-based and mutual information （Ml）-based jamming design techniques were proposed. To interfere with the target detection, the jamming was designed to minimize the S1NR of the radar seeker. To impair the estimation performance, the mutual information between the radar echo and the random target impulse response was used as the criterion. The spectral of optimal jamming under the two criteria were achieved with the power constraints. Simulation results show the effectiveness of the jamming techniques. SINR and MI of the SINR-based jamming, the MI-based jamming as well as the predefined jamming under the same power constraints were compared. Furthermore, the probability of detection and minimum mean-square error （MMSE） were also utilized to validate the jamming performance. Under the jamming power constraint of I W, the relative decrease of the probability of detection using S1NR-based optimal jamming is about 47%, and the relative increase of MMSE using Ml-based optimal jamming is about 8%. Besides, two useful jamming design principles are concluded which can be used in limited jamming power situations.

Mutual Information and Relative Entropy of Sequential Effect Algebras

In this paper,we introduce and investigate the mutual information and relative entropy on the sequentialeffect algebra,we also give a comparison of these mutual information and relative entropy with the classical ones by thevenn diagrams.Finally,a nice example shows that the entropies of sequential effect algebra depend extremely on theorder of its sequential product.

k-NN Based Bypass Entropy and Mutual Information Estimation for Incremental Remote-Sensing Image Compressibility Evaluation

Incremental image compression techniques using priori information are of significance to deal with the explosively increasing remote-sensing image data. However, the potential benefi ts of priori information are still to be evaluated quantitatively for effi cient compression scheme designing. In this paper, we present a k-nearest neighbor(k-NN) based bypass image entropy estimation scheme, together with the corresponding mutual information estimation method. Firstly, we apply the k-NN entropy estimation theory to split image blocks, describing block-wise intra-frame spatial correlation while avoiding the curse of dimensionality. Secondly, we propose the corresponding mutual information estimator based on feature-based image calibration and straight-forward correlation enhancement. The estimator is designed to evaluate the compression performance gain of using priori information. Numerical results on natural and remote-sensing images show that the proposed scheme obtains an estimation accuracy gain by 10% compared with conventional image entropy estimators. Furthermore, experimental results demonstrate both the effectiveness of the proposed mutual information evaluation scheme, and the quantitative incremental compressibility by using the priori remote-sensing frames.

Exploring the characteristics of acupoints in the treatment of stroke with complex network and point mutual information method

Objective:Explore the characteristics of acupoints in the treatment of stroke with complex network and point mutual information method.Methods:The complex network and point wise mutual information system-developed by Chinese academy of Chinese medical sciences wereused to analyze the specific acupoints,compatibility,frequency etc.Results:174 acumoxibustion prescriptions were collected,including 163 acupoints.among them eighteen acupoints were used more than 30 times such as Hegu(LI4),Zusanli(ST36),Quchi(LI11)and Fengshi(GB31).The combinations of 31 acupoints were used more than 15 times,such as the combination of Quchi(LI11)and Zusanli(ST36),the combination of acupoint Quchi(LI11)and Jianyu(LI15),Hegu and Quchi(LI11).The most commonly used treatment method for stroke treatment is to dredge the Yangming meridian and Shaoyang meridian through acupuncture the multiple acupoints located on these two meridians..The commonly used acupoints are mainly distributed in the limbs,head and face.The most commonly used specific acupoint is intersection acupoint.The usage frequency of specific acupoints are higher than that of non-specific acupoints.Conclusion:Dredging the collaterals,dispelling wind-evil and restoring consciousness are the main principle for the treatment of stroke.Specific acupoints in head,face and climbs maybe the main targeted acupoints.Combination of Yang meridians with other meridians is needed to improve the effects.The Yangming meridian and Shaoyang meridian are most used meridians and Hegu(LI4),Quchi(LI11)and Zusanli(ST36)are the most used acupionts.

Medical Image Registration Based on Phase Congruency and Regional Mutual Information

In this paper, a new approach of muhi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. Firstly, instead of standard mutual information （ MI ） based on joint intensity histogram, regional mutual information （ RMI ） is employed, which allows neighborhood information to be taken into account. Secondly, a new feature images obtained by means of phase congruency are invariants to brightness or contrast changes. By incorporating these features and intensity into RMI, we can combine the aspects of both structural and neighborhood information together, which offers a more robust and a high level of registration accuracy.

Use of Mutual Information Arrays to Predict Coevolving Sites in the Full Length HIV gp120 Protein for Subtypes B and C

It is well established that different sites within a protein evolve at different rates according to their role within the protein; identification of these correlated mutations can aid in tasks such as ab initio protein structure, structure function analysis or sequence alignment. Mutual Information is a standard measure for coevolution between two sites but its application is limited by signal to noise ratio. In this work we report a preliminary study to investigate whether larger sequence sets could circumvent this problem by calculating mutual information arrays for two sets of drug naive sequences from the HIV gpl20 protein for the B and C subtypes. Our results suggest that while the larger sequences sets can improve the signal to noise ratio, the gain is offset by the high mutation rate of the HIV virus which makes it more difficult to achieve consistent alignments. Nevertheless, we were able to predict a number of coevolving sites that were supported by previous experimental studies as well as a region close to the C terminal of the protein that was highly variable in the C subtype but highly conserved in the B subtype.

A bidirectional feature selection method based on mutual information and redundancy-synergy coefficient

Feature subset selection is a fundamental problem of data mining. The mutual information of feature subset is a measure for feature subset containing class feature information. A hashing mechanism is proposed to calculate the mutual information of feature subset. The feature relevancy is defined by mutual information. Redundancy-synergy coefficient, a novel redundancy and synergy measure for features to describe the class feature, is defined. In terms of information maximization rule, a bidirectional heuristic feature subset selection method based on mutual information and redundancy-synergy coefficient is presented. This study’s experiments show the good performance of the new method.

Modeling of unsupervised knowledge graph of events based on mutual information among neighbor domains and sparse representation

Text event mining,as an indispensable method of text mining processing,has attracted the extensive attention of researchers.A modeling method for knowledge graph of events based on mutual information among neighbor domains and sparse representation is proposed in this paper,i.e.UKGE-MS.Specifically,UKGE-MS can improve the existing text mining technology's ability of understanding and discovering high-dimensional unmarked information,and solves the problems of traditional unsupervised feature selection methods,which only focus on selecting features from a global perspective and ignoring the impact of local connection of samples.Firstly,considering the influence of local information of samples in feature correlation evaluation,a feature clustering algorithm based on average neighborhood mutual information is proposed,and the feature clusters with certain event correlation are obtained;Secondly,an unsupervised feature selection method based on the high-order correlation of multi-dimensional statistical data is designed by combining the dimension reduction advantage of local linear embedding algorithm and the feature selection ability of sparse representation,so as to enhance the generalization ability of the selected feature items.Finally,the events knowledge graph is constructed by means of sparse representation and l1 norm.Extensive experiments are carried out on five real datasets and synthetic datasets,and the UKGE-MS are compared with five corresponding algorithms.The experimental results show that UKGE-MS is better than the traditional method in event clustering and feature selection,and has some advantages over other methods in text event recognition and discovery.

New bounds on the mutual information for discrete constellations and application to wireless channel estimation

The lack of closed-form expressions of the mutual information for discrete constellations has limited its uses for analyzing reliable communication over wireless fading channels.In order to address this issue,this paper proposes analytically-tractable lower bounds on the mutual information based on Arithmetic-Mean-Geometric-Mean(AMGM)inequality.The new bounds can apply to a wide range of discrete constellations and reveal some insights into the rate behavior at moderate to high Signal-to-Noise Ratio(SNR)values.The usability of the bounds is further demonstrated to approximate the optimum pilot overhead in stationary fading channels.

Mutual Information-Based Modified Randomized Weights Neural Networks

Randomized weights neural networks have fast learning speed and good generalization performance with one single hidden layer structure. Input weighs of the hidden layer are produced randomly. By employing certain activation function, outputs of the hidden layer are calculated with some randomization. Output weights are computed using pseudo inverse. Mutual information can be used to measure mutual dependence of two variables quantitatively based on the probability theory. In this paper, these hidden layer’s outputs that relate to prediction variable closely are selected with the simple mutual information based feature selection method. These hidden nodes with high mutual information values are maintained as a new hidden layer. Thus, the size of the hidden layer is reduced. The new hidden layer’s output weights are learned with the pseudo inverse method. The proposed method is compared with the original randomized algorithms using concrete compressive strength benchmark dataset.

Study of three-dimensional PET and MR image registration based on higher-order mutual information

Mutual information has currently been one of the most intensivelyresearched measures. It has been proven to be accurate and effective registrationmeasure. Despite the general promising results, mutual information sometimes mightlead to misregistration because of neglecting spatial information and treating intensityvariations with undue sensitivity. In this paper, an extension of mutual informationframework was proposed in which higher-order spatial information regarding imagestructures was incorporated into the registration processing of PET and MR. Thesecond-order estimate of mutual information algorithm was applied to the registrationof seven patients. Evaluation from Vanderbilt University and our visual inspectionshowed that sub-voxel accuracy and robust results were achieved in all cases withsecond-order mutual information as the similarity measure and with Powell's multi-dimensional direction set method as optimization strategy.

Double Pruning Structure Design for Deep Stochastic Configuration Networks Based on Mutual Information and Relevance

Deep stochastic configuration networks(DSCNs)produce redundant hidden nodes and connections during training,which complicates their model structures.Aiming at the above problems,this paper proposes a double pruning structure design algorithm for DSCNs based on mutual information and relevance.During the training process,the mutual information algorithm is used to calculate and sort the importance scores of the nodes in each hidden layer in a layer-by-layer manner,the node pruning rate of each layer is set according to the depth of the DSCN at the current time,the nodes that contribute little to the model are deleted,and the network-related parameters are updated.When the model completes the configuration procedure,the correlation evaluation strategy is used to sort the global connection weights and delete insignificance connections;then,the network parameters are updated after pruning is completed.The experimental results show that the proposed structure design method can effectively compress the scale of a DSCN model and improve its modeling speed;the model accuracy loss is small,and fine-tuning for accuracy restoration is not needed.The obtained DSCN model has certain application value in the field of regression analysis.