Application of Maximum Entropy Principle to Studying the Distribution of Wave Heights in A Random Wave Field

Based on the maximum entropy principle, a probability density function (PDF) is derived for the distribution of wave heights in a random wave field, without any more hypothesis. The present PDF, being a non-Rayleigh form, involves two parameters: the average wave height H— and the state parameter γ. The role of γ in the distribution of wave heights is examined. It is found that γ may be a certain measure of sea state. A least square method for determining γ from measured data is proposed. In virtue of the method, the values of γ are determined for three sea states from the data measured in the East China Sea. The present PDF is compared with the well known Rayleigh PDF of wave height and it is shown that it much better fits the data than the Rayleigh PDF. It is expected that the present PDF would fit some other wave variables, since its derivation is not restricted only to the wave height.

Subjectivity in Application of the Principle of Maximum Entropy

Complete prior statistical information is currently required in the majority of statistical evaluations of complex models. The principle of maximum entropy is often utilized in this context to fill in the missing pieces of available information and is normally claimed to be fair and objective. A rarely discussed aspect is that it relies upon testable information, which is never known but estimated, i.e. results from processing of raw data. The subjective choice of this processing strongly affects the result. Less conventional posterior completion of information is equally accurate but is computationally superior to prior, as much less information enters the analysis. Our recently proposed methods of lean deterministic sampling are examples of very few approaches that actively promote the use of minimal incomplete prior information. The inherited subjective character of maximum entropy distributions and the often critical implications of prior and posterior completion of information are here discussed and illustrated, from a novel perspective of consistency, rationality, computational efficiency and realism.

Surface Elevation Distribution of Sea Waves Based on the Maximum Entropy Principle

A probability density function of surface elevation is obtained through improvement of the method introduced by Cieslikiewicz who employed the maximum entropy principle to investigate the surface elevation distribution. The density function can be easily extended to higher order according to demand and is non-negative everywhere, satisfying the basic behavior of the probability, Moreover because the distribution is derived without any assumption about sea waves, it is found from comparison with several accepted distributions that the new form of distribution can be applied in a wider range of wave conditions, In addition, the density function can be used to fit some observed distributions of surface vertical acceleration although something remains unsolved.

Extraction of Information from Crowdsourcing: Experimental Test Employing Bayesian, Maximum Likelihood, and Maximum Entropy Methods

A crowdsourcing experiment in which viewers (the “crowd”) of a British Broadcasting Corporation (BBC) television show submitted estimates of the number of coins in a tumbler was shown in an antecedent paper (Part 1) to follow a log-normal distribution ∧(m,s2). The coin-estimation experiment is an archetype of a broad class of image analysis and object counting problems suitable for solution by crowdsourcing. The objective of the current paper (Part 2) is to determine the location and scale parameters (m,s) of ∧(m,s2) by both Bayesian and maximum likelihood (ML) methods and to compare the results. One outcome of the analysis is the resolution, by means of Jeffreys’ rule, of questions regarding the appropriate Bayesian prior. It is shown that Bayesian and ML analyses lead to the same expression for the location parameter, but different expressions for the scale parameter, which become identical in the limit of an infinite sample size. A second outcome of the analysis concerns use of the sample mean as the measure of information of the crowd in applications where the distribution of responses is not sought or known. In the coin-estimation experiment, the sample mean was found to differ widely from the mean number of coins calculated from ∧(m,s2). This discordance raises critical questions concerning whether, and under what conditions, the sample mean provides a reliable measure of the information of the crowd. This paper resolves that problem by use of the principle of maximum entropy (PME). The PME yields a set of equations for finding the most probable distribution consistent with given prior information and only that information. If there is no solution to the PME equations for a specified sample mean and sample variance, then the sample mean is an unreliable statistic, since no measure can be assigned to its uncertainty. Parts 1 and 2 together demonstrate that the information content of crowdsourcing resides in the distribution of responses (very often log-normal in form), which can be obtained empirically or by appropriate modeling.

Tree Network Formation in Poisson Equation Models and the Implications for the Maximum Entropy Production Principle

This paper presents not only practical but also instructive mathematical models to simulate tree network formation using the Poisson equation and the Finite Difference Method (FDM). Then, the implications for entropic theories are discussed from the viewpoint of Maximum Entropy Production (MEP). According to the MEP principle, open systems existing in the state far from equilibrium are stabilized when entropy production is maximized, creating dissipative structures with low entropy such as the tree-shaped network. We prepare two simulation models: one is the Poisson equation model that simulates the state far from equilibrium, and the other is the Laplace equation model that simulates the isolated state or the state near thermodynamic equilibrium. The output of these equations is considered to be positively correlated to entropy production of the system. Setting the Poisson equation model so that entropy production is maximized, tree network formation is advanced. We suppose that this is due to the invocation of the MEP principle, that is, entropy of the system is lowered by emitting maximal entropy out of the system. On the other hand, tree network formation is not observed in the Laplace equation model. Our simulation results will offer the persuasive evidence that certifies the effect of the MEP principle.

A Maximum-Entropy Compound Distribution Model for Extreme Wave Heights of Typhoon-Affected Sea Areas

A new compound distribution model for extreme wave heights of typhoon-affected sea areas is proposed on the basis of the maximum-entropy principle. The new model is formed by nesting a discrete distribution in a continuous one, having eight parameters which can be determined in terms of observed data of typhoon occurrence-frequency and extreme wave heights by numerically solving two sets of equations derived in this paper. The model is examined by using it to predict the N-year return-period wave height at two hydrology stations in the Yellow Sea, and the predicted results are compared with those predicted by use of some other compound distribution models. Examinations and comparisons show that the model has some advantages for predicting the N-year return-period wave height in typhoon-affected sea areas.

A Mean-Field Stochastic Maximum Principle for Optimal Control of Forward-Backward Stochastic Differential Equations with Jumps via Malliavin Calculus

This paper considers a mean-field type stochastic control problem where the dynamics is governed by a forward and backward stochastic differential equation (SDE) driven by Lévy processes and the information available to the controller is possibly less than the overall information. All the system coefficients and the objective performance functional are allowed to be random, possibly non-Markovian. Malliavin calculus is employed to derive a maximum principle for the optimal control of such a system where the adjoint process is explicitly expressed.

A modified method to calculate reliability index using maximum entropy principle

Routine reliability index method, first order second moment (FOSM), may not ensure convergence of iteration when the performance function is strongly nonlinear. A modified method was proposed to calculate reliability index based on maximum entropy (MaxEnt) principle. To achieve this goal, the complicated iteration of first order second moment (FOSM) method was replaced by the calculation of entropy density function. Local convergence of Newton iteration method utilized to calculate entropy density function was proved, which ensured the convergence of iteration when calculating reliability index. To promote calculation efficiency, Newton down-hill algorithm was incorporated into calculating entropy density function and Monte Carlo simulations (MCS) were performed to assess the efficiency of the presented method. Two numerical examples were presented to verify the validation of the presented method. Moreover, the execution and advantages of the presented method were explained. From Example 1, after seven times iteration, the proposed method is capable of calculating the reliability index when the performance function is strongly nonlinear and at the same time the proposed method can preserve the calculation accuracy; From Example 2, the reliability indices calculated using the proposed method, FOSM and MCS are 3.823 9, 3.813 0 and 3.827 6, respectively, and the according iteration times are 5, 36 and 10 6 , which shows that the presented method can improve calculation accuracy without increasing computational cost for the performance function of which the reliability index can be calculated using first order second moment (FOSM) method.

RESEARCH OF PINYIN-TO-CHARACTER CONVERSION BASED ON MAXIMUM ENTROPY MODEL

This paper applied Maximum Entropy (ME) model to Pinyin-To-Character (PTC) conversion in-stead of Hidden Markov Model (HMM) that could not include complicated and long-distance lexical informa-tion. Two ME models were built based on simple and complex templates respectively, and the complex one gave better conversion result. Furthermore, conversion trigger pair of y A → y B cBwas proposed to extract the long-distance constrain feature from the corpus; and then Average Mutual Information (AMI) was used to se-lect conversion trigger pair features which were added to the ME model. The experiment shows that conver-sion error of the ME with conversion trigger pairs is reduced by 4% on a small training corpus, comparing with HMM smoothed by absolute smoothing.

A microscopic ancient river channel identification method based on maximum entropy principle and Wigner-Ville Distribution and its application

In view of the problem of fine characterization of narrow and thin channels,the maximum entropy criterion is used to enhance the focusing characteristics of Wigner-Ville Distribution.On the basis of effectively improving the time-frequency resolution of seismic signal,a new method of microscopic ancient river channel identification is established.Based on the principle of the equivalence between the maximum entropy power spectrum and the AR model power spectrum,the prediction error and the autoregression coefficient of AR model are obtained using the Burg algorithm and Levinson-Durbin recurrence rule.Under the condition of the first derivative of autocorrelation function being 0,the Wigner-Ville Distribution of seismic signal is calculated,and the Wigner-Ville Distribution time-frequency power spectrum(MEWVD)is obtained under the maxi-mum entropy criterion of the microscopic ancient river channel.Through analysis of emulational seismic signal and forward numerical simulation signal of narrow thin model,it is found that MEWVD can effectively avoid the interference of cross term of Wigner-Ville Distribution,and obtain more accurate spectral characteristics than STFT and CWT signal analysis methods.It is also proved that the narrow and thin river channels of different scales can be identified effectively by MEWVD of different frequencies.The method is applied to the third member of Jurassic Shaximiao Formation(J2s33-2)gas reservoir of the Zhongji-ang gas field in Sichuan Basin.The spatial information of width and direction of narrow and thin river channels with width less than 500 m and sandstone thickness less than 35 m is accurately identified,providing bases for well deployment and horizontal well fracturing section selection.

Maximum Entropy and Bayesian Inference for the Monty Hall Problem

We devise an approach to Bayesian statistics and their applications in the analysis of the Monty Hall problem. We combine knowledge gained through applications of the Maximum Entropy Principle and Nash equilibrium strategies to provide results concerning the use of Bayesian approaches unique to the Monty Hall problem. We use a model to describe Monty’s decision process and clarify that Bayesian inference results in an “irrelevant, therefore invariant” hypothesis. We discuss the advantages of Bayesian inference over the frequentist inference in tackling the uneven prior probability Monty Hall variant. We demonstrate that the use of Bayesian statistics conforms to the Maximum Entropy Principle in information theory and Bayesian approach successfully resolves dilemmas in the uneven probability Monty Hall variant. Our findings have applications in the decision making, information theory, bioinformatics, quantum game theory and beyond.

Bifurcation and optimal control for an infectious disease model with the impact of information

A nonlinear infectious disease model with information-influenced vaccination behavior and contact patterns is proposed in this paper,and the impact of information related to disease prevalence on increasing vaccination coverage and reducing disease incidence during the outbreak is considered.First,we perform the analysis for the existence of equilibria and the stability properties of the proposed model.In particular,the geometric approach is used to obtain the sufficient condition which guarantees the global asymptotic stability of the unique endemic equilibrium Ee when the basic reproduction number Ro>1.Second,mathematical derivation combined with numerical simulation shows the existence of the double Hopf bifurcation around Ee.Third,based on the numerical results,it is shown that the information coverage and the average information delay may lead to more complex dynamical behaviors.Finally,the optimal control problem is established with information-infuenced vaccination and treatment as control variables.The corresponding optimal paths are obtained analytically by using Pontryagin's maximum principle,and the applicability and validity of virous intervention strategies for the proposed controls are presented by numerical experiments.

Maximum Entropy Distribution Function and Uncertainty Evaluation Criteria

Marine environmental design parameter extrapolation has important applications in marine engineering and coastal disaster prevention.The distribution models used for environmental design parameter usually pass the hypothesis tests in statistical analysis,but the calculation results of different distribution models often vary largely.In this paper,based on the information entropy,the overall uncertainty test criteria were studied for commonly used distributions including Gumbel,Weibull,and Pearson-III distribution.An improved method for parameter estimation of the maximum entropy distribution model is proposed on the basis of moment estimation.The study in this paper shows that the number of sample data and the degree of dispersion are proportional to the information entropy,and the overall uncertainty of the maximum entropy distribution model is minimal compared with other models.

Application of MEP Method to the Study of statistical Properties of Random Waves

The maximum entropy principle (MEP) method and the corresponding probability evaluation method are introduced, and the maximum entropy probability distribution expression is deduced in moment of the second order. Fully developed wave height distribution in deep water and wave height and period distribution for different depths in wind wave channel experiment are obtained from the MEP method, and the results are compared with the distribution and the experimental histogram. The wave height and period distribution for the Lianyungang port is also obtained by the MEP method, and the results are compared with the Weibull distribution and the field histogram.

Information Worth of MinMaxEnt Models for Time Series

In this study, by starting from Maximum entropy (MaxEnt) distribution of time series, we introduce a measure that quantifies information worth of a set of autocovariances. The information worth of autocovariences is measured in terms of entropy difference of MaxEnt distributions subject to different autocovariance sets due to the fact that the information discrepancy between two distributions is measured in terms of their entropy difference in MaxEnt modeling. However, MinMaxEnt distributions (models) are obtained on the basis of MaxEnt distributions dependent on parameters according to autocovariances for time series. This distribution is the one which has minimum entropy and maximum information out of all MaxEnt distributions for family of time series constructed by considering one or several values as parameters. Furthermore, it is shown that as the number of autocovariances increases, the entropy of approximating distribution goes on decreasing. In addition, it is proved that information worth of each model defined on the basis of MinMaxEnt modeling about stationary time series is equal to sum of all possible information increments corresponding to each model with respect to preceding model starting with first model in the sequence of models. The fulfillment of obtained results is demonstrated on an example by using a program written in Matlab.

Theoretical Properties of Composite Likelihoods

The general functional form of composite likelihoods is derived by minimizing the Kullback-Leibler distance under structural constraints associated with low dimensional densities. Connections with the I-projection and the maximum entropy distributions are shown. Asymptotic properties of composite likelihood inference under the proposed information-theoretical framework are established.

Information Theory Model for Radiation

Information based models for radiation emitted by a Black Body which passes through a scattering medium are analyzed. In the limit, when there is no scattering this model reverts to the Black Body Radiation Law. The advantage of this mathematical model is that it includes the effect of the scattering of the radiation between source and detector. In the case when the exact form of the scattering mechanism is not known a model using a single scattering parameter is derived. A simple version of this model is derived which is useful for analyzing large data.

Motoyosi Sugita—A “Widely Unknown” Japanese Thermodynamicist Who Explored the 4th Law of Thermodynamics for Creation of the Theory of Life

The purpose of this paper is to introduce to you, the Western people, nowadays a “widely unknown” Japanese thermodynamicist by the name of Motoyosi Sugita and his study on the thermodynamics of transient phenomena and his theory of life. This is because although he was one of the top theoretical physicists in Japan before, during and after WWII and after WWII he promoted the establishment of the biophysical society of Japan as one of the founding members, he himself and his studies themselves have seemed to be totally forgotten nowadays in spite that his study was absolutely important for the study of life. Therefore, in this paper I would like to present what kind of person he was and what he studied in physics as a review on the physics work of Motoyosi Sugita for the first time. I will follow his past studies to introduce his ideas in theoretical physics as well as in biophysics as follows: He proposed the bright ideas such as the quasi-static change in the broad sense, the virtual heat, and the field of chemical potential etc. in order to establish his own theory of thermodynamics of transient phenomena, as the generalization of the Onsager-Prigogine’s theory of the irreversible processes. By the concept of the field of chemical potential that acquired the nonlinear transport, he was seemingly successful to exceed and go beyond the scope of Onsager and Prigogine. Once he established his thermodynamics, he explored the existence of the 4th law of thermodynamics for the foundation of theory of life. He applied it to broad categories of transient phenomena including life and life being such as the theory of metabolism. He regarded the 4th law of thermodynamics as the maximum principle in transient phenomena. He tried to prove it all life long. Since I have recently found that his maximum principle can be included in more general maximum principle, which was known as the Pontryagin’s maximum principle in the theory of optimal control, I would like to explain such theories produced by Motoyosi Sugita as detailed as possible. And also I have put short history of Motoyosi Sugita’s personal life in order for you to know him well. I hope that this article helps you to know this wonderful man and understand what he did in the past, which was totally forgotten in the world and even in Japan.

月球探测器鲁棒环形山检测及光学导航方法

针对月球探测器环形山检测方法受光照影响、鲁棒性差的问题,本文提出一种基于极大熵阈值三值化的鲁棒环形山检测算法。采用不同滤波核对图像进行去噪平滑,然后对处理后的图像进行极大熵阈值分割、将图像信息三值化,去除图像对光源的敏感性,同时最大程度保留图像信息;提出一种归一化多指标约束环形山匹配和拟合方法完成环形山提取,将环形山提取算法应用于光学导航中进行打靶实验验证算法实时性表现。仿真结果表明:与传统基于形态学或自适应边缘检测的方法相比,本文方法在较大尺度条件下提取出连续、光滑的环形山边缘,有效环形山数量提升35%以上,同时实时性更好、计算消耗降低40%;基于鲁棒环形山提取的光学导航算法实时性更好。

Principle of maximum entropy for reliability analysis in the design of machine components

We studied the reliability of machine components with parameters that follow an arbitrary statistical distribution using the principle of maximum entropy(PME).We used PME to select the statistical distribution that best fits the available information.We also established a probability density function(PDF)and a failure probability model for the parameters of mechanical components using the concept of entropy and the PME.We obtained the first four moments of the state function for reliability analysis and design.Furthermore,we attained an estimate of the PDF with the fewest human bias factors using the PME.This function was used to calculate the reliability of the machine components,including a connecting rod,a vehicle half-shaft,a front axle,a rear axle housing,and a leaf spring,which have parameters that typically follow a non-normal distribution.Simulations were conducted for comparison.This study provides a design methodology for the reliability of mechanical components for practical engineering projects.