On a Rayleigh-Faber-Krahn Inequality for the Regional Fractional Laplacian

We study a Rayleigh-Faber-Krahn inequality for regional fractional Laplacian operators.In particular,we show that there exists a compactly supported nonnegative Sobolev function u_(0)that attains the infimum(which will be a positive real number)of the set{{∫∫(u>0)×(u>0)|u(x)-u(y)|^(2)/|x-y|^(n+2σ)dxdy:u∈^(σ)(R^(n)),∫R^(n)u^(2)=1,|{u>0}|≤1}.Unlike the corresponding problem for the usual fractional Laplacian,where the domain of the integration is R^(n)×R^(n),symmetrization techniques may not apply here.Our approach is instead based on the direct method and new a priori diameter estimates.We also present several remaining open questions concerning the regularity and shape of the minimizers,and the form of the Euler-Lagrange equations.