Ball covariance

Ball covariance is a statistical measure that can be used to test the independence of two random variables defined on metric spaces. The ball covariance is zero if and only if two random variables are independent, making it a good measure of correlation. Its significant contribution lies in proposing an alternative measure of independence in metric spaces. Prior to this, distance covariance in metric spaces could only detect independence for distance types with strong negative type. However, ball covariance can determine independence for any distance measure.

Ball covariance uses permutation tests to calculate the p-value. This involves first computing the ball covariance for two sets of samples, then comparing this value with many permutation values.

Background

Correlation, as a fundamental concept of dependence in statistics, has been extensively developed in Hilbert spaces, exemplified by the Pearson correlation coefficient, Spearman correlation coefficient, and Hoeffding's dependence measure. However, with the advancement of time, many fields require the measurement of dependence or independence between complex objects, such as in medical imaging, computational biology, and computer vision. Examples of complex objects include Grassmann manifolds, planar shapes, tree-structured data, matrix Lie groups, deformation fields, symmetric positive definite (SPD) matrices, and shape representations of cortical and subcortical structures. These complex objects mostly exist in non-Hilbert spaces and are inherently nonlinear and high-dimensional (or even infinite-dimensional). Traditional statistical techniques, developed in Hilbert spaces, may not be directly applicable to such complex objects. Therefore, analyzing objects that may reside in non-Hilbert spaces poses significant mathematical and computational challenges.

Previously, a groundbreaking work in metric space independence tests was the distance covariance in metric spaces proposed by Lyons (2013). This statistic equals zero if and only if random variables are independent, provided the metric space is of strong negative type. However, testing the independence of random variables in spaces that do not meet the strong negative type condition requires new explorations.

Definition

Ball covariance

Next, we will introduce ball covariance in detail, starting with the definition of a ball. Suppose two Banach spaces: $(\mathbf {X} ,\rho )$ and $(\mathbf {Y} ,\zeta )$, where the norms $\rho$ and $\zeta$ also represent their induced distances. Let $\theta$ be a Borel probability measure on $\mathbf {X} \times \mathbf {Y} ,\mu ,\nu$ be two Borel probability measures on $\mathbf {X} ,\mathbf {Y}$, and $(X,Y)$ be a $B$-valued random variable defined on a probability space such that $(X,Y)\sim \theta ,X\sim \mu$, and $Y\sim \nu$. Denote the closed ball with the center $x_{1}$ and the radius $\rho \left(x_{1},x_{2}\right)$ in $\mathbf {X}$ as ${\bar {B}}\left(x_{1},\rho \left(x_{1},x_{2}\right)\right)$ or ${\bar {B}}_{\rho }\left(x_{1},x_{2}\right)$, and the closed ball with the center $y_{1}$ and the radius $\zeta \left(y_{1},y_{2}\right)$ in $\mathbf {Y}$ as ${\bar {B}}\left(y_{1},\zeta \left(y_{1},y_{2}\right)\right)$ or ${\bar {B}}_{\zeta }\left(y_{1},y_{2}\right)$. Let $\left\{W_{i}=\left(X_{i},Y_{i}\right),i=1,2,\ldots \right\}$ be an infinite sequence of iid samples of $(X,Y)$, and $\omega =\left(\omega _{1},\omega _{2}\right)$ be the positive weight function on the support set of $\theta$. Then, the population ball covariance can be defined as follows:

$\operatorname {BCov} _{\omega }^{2}(X,Y)=\int [\theta -\mu \otimes \nu ]^{2}\left({\bar {B}}_{\rho }\left(x_{1},x_{2}\right)\times {\bar {B}}_{\zeta }\left(y_{1},y_{2}\right)\right)\omega _{1}\left(x_{1},x_{2}\right)\omega _{2}\left(y_{1},y_{2}\right)\theta \left(dx_{1},dy_{1}\right)\theta \left(dx_{2},dy_{2}\right)$

where$[\theta -\mu \otimes \nu ]^{2}(A\times B):=[\theta (A\times B)-\mu (A)v(B)]$ for $A\in \mathbf {X}$ and $B\in \mathbf {Y}$.

Next, we will introduce another form of population ball covariance. Suppose $\delta _{ij,k}^{X}:=I\left(X_{k}\in {\bar {B}}_{\rho }\left(X_{i},X_{j}\right)\right)$ which indicates whether $X_{k}$ is located in the closed ball ${\bar {B}}_{\rho }\left(X_{i},X_{j}\right)$. Then, let $\delta _{ij,kl}^{X}=\delta _{ij,k}^{X}\delta _{ij,l}^{X}$ means whether both $X_{k}$ and $X_{l}$ is located in ${\bar {B}}_{\rho }\left(X_{i},X_{j}\right)$, and $\xi _{ij,klst}^{X}=\left(\delta _{ij,kl}^{X}+\delta _{ij,st}^{X}-\delta _{ij,ks}^{X}-\delta _{ij,lt}^{X}\right)/2$. So does $\delta _{ij,k}^{Y}$, $\delta _{ij,kl}^{Y}$ and $\xi _{ij,klst}^{Y}$ for $Y$. Then, let $(X_{i},Y_{i})$, $i=1,2,\dots ,6$ be iid samples from $\theta$. Another form of population ball covariance can be shown as

$\operatorname {BCov} _{\omega }^{2}(X,Y)=E\left\{\xi _{12,3456}^{X}\xi _{12,3456}^{Y}\omega _{1}\left(X_{1},X_{2}\right)\omega _{2}\left(Y_{1},Y_{2}\right)\right\}$

Now, we can finally express the sample ball covariance. Consider the random sample $(\mathbf {X} ,\mathbf {Y} ={X_{k},Y_{k}},k=1,\ldots ,n)$. Let ${\hat {\omega }}_{1,n}$ and ${\hat {\omega }}_{2,n}$ be the estimate of ${\omega }_{1}$ and ${\omega }_{2}$. Denote $\Delta _{ij,n}^{XY}={\frac {1}{n}}\sum _{k=1}^{n}\delta _{ij,k}^{X}\delta _{ij,k}^{Y},\Delta _{ij,n}^{X}={\frac {1}{n}}\sum _{k=1}^{n}\delta _{ij,k}^{X},\Delta _{ij,n}^{Y}={\frac {1}{n}}\sum _{k=1}^{n}\delta _{ij,k}^{Y},$ the sample ball covariance is $\mathbf {BCov} _{\omega ,n}^{2}(\mathbf {X} ,\mathbf {Y} ):={\frac {1}{n^{2}}}\sum _{i,j=1}^{n}\left(\Delta _{ij,n}^{XY}-\Delta _{ij,n}^{X}\Delta _{ij,n}^{Y}\right)^{2}\times {\hat {\omega }}_{1,n}\left(X_{i},X_{j}\right){\hat {\omega }}_{2,n}\left(Y_{i},Y_{j}\right).$

Ball correlation

Just like the relationship between the Pearson correlation coefficient and covariance, we can define the ball correlation coefficient through ball covariance. The ball correlation is defined as the square root of

$\operatorname {BCor} _{\omega }^{2}(X,Y):=\operatorname {BCov} _{\omega }^{2}(X,Y)/{\sqrt {\mathbf {BCov} _{\omega }^{2}(X)\mathbf {BCov} _{\omega }^{2}(Y)}},$

where $\mathbf {BCov} _{\omega }^{2}(X)=\mathbf {BCov} _{\omega }^{2}(X,X)=E\left(\xi _{12,3456}^{X}\omega _{1}\left(X_{1},X_{2}\right)\right)^{2},$ and $\mathbf {BCov} _{\omega }^{2}(Y)=\mathbf {BCov} _{\omega }^{2}(Y,Y)=E\left(\xi _{12,3456}^{Y}\omega _{1}\left(Y_{1},Y_{2}\right)\right)^{2}.$ And the sample ball correlation is defined similarly, $\operatorname {BCor} _{\omega ,n}^{2}(X,Y):=\operatorname {BCov} _{\omega ,n}^{2}(X,Y)/{\sqrt {\mathbf {BCov} _{\omega ,n}^{2}(X)\mathbf {BCov} _{\omega ,n}^{2}(Y)}},$ where $\mathbf {BCov} _{\omega ,n}^{2}(X)=\mathbf {BCov} _{\omega ,n}^{2}(X,X),$ and $\mathbf {BCov} _{\omega ,n}^{2}(Y)=\mathbf {BCov} _{\omega ,n}^{2}(Y,Y).$

Properties

1.Independence-zero equivalence property: Let $S_{\theta }$, $S_{\mu }$ and $S_{\nu }$ denote the support sets of $\theta$, $\mu$ and $\nu$, respectively. $\operatorname {BCov} _{\omega }(X,Y)=0$ implies $\theta =\mu \otimes \nu$ if one of the following conditions establish:

(a).$\mathbf {X} \times \mathbf {Y}$ is a finite dimensional Banach space with $S_{\theta }=S_{\mu }\times S_{\nu }$.

(b).$\theta =a_{1}\theta _{d}+a_{2}\theta _{a}$, where $a_{1}$ and $a_{2}$ are positive constants, $\theta _{d}$ is a discrete measure, and $\theta _{a}$ is an absolutely continuous measure with a continues Radon–Nikodym derivative with respect to the Gaussian measure.

2.Cauchy–Schwarz type inequality: $\operatorname {BCov} _{\omega }^{2}(X,Y)\leq \operatorname {BCov} _{\omega }(X)\operatorname {BCov} _{\omega }(X)$

3.Consistence: If ${\hat {\omega }}_{1,n}$ and ${\hat {\omega }}_{2,n}$ uniformly converge ${\omega }_{1}$ and ${\omega }_{2}$ with $E(\omega _{1}\omega _{2})<\infty$ respectively, we have $\operatorname {BCov} _{\omega ,n}(\mathbf {X} ,\mathbf {Y} ){\underset {n\rightarrow \infty }{\stackrel {a.s.}{\longrightarrow }}}\operatorname {BCov} _{\omega }(X,Y)$ and $\operatorname {BCor} _{\omega ,n}(\mathbf {X} ,\mathbf {Y} ){\underset {n\rightarrow \infty }{\stackrel {a.s.}{\longrightarrow }}}\operatorname {BCor} _{\omega }(X,Y)$.

4.Asymptotics: If ${\hat {\omega }}_{1,n}$ and ${\hat {\omega }}_{2,n}$ uniformly converge ${\omega }_{1}$ and ${\omega }_{2}$ with $E(\omega _{1}\omega _{2})<\infty$ respectively, (a)under the null hypothesis, we have $n\mathbf {BCov} {}_{\omega ,n}^{2}(\mathbf {X} ,\mathbf {Y} ){\xrightarrow[{n\rightarrow \infty }]{d}}\sum _{v=1}^{\infty }\lambda _{v}Z_{v}^{2}$, where $Z_{v}$ are independent standard normal random variables.

(b)under the alternative hypothesis, we have ${\sqrt {n}}\left(\mathbf {BCov} _{\omega ,n}^{2}(\mathbf {X} ,\mathbf {Y} )-\mathbf {BCov} _{\omega }^{2}(X,Y)\right){\xrightarrow[{n\rightarrow \infty }]{d}}N(0,\Sigma )$.