sys_mapping.bootstrap

Spatial covariance estimation via block bootstrap and jack-knife resampling.

Both estimators divide the survey footprint into \(K\) equal-area patches using HEALPix coarsening, capturing spatial correlations that pixel-level bootstrap misses.

Key papers: Ross et al. 2011; Ho et al. 2012; Berlfein et al. 2024, Sec. 6.2 — see also Methods Reference.

Spatial covariance estimation: block bootstrap and jack-knife (Sec. 6.2).

Block bootstrap (Berlfein+2024): resamples spatial patches with replacement. Jack-knife (Ross+2011, Ho+2012): leave-one-patch-out deterministic estimator. Both capture spatial correlations that pixel-level bootstrap misses.

sys_mapping.bootstrap.block_bootstrap_variance(delta_g_obs, delta_t, good_pixels, nside, estimator, n_bootstrap=100, n_patches=10, seed=0)[source]

Estimate variance of an estimator via spatial block bootstrap.

Divides the footprint into ~``n_patches`` spatial regions using HEALPix coarsening, then resamples regions with replacement n_bootstrap times. This captures spatial correlations that pixel-level bootstrap misses.

Parameters:

  • delta_g_obs ((n_good_pix,) observed overdensity at unmasked pixels)

  • delta_t ((n_sys, n_good_pix) template values at unmasked pixels)

  • good_pixels ((n_pix,) boolean mask that defines pixel sky positions)

  • nside (int full-resolution HEALPix NSIDE)

  • estimator (callable(delta_g, delta_t) -> (n_out,) array) – The quantity whose variance is to be estimated. Called once per bootstrap resample.

  • n_bootstrap (int number of bootstrap resamples (default 100))

  • n_patches (int approximate number of spatial blocks (default 10))

  • seed (int random seed for reproducibility)

Returns:

  • variance ((n_out,) bootstrap variance estimate (ddof=1))

  • Performance

  • ———–

  • Measured on CPU (n_boot=50, n_patches=8, nside=32, simple mean estimator)

  • **~32 ms/call**. Total cost scales as O(n_bootstrap × cost(estimator)).

  • Precision

  • ———

  • Bootstrap variance is non-negative by construction.

  • For the sample mean, variance decreases monotonically as n_pix increases.

  • Convergence to the true variance typically requires n_bootstrap ≥ 100.

Return type:

ndarray

Examples

>>> import numpy as np
>>> from sys_mapping import block_bootstrap_variance
>>> rng = np.random.default_rng(0)
>>> nside = 16
>>> n_pix = 12 * nside ** 2
>>> good   = np.ones(n_pix, dtype=bool)
>>> dg     = rng.standard_normal(n_pix) * 0.1
>>> dt     = rng.standard_normal((3, n_pix))
>>> def mean_estimator(dg, dt):
...     return np.array([np.mean(dg)])
>>> var = block_bootstrap_variance(dg, dt, good, nside,
...                                mean_estimator,
...                                n_bootstrap=100, n_patches=8, seed=1)
>>> var.shape
(1,)
>>> float(var[0]) >= 0
True
sys_mapping.bootstrap.jackknife_covariance(delta_g_obs, delta_t, good_pixels, nside, estimator, n_patches=10)[source]

Estimate the covariance matrix of an estimator via spatial jack-knife.

Divides the footprint into n_patches spatial regions (using the same HEALPix coarsening as block_bootstrap_variance()), then runs a leave-one-patch-out jack-knife. The standard jack-knife prefactor (K-1)/K is applied so the estimate is unbiased.

Parameters:

  • delta_g_obs ((n_good_pix,) observed overdensity at unmasked pixels)

  • delta_t ((n_sys, n_good_pix) template values at unmasked pixels)

  • good_pixels ((n_pix,) boolean mask that defines pixel sky positions)

  • nside (int full-resolution HEALPix NSIDE)

  • estimator (callable(delta_g, delta_t) -> (n_out,) array) – The quantity whose covariance is to be estimated. Called K times (once per patch dropped).

  • n_patches (int approximate number of spatial patches (default 10))

Returns:

cov

Return type:

(n_out, n_out) jack-knife covariance matrix

Notes

The jack-knife covariance is:

\[C_{ij}^{\rm JK} = \frac{K-1}{K} \sum_{k=1}^{K} (\hat\theta_k - \bar\theta)_i\, (\hat\theta_k - \bar\theta)_j\]

where \(\hat\theta_k\) is the estimator with patch \(k\) dropped and \(\bar\theta\) is the mean over all leave-one-out estimates.

Unlike block bootstrap, jack-knife is deterministic (no random resampling). For small footprints or fewer than 5 patches the estimate may be noisy; a warning is issued in that case.

Performance

Cost: O(K × cost(estimator)). Much cheaper than bootstrap since K (number of patches) is typically 10–30 vs. 100+ bootstrap resamples.

Precision

For a simple mean estimator, jack-knife and block bootstrap agree to within an order of magnitude. Jack-knife tends to be slightly conservative.

Examples

>>> import numpy as np
>>> from sys_mapping import jackknife_covariance
>>> rng = np.random.default_rng(0)
>>> nside = 16
>>> n_pix = 12 * nside ** 2
>>> good   = np.ones(n_pix, dtype=bool)
>>> dg     = rng.standard_normal(n_pix) * 0.1
>>> dt     = rng.standard_normal((3, n_pix))
>>> def mean_estimator(dg, dt):
...     return np.array([np.mean(dg)])
>>> cov = jackknife_covariance(dg, dt, good, nside,
...                            mean_estimator, n_patches=8)
>>> cov.shape
(1, 1)
>>> float(cov[0, 0]) >= 0
True

References

Ross et al. 2011, MNRAS 417, 1350. Ho et al. 2012, ApJ 761, 14.