Dirichlet; familywise error rate; Kolmogorov–Smirnov; probability integral transform; regression discontinuity

Comparing distributions by multiple testing across quantiles or CDF values

When comparing two distributions, it is often helpful to learn at which quantiles or values there is a statistically significant difference. This provides more information than the binary "reject" or "do not reject" decision of a global goodness-of-fit test. Framing our question as multiple testing across the continuum of quantiles tau in (0,1) or values r, we show that the Kolmogorov–Smirnov test (interpreted as a multiple testing procedure) achieves strong control of the familywise error rate. However, its well-known flaw of low sensitivity in the tails remains.

Subscribe to Dirichlet; familywise error rate; Kolmogorov–Smirnov; probability integral transform; regression discontinuity