HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION

Size: px

Start display at page:

Download "HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION"

Anne Cook
5 years ago
Views:

HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION VIA SPARSE PROJECTION 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 6 63 67 71 73 77 8 83 86 88 91 93 96 98 11 15 19 113 114 118 12 121 125 126 129

275 278 279 281 284 286 289 291 294 296 3 34 38 311 314 317 32 324 326 33 334 336 339 341 345 349 351 354 356 36 364 368 369 371 372 376 38 381 385 388 391 394 396 4 42 46 41 411 415 417 42 423 424

1 HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION VIA SPARSE PROJECTION nodes in binary segmentation algorithm peak of projected CUSUM Richard Samworth, University of Cambridge Joint work with Tengyao Wang

2 Tengyao August 13; 2/31

3 Heterogeneity in Big Data One of the most commonly-encountered issues with Big Data is heterogeneity. Departures from traditional, stylised i.i.d. models can take many forms, e.g. missing data, correlated errors, data combined from multiple sources,... In data streams, heterogeneity is manifested through non-stationarity. Perhaps the simplest model assumes population changes occur at a finite set of time points. August 13; 3/31

4 Changepoint estimation Changepoint problems have a rich history (Page, 1955). State-of-the-art univariate methods include PELT (Killick, Fearnhead and Eckley, 212), Wild Binary Segmentation (Fryzlewicz, 214) and SMUCE (Frick, Munk and Sieling, 214). Some ideas extend to multivariate settings (Horváth, Kokoszka and Steinebach,1999; Ombao, Von Sachs and Guo, 25; Aue et al., 29; Kirch, Mushal and Ombao, 214). Increasing interest in high-dimensional setting, possibly with a sparsity condition on coordinates of change (Aston and Kirch, 214; Enikeeva and Harchaoui, 214; Jirak, 215; Cho and Fryzlewicz, 215; Cho, 216). August 13; 4/31

5 Basic model Let X = (X 1,..., X n ) R p n have independent columns X t N p (µ t, σ 2 I p ). Assume there exist changepoints 1 z 1 < z 2 < < z ν n 1 such that µ zi +1 = = µ zi+1 =: µ (i), i ν, where z := and z ν+1 := n. Writing θ (i) := µ (i) µ (i 1), 1 i ν, we assume k {1,..., p} s.t. θ (i) k for 1 i ν. August 13; 5/31

6 Further model assumptions Assume stationary run lengths satisfy 1 n min{z i+1 z i : i ν} τ, and the magnitudes of mean changes are such that θ (i) 2 ϑ, 1 i ν. Let P(n, p, k, ν, ϑ, τ, σ 2 ) be the set of distributions of such X. August 13; 6/31

7 Projection-based single changepoint estimation µ + W = X Let ν = 1, write z := z 1, θ := θ (1) and τ := n 1 min{z, n z}. For any a S p 1, a X t N(a µ t, σ 2 ). Hence a = θ/ θ 2 =: v maximises the magnitude of the difference in means between the two segments. August 13; 7/31

8 CUSUM transformation Define CUSUM transformation T p,n : R p n R p (n 1) by [T (M)] j,t = [T p,n (M)] j,t := ( n t(n t) n r=t+1 M j,r t n t r=1 M j,r t ). T (µ) + T (W ) = T (X) A + E = T August 13; 8/31

9 SVD of CUSUM transformation When ν = 1, we can compute A explicitly: t n(n t) A j,t = (n z)θ j, if t z =: (θγ ) j,t, n t nt zθ j, if t > z so the oracle projection direction is the leading left singular vector of the rank 1 matrix A. We could therefore consider estimating v by ˆv max,k argmaxṽ S p 1 (k) T ṽ 2, and indeed when n 6, with probability at least 1 4(p log n) 1/2, sin (ˆv max,k, v) 16 2σ k log(p log n). τϑ n August 13; 9/31

10 A computationally efficient projection Computing the k-sparse leading left singular vector of a matrix is NP-hard (Tillmann and Pfetsch, 214). However, max u S p 1 (k) u T 2 = max u S p 1 (k),w S u T w n 2 = max u S p 1,w S n 2, u k uw, T = max M, T, M M where M := {M : M = 1, rk(m) = 1, nnzr(m) k}. For λ >, we therefore consider computing ˆM argmax M S 1 { T, M λ M 1 }, where S 1 := {M R p (n 1) : M 1}, using ADMM. We can then let ˆv be a leading left singular vector of ˆM. August 13; 1/31

11 Alternative relaxation Let S 2 := {M R p (n 1) : M 2 1}. Then the simple dual formulation leads to M := soft(t, λ) soft(t, λ) 2 = argmax M S 2 { T, M λ M 1 }. Suppose ˆM argmax M S { T, M λ M 1 } for S = S1 or S = S 2 and let ˆv argmaxṽ S p 1 ˆM ṽ 2. If n 6 and λ 2σ log(p log n), then w.p. at least 1 4(p log n) 1/2, sin (ˆv, v) 32λ k τϑ n. August 13; 11/31

12 Changepoint estimation after projection Input: X R p n, λ >. Step 1: Perform CUSUM transformation T T (X) Step 2: Find ˆM { } argmax M S T, M λ M 1 for S = S 1 or S 2 Step 3: Find ˆv argmaxṽ S p 1 ˆM ṽ 2. Step 4: Let ẑ argmax 1 t n 1 ˆv T t, where T t is the tth column of T, and set T max ˆv Tẑ Output: ẑ, T max August 13; 12/31

13 Sample-splitting version performance Let σ > be known and X P P(n, p, k, 1, ϑ, τ, σ 2 ). Let ẑ be the output of sample-splitting algorithm with input X, σ and λ := 2σ log(p log n). C, C > such that if n 12, z is even and Cσ ϑτ k log(p log n) n 1, then w.p. at least 1 4{p log(n/2)} 1/2 17/ log(n/2), 1 n ẑ z C σ 2 log log n nϑ 2. If σ is constant, log p = O(log n), ϑ n a, τ n b, k n c and a + b + c/2 < 1/2, then rate of convergence is o(n 1+2a+δ ) for all δ >. August 13; 13/31

14 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 214) August 13; 14/31

15 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 214) August 13; 15/31

16 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 214) August 13; 16/31

Example 5 1 15 2 25 3 5 1 15 2 candidate changepoint location projected CUSUM statistics 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 6 63 67 71 73 77 8 83 86 88 91 93 96 98 11 15 19 113

255 259 262 265 268 269 272 275 278 279 281 284 286 289 291 294 296 3 34 38 311 314 317 32 324 326 33 334 336 339 341 345 349 351 354 356 36 364 368 369 371 372 376 38 381 385 388 391 394 396 4 42 46

17 Example candidate changepoint location projected CUSUM statistics nodes in binary segmentation algorithm peak of projected CUSUM August 13; 17/31

18 Multiple changepoint estimation inspect Input: X R p n, λ >, ξ >, β >, Q N. Step 1: Set Ẑ. Draw (s 1, e 1 ),..., (s Q, e Q ) from {(l, r) Z 2 : l < r n}. Step 2: Run wbs(, n) where wbs is defined below. Step 3: Let ˆν Ẑ and sort Ẑ to yield ẑ 1 < < ẑˆν. Output: ẑ 1,..., ẑˆν Function wbs(s, e) Set Q s,e {q : s + nβ s q < e q e nβ} T [q] For q Q s,e, let (ẑ [q], max) SingleCP(X [q], λ) [q] Find q argmax q Qs,e T max and set b s q + ẑ [q ] If T [q ] max > ξ then Ẑ Ẑ {b}; wbs(s,b); wbs(b,e) end August 13; 18/31

19 Theory for inspect inspect : whenever SingleCP is called, second and third steps are on an independent copy X of X. Assume σ > known and X, X iid P P(n, p, k, ν, ϑ, τ, σ 2 ). Let ẑ 1 < < ẑˆν be output of inspect with input X, X, λ := 4σ log(np), ξ := λ, β and Q. Define ρ = ρ n := λ 2 n 1 ϑ 2 τ 4 and assume nτ 14. C, C > such that if C ρ < β/2 < τ/c and Cρkτ 2 1, then P P {ˆν = ν & ẑi z i C nρ, 1 i ν } 1 e τ 2 Q/9 Assume log p = O(log n), ϑ n a, τ n b, k n c. If a + b + c/2 < 1/2 and 2a + 5b < 1, then conditions can τ 6 log n np 4. hold for large n and rate is o(n (1 2a 4b)+δ ) for all δ >. August 13; 19/31

20 S 1 or S 2? Angles (in degrees) between oracle projection direction v and estimated projection directions ˆv S1 (using S 1 ) and ˆv S2 (using S 2 ), for different choices of ϑ. ϑ (ˆv S1, v) (ˆv S2, v) ϑ (ˆv S1, v) (ˆv S2, v) Other parameters: n = 5, p = 1, k = 3, z = 2, σ 2 = 1. August 13; 2/31

21 Single changepoint simulations RMSE θ = (1, 2 1/2,..., k 1/2,,..., ) R p, ϑ =.8, σ 2 = 1 n p k z inspect dc sbs scan agg 2 agg August 13; 21/31

22 Changepoint density estimates Left: (n, p, k, z, ϑ, σ 2 ) = (2, 1, 32, 8,.5, 1). Right: (n, p, k, z, ϑ, σ 2 ) = (2, 1, 32, 8, 1, 1). density inspect dc sbs scan agg 2 agg estimated changepoint location density estimated changepoint location inspect dc sbs scan agg 2 agg August 13; 22/31

23 Misspecified settings (n, p, k, z, ϑ) = (2, 1, 32, 8, 1.5). Model inspect dc sbs scan agg agg 2 M unif M exp M cs,loc (.2) M cs,loc (.5) M cs (.5) M cs (.9) M temp (.1) M temp (.3) M async (1) August 13; 23/31

24 Multiple changepoint simulations n = 2, p = 2, k = 4, z = (5, 1, 15), σ 2 = 1. (ϑ (1), ϑ (2), ϑ (3) ) method ˆν ARI % best (.4,.8, 1.2) inspect dc sbs scan agg agg (.6, 1.2, 1.8) inspect dc sbs scan agg agg August 13; 24/31

25 Histograms of estimated changepoints n = 2, p = 2, k = 4, z = (5, 1, 15), (ϑ (1), ϑ (2), ϑ (3) ) = (.6, 1.2, 1.8), σ = 1. frequency frequency changepoints estimated by inspect changepoints estimated by dc frequency frequency changepoints estimated by sbs changepoints estimated by scan frequency frequency changepoints estimated by agg 2 changepoints estimated by agg August 13; 25/31

26 Comparative genomic hybridisation dataset patient number loci August 13; 26/31

27 Temporal dependence Now assume noise vectors W 1,..., W n form a stationary Gaussian process with covariance function K, so K(u) = Cov(W t, W t+u ). Assume n 1 u= K(u) op B, where B is known, and let ẑ be the output of inspect with λ := σ 8B log(np). C, C > such that if n 12 and z are even, and Cσ ϑτ kb log(np) n 1, then ( 1 P n ẑ z C σ 2 ) B log n nϑ n. August 13; 27/31

28 Spatial dependence Now suppose W 1,..., W n iid Np (, Σ) with Σ. Then a θ v proj = argmax a S p 1 a Σa = Σ 1/2 argmax b Σ 1/2 θ = Σ 1 θ b S p 1 Σ 1. θ 2 If ˆΘ is an estimator of Θ := Σ 1, and ˆv is a leading left singular vector of ˆM as before, then we can estimate v proj by ˆv proj := ˆΘˆv ˆΘˆv 2. Under assumptions on Θ that allow us to control ˆΘ Θ op, analogous theory can be obtained. August 13; 28/31

29 Summary inspect is a new method for high-dimensional changepoint estimation Convex relaxation used to find projection direction, then CUSUM and WBS to identify multiple changepoints R package InspectChangepoint available! August 13; 29/31

30 References Aston, J. A. D. and Kirch, C. (214) Change points in high dimensional settings. arxiv: Aue, A., Hörmann, S., Horváth, L. and Reimherr, M. (29). Break detection in the covariance structure of multivariate time series models. Ann. Statist. 37, Cho, H. (216) Change-point detection in panel data via double CUSUM statistic. preprint. Cho, H. and Fryzlewicz, P. (215) Multiple changepoint detection for high dimensional time series via sparsified binary segmentation. J. R. Stat. Soc. Ser. B, 77, Enikeeva, F. and Harchaoui, Z. (214) High-dimensional change-point detection with sparse alternatives. arxiv: v2. Frick, K., Munk, A. and Sieling, H. (214) Multiscale change point inference. J. R. Stat. Soc. Ser. B 76, Fryzlewicz, P. (214) Wild binary segmentation for multiple change-point detection. Ann. Statist., 42, August 13; 3/31

31 Horváth, L., Kokoszka, P. and Steinebach, J. (1999) Testing for changes in dependent observations with an application to temperature changes. J. Multi. Anal., 68, Jirak, M. (215) Uniform change point tests in high dimension. Ann. Statist., 43, Killick, R., Fearnhead, P. and Eckley, I. A. (212) Optimal detection of changepoints with a linear computational cost. J. Amer. Stat. Assoc. 17, Kirch, C., Mushal, B. and Ombao, H. (215) Detection of changes in multivariate time series with applications to EEG data. J. Amer. Statist. Assoc., 11, Ombao, H., Von Sachs, R. and Guo, W. (25) SLEX analysis of multivariate nonstationary time series. J. Amer. Statist. Assoc., 1, Page, E. S. (1955) A test for a change in a parameter occurring at an unknown point. Biometrika, 42, Tillmann, A. N. and Pfetsch M. E. (214) The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans. Inform. Theory, 6, Wang, T. and Samworth, R. J. (217) High-dimensional changepoint estimation via sparse projection. J. Roy. Statist. Soc., Ser. B, to appear. August 13; 31/31

HIGH-DIMENSIONAL CHANGEPOINT DETECTION

HIGH-DIMENSIONAL CHANGEPOINT DETECTION VIA SPARSE PROJECTION 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 60 63 67 71 73 77 80 83 86 88 91 93 96 98 101 105 109 113 114 118 120 121 125 126 129