HIGH-DIMENSIONAL CHANGEPOINT DETECTION

Size: px

Start display at page:

Download "HIGH-DIMENSIONAL CHANGEPOINT DETECTION"

Dennis Crawford
5 years ago
Views:

HIGH-DIMENSIONAL CHANGEPOINT DETECTION VIA SPARSE PROJECTION 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 60 63 67 71 73 77 80 83 86 88 91 93 96 98 101 105 109 113 114 118 120 121 125 126

268 269 272 275 278 279 281 284 286 289 291 294 296 300 304 308 311 314 317 320 324 326 330 334 336 339 341 345 349 351 354 356 360 364 368 369 371 372 376 380 381 385 388 391 394 396 400 402 406 410

1 HIGH-DIMENSIONAL CHANGEPOINT DETECTION VIA SPARSE PROJECTION nodes in binary segmentation algorithm peak of projected CUSUM Richard Samworth, University of Cambridge Joint work with Tengyao Wang

2 Tengyao November 15-2

3 Heterogeneity in Big Data One of the most commonly-encountered issues with Big Data is heterogeneity. Departures from traditional, stylised i.i.d. models can take many forms, e.g. missing data, correlated errors, data combined from multiple sources,... In data streams, heterogeneity is manifested through non-stationarity. Perhaps the simplest model assumes population changes occur at a finite set of time points. November 15-3

4 Changepoint estimation Changepoint problems have a rich history (Page, 1955). State-of-the-art univariate methods include PELT (Killick, Fearnhead and Eckley, 2012), Wild Binary Segmentation (Fryzlewicz, 2014) and SMUCE (Frick, Munk and Sieling, 2014). Some ideas extend to multivariate settings (Horváth, Kokoszka and Steinebach,1999; Ombao, Von Sachs and Guo, 2005; Aue et al., 2009; Kirch, Mushal and Ombao, 2014). Increasing interest in high-dimensional setting, possibly with a sparsity condition on coordinates of change (Aston and Kirch, 2014; Enikeeva and Harchaoui, 2014; Jirak, 2015; Cho and Fryzlewicz, 2015; Cho, 2016). November 15-4

5 Basic model Let X = (X 1,..., X n ) R p n have independent columns X t N p (µ t, σ 2 I p ). Assume there exist changepoints 1 z 1 < z 2 < < z ν n 1 such that µ zi +1 = = µ zi+1 =: µ (i), 0 i ν, where z 0 := 0 and z ν+1 := n. Writing θ (i) := µ (i) µ (i 1), 1 i ν, we assume k {1,..., p} s.t. θ (i) 0 k for 1 i ν. November 15-5

6 Further model assumptions Assume stationary run lengths satisfy 1 n min{z i+1 z i : 0 i ν} τ, and the magnitudes of mean changes are such that θ (i) 2 ϑ, 1 i ν. Let P(n, p, k, ν, ϑ, τ, σ 2 ) be the set of distributions of such X. November 15-6

7 Projection-based single changepoint estimation µ + W = X Let ν = 1, write z := z 1, θ := θ (1) and τ := n 1 min{z, n z}. For any a S p 1, a X t N(a µ t, σ 2 ). Hence a = θ/ θ 2 =: v maximises the magnitude of the difference in means between the two segments. November 15-7

8 CUSUM transformation Define CUSUM transformation T p,n : R p n R p (n 1) by [T (M)] j,t = [T p,n (M)] j,t := ( n t(n t) n r=t+1 M j,r t n t r=1 M j,r t ). T (µ) + T (W ) = T (X) A + E = T November 15-8

9 SVD of CUSUM transformation When ν = 1, we can compute A explicitly: t n(n t) A j,t = (n z)θ j, if t z =: (θγ ) j,t, n t nt zθ j, if t > z so the oracle projection direction is the leading left singular vector of the rank 1 matrix A. We could therefore consider estimating v by ˆv max,k argmaxṽ S p 1 (k) T ṽ 2, and indeed when n 6, with probability at least 1 4(p log n) 1/2, sin (ˆv max,k, v) 16 2σ k log(p log n). ϑ nτ November 15-9

10 A computationally efficient projection Computing the k-sparse leading left singular vector of a matrix is NP-hard (Tillmann and Pfetsch, 2014). However, max u S p 1 (k) u T 2 = max u S p 1 (k),w S u T w n 2 = max u S p 1,w S n 2, u 0 k uw, T = max M, T, M M where M := {M : M = 1, rk(m) = 1, nnzr(m) k}. For λ > 0, we therefore consider computing ˆM argmax M S 1 { T, M λ M 1 }, where S 1 := {M R p (n 1) : M 1}, using ADMM. We can then let ˆv be a leading left singular vector of ˆM. November 15-10

11 Alternative relaxation Let S 2 := {M R p (n 1) : M 2 1}. Then the simple dual formulation leads to M := soft(t, λ) soft(t, λ) 2 = argmax M S 2 { T, M λ M 1 }. Suppose ˆM argmax M S { T, M λ M 1 } for S = S1 or S = S 2 and let ˆv argmaxṽ S p 1 ˆM ṽ 2. If n 6 and λ 2σ log(p log n), then w.p. at least 1 4(p log n) 1/2, sin (ˆv, v) 32λ k τϑ n. November 15-11

12 Changepoint estimation after projection Input: X R p n, λ > 0. Step 1: Perform CUSUM transformation T T (X) Step 2: Find ˆM { } argmax M S T, M λ M 1 for S = S 1 or S 2 Step 3: Find ˆv argmaxṽ S p 1 ˆM ṽ 2. Step 4: Let ẑ argmax 1 t n 1 ˆv T t, where T t is the tth column of T, and set T max ˆv Tẑ Output: ẑ, T max November 15-12

13 Sample-splitting version performance Suppose σ > 0 is known and X P P(n, p, k, 1, ϑ, τ, σ 2 ). Let ẑ be the output of sample-splitting algorithm with input X, σ and λ := 2σ log(p log n). If n 6 is even and σ ϑτ k log(p log n) n 3 128, then with probability at least 1 4{p log(n/2)} 1/2 2/n, 1 32σ log n ẑ z n ϑ nτ. If log p = O(log n), ϑ n a, τ n b, k n c and a + b + c/2 < 1/2, then rate of convergence is o(n 1 2a b 2 +δ ) for all δ > 0. November 15-13

14 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 2014) November 15-14

15 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 2014) November 15-15

16 Multiple changepoint estimation inspect Wild binary segmentation scheme (Fryzlewicz, 2014) November 15-16

Example 0 5 10 15 20 25 30 500 1000 1500 2000 candidate changepoint location projected CUSUM statistics 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 60 63 67 71 73 77 80 83 86 88 91 93 96

239 240 241 244 246 248 252 255 259 262 265 268 269 272 275 278 279 281 284 286 289 291 294 296 300 304 308 311 314 317 320 324 326 330 334 336 339 341 345 349 351 354 356 360 364 368 369 371 372 376

17 Example candidate changepoint location projected CUSUM statistics nodes in binary segmentation algorithm peak of projected CUSUM November 15-17

18 S 1 or S 2? Angles (in degrees) between oracle projection direction v and estimated projection directions ˆv S1 (using S 1 ) and ˆv S2 (using S 2 ), for different choices of ϑ. ϑ (ˆv S1, v) (ˆv S2, v) ϑ (ˆv S1, v) (ˆv S2, v) November 15-18

19 Single changepoint simulations RMSE θ = (1, 2 1/2,..., k 1/2, 0,..., 0) R p. n p k z ϑ inspect dc sbs scan November 15-19

20 Changepoint density estimates Left: (n, p, k, z, ϑ) = (2000, 1000, 32, 800, 0.40). Right: (n, p, k, z, ϑ) = (2000, 1000, 32, 800, 1.02). density inspect dc sbs scan estimated changepoint location density inspect dc sbs scan estimated changepoint location November 15-20

21 Misspecified settings (n, p, k, z, ϑ) = (2000, 1000, 32, 800, 1.7). Model inspect dc sbs scan M unif M exp M cs,loc (0.2) M cs,loc (0.5) M cs (0.5) M cs (0.9) M temp (0.1) M temp (0.3) November 15-21

22 Multiple changepoint simulations n = 2000, p = 200, k = 40, z = (500, 1000, 1500). Writing ϑ (i) := θ (i) 2, set (ϑ (1), ϑ (2), ϑ (3) ) = ϑ(1, 1.5, 2). ˆν ϑ method Rand % best inspect dc sbs scan inspect dc sbs scan inspect dc sbs scan November 15-22

23 Histograms of estimated changepoints n = 2000, p = 200, k = 40, z = (500, 1000, 1500), (ϑ (1), ϑ (2), ϑ (3) ) = (0.63, 0.95, 1.26), σ = frequency frequency changepoints estimated by inspect changepoints estimated by dc 0 0 frequency frequency changepoints estimated by sbs changepoints estimated by scan November 15-23

24 Summary inspect is a new method for high-dimensional changepoint estimation Convex relaxation used to find projection direction, then CUSUM and WBS to identify multiple changepoints R package InspectChangepoint available! November 15-24

25 References Aston, J. A. D. and Kirch, C. (2014) Change points in high dimensional settings. arxiv: Aue, A., Hörmann, S., Horváth, L. and Reimherr, M. (2009). Break detection in the covariance structure of multivariate time series models. Ann. Statist. 37, Cho, H. (2016) Change-point detection in panel data via double CUSUM statistic. preprint. Cho, H. and Fryzlewicz, P. (2015) Multiple changepoint detection for high dimensional time series via sparsified binary segmentation. J. R. Stat. Soc. Ser. B, 77, Enikeeva, F. and Harchaoui, Z. (2014) High-dimensional change-point detection with sparse alternatives. arxiv: v2. Frick, K., Munk, A. and Sieling, H. (2014) Multiscale change point inference. J. R. Stat. Soc. Ser. B 76, Fryzlewicz, P. (2014) Wild binary segmentation for multiple change-point detection. Ann. Statist., 42, November 15-25

26 Horváth, L., Kokoszka, P. and Steinebach, J. (1999) Testing for changes in dependent observations with an application to temperature changes. J. Multi. Anal., 68, Jirak, M. (2015) Uniform change point tests in high dimension. Ann. Statist., 43, Killick, R., Fearnhead, P. and Eckley, I. A. (2012) Optimal detection of changepoints with a linear computational cost. J. Amer. Stat. Assoc. 107, Kirch, C., Mushal, B. and Ombao, H. (2015) Detection of changes in multivariate time series with applications to EEG data. J. Amer. Statist. Assoc., 110, Ombao, H., Von Sachs, R. and Guo, W. (2005) SLEX analysis of multivariate nonstationary time series. J. Amer. Statist. Assoc., 100, Page, E. S. (1955) A test for a change in a parameter occurring at an unknown point. Biometrika, 42, Tillmann, A. N. and Pfetsch M. E. (2014) The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans. Inform. Theory, 60, Wang, T. and Samworth, R. J. (2016) High-dimensional changepoint estimation via sparse projection. November 15-26

HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION

HIGH-DIMENSIONAL CHANGEPOINT ESTIMATION VIA SPARSE PROJECTION 3 6 8 11 14 16 19 22 26 28 31 33 35 39 43 47 48 52 53 56 6 63 67 71 73 77 8 83 86 88 91 93 96 98 11 15 19 113 114 118 12 121 125 126 129 133