Small Area Co-Modelng of Pont Estmates and Ther Varances for Domans n the Current Employment Statstcs Survey Jule Gershunskaya, Terrance D. Savtsky U.S. Bureau of Labor Statstcs FCSM, March 2018 Dsclamer: Any opnons expressed n ths paper are those of the authors and do not consttute polcy of the Bureau of Labor Statstcs 1 U.S. BUREAU OF LABOR STATISTICS bls.gov
2 U.S. BUREAU OF LABOR STATISTICS bls.gov Introducton CES publshes monthly estmates of employment at detaled levels by ndustry and geography In small samples, estmates can be unstable: SAE methods are used currently n use: Fay-Herrot (FH) model at supersector ndustral level for select States. Goal: extend the use of FH to detaled sub-ndustres and MSAs Concerns: - possble exstence of clusterng of employment growth by sub-ndustres and geographc areas - nstablty of estmated samplng varances, especally n small samples: (co-modelng consdered by Mat et al. 2014, Sugasawa et al. 2017)
M1: Fay Herrot (FH) For a gven month, for domans = 1,..., N: q -- over-the-month change n employment level n doman q y -- drect sample-based estmate of v -- varance of sample-based estmate y (assumed known) x -- covarate (hstorcal change, from admnstratve data source: Quarterly Census of Employment and Wages (QCEW) ) 3 U.S. BUREAU OF LABOR STATISTICS bls.gov
M1: Fay Herrot (FH) For a gven month, for domans = 1,..., N: What may go wrong? In samplng model: - v s assumed known (yet estmate or a proxy s used) - normalty may not hold: possble outlers In lnkng model: - lnearty assumpton m+ bx may be volated - normalty may not hold: e.g. exstence of clusters 4 U.S. BUREAU OF LABOR STATISTICS bls.gov
M2: Fay Herrot wth clusters n lnkng model (CFH) Assume a mxture of normal dstrbutons for the lnkng model: Prors: æa a ö pk ~ Drchlet,..., ç èk K ø, 1 k N m m ~ ( 0, l - ), ~ N ( 0, ) 1 b l - 2 ; al,, l, t - ~ G( 1,1) b m b 5 U.S. BUREAU OF LABOR STATISTICS bls.gov
M3: Co modelng: pont estmates and varances (FHS) 2 (, ) q s -- target parameters for doman (, ) y v -- drect sample-based estmates for doman 6 U.S. BUREAU OF LABOR STATISTICS bls.gov
M4: Co modelng and clusterng (CFHS) 2 (, ) q s -- target parameters for doman (, ) y v -- drect sample-based estmates for doman 7 U.S. BUREAU OF LABOR STATISTICS bls.gov
M5: Co modelng and co clusterng (CFHSc) 2 (, ) q s -- target parameters for doman (, ) y v -- drect sample-based estmates for doman 8 U.S. BUREAU OF LABOR STATISTICS bls.gov
the cheat sheet Level FH CFH FHS CFHS CFHSc Pont Estmate Samplng Varance Samplng model Lnkng model Samplng model Lnkng model normal normal normal normal normal normal normal mxture normal normal mxture normal mxture gamma gamma gamma gamma gamma gamma mxture 9 U.S. BUREAU OF LABOR STATISTICS bls.gov
Models comparson based on CES data Domans: Statewde/detaled ndustry Drect estmator of over-the-month relatve change: Ft models for each of 12 months, obtan ftted values: Derve level estmate as: Due to dfferent seasonalty patterns between QCEW (admnstratve source) and CES, the most mportant comparson s after 12 months of estmaton: MAD N 1 N Y,12 Y,12 10 U.S. BUREAU OF LABOR STATISTICS bls.gov 1
Real Data Results (after 12 months), Sep 2008-Sep 2009 estmaton cycle Ind N Drect FH BRR CFH BRR FH V CFH V FHS CFHS CFHSc 1000 50 792 774 757 789 784 840 777 817 2000 141 2152 1782 1783 1801 1825 1770 1760 1779 3100 234 1112 1072 1058 968 970 1075 1053 1081 3200 140 955 918 894 862 844 949 898 960 4100 124 1485 1234 1230 1215 1212 1201 1164 1179 4200 286 1439 1364 1365 1366 1340 1321 1305 1311 4300 194 1310 1023 1022 1044 1045 903 928 929 5000 83 1204 768 781 718 750 728 711 736 5500 149 1473 1031 1041 1047 1072 1051 1042 1060 6054 150 1450 1185 1193 1152 1177 1160 1145 1165 6055 45 1066 992 1005 917 950 892 975 937 6056 115 2344 1862 1876 2001 2040 2034 1853 1955 6561 59 1901 1779 1768 1538 1581 1537 1614 1593 6562 214 1551 1226 1244 1242 1247 1178 1225 1192 7071 59 2047 1431 1421 1208 1352 1136 1243 257 7072 80 1912 1819 1708 1641 1665 1736 1689 1733 8000 110 1773 1211 1250 1175 1219 1097 1186 1128 Overall 2233 1502 1252 1250 1224 1236 1215 1207 1221 11 U.S. BUREAU OF LABOR STATISTICS bls.gov 1 N 1,12,12 MAD N Y Y
Real Data Results (after 12 months), Sep 2009-Sep 2010 estmaton cycle Ind N Drect FH BRR CFH BRR FH V CFH V FHS CFHS CFHSc 1000 50 863 584 624 642 678 572 627 606 2000 141 1940 1356 1376 1361 1410 1268 1327 1269 3100 235 1357 766 771 829 829 659 654 631 3200 140 804 653 654 629 637 585 583 584 4100 124 1207 806 841 849 876 780 831 786 4200 286 1164 935 953 929 939 870 880 875 4300 195 1287 1202 1182 986 1000 915 945 912 5000 83 1011 919 913 865 839 759 774 784 5500 149 1052 773 782 731 769 715 729 700 6054 150 1516 1130 1129 1115 1110 1071 1111 1071 6055 45 1119 1147 1154 1056 1079 1120 1090 1112 6056 115 2549 2532 2455 2419 2360 2567 2426 2558 6561 59 1966 1833 1807 1704 1696 1649 1701 1681 6562 215 1364 1069 1076 1024 1037 1026 1036 1040 7071 59 1987 1422 1410 1401 1464 1274 1364 1300 7072 80 2307 1768 1810 1665 1769 1582 1681 1702 8000 110 1628 938 981 1051 1081 887 982 895 Overall 2236 1424 1102 1107 1072 1087 1011 1032 1017 12 U.S. BUREAU OF LABOR STATISTICS bls.gov 1 N 1,12,12 MAD N Y Y
Real Data Results (after 12 months), Sep 2010-Sep 2011 estmaton cycle Ind N Drect FH BRR CFH BRR FH V CFH V FHS CFHS CFHSc 1000 50 908 759 782 810 827 683 692 649 2000 141 2015 1412 1405 1473 1485 1355 1395 1361 3100 235 967 777 786 757 768 674 670 643 3200 141 918 673 680 674 696 616 609 567 4100 124 1338 818 828 869 897 761 792 749 4200 286 1151 872 875 842 860 816 824 822 4300 196 977 817 798 764 766 737 723 727 5000 83 928 608 627 632 649 593 580 592 5500 149 1192 776 806 751 787 694 736 710 6054 150 1240 838 838 788 823 797 823 799 6055 45 1369 1411 1444 1337 1352 1377 1419 1410 6056 115 2708 2231 2294 2080 2113 2210 2209 2261 6561 59 2259 1972 1987 1932 1950 1705 1697 1770 6562 215 1345 1039 1044 967 1009 955 969 962 7071 59 1674 1110 1121 1130 1137 811 869 836 7072 80 2203 1601 1689 1780 1877 1664 1815 1625 8000 110 1636 997 1041 996 1053 955 1024 956 Overall 2238 1369 1019 1033 1002 1028 949 968 947 13 U.S. BUREAU OF LABOR STATISTICS bls.gov 1 N 1,12,12 MAD N Y Y
Estmated vs true varances of drect estmator (nd 6562, month #1) (sze of ponts s proportonal to y - q v ) BRR FHS CFHS CFHSc ( ) 2 T 2 z E és... ù γ ë û = w1 y - q + w 2v + w 3be, w depends on an. weght 2 14 U.S. BUREAU OF LABOR STATISTICS bls.gov
Model based screenng It s a good practce to perform careful model checks However, thorough model evaluaton may be unrealstc n a tghtly scheduled CES producton envronment The usual strategy: canddate models 1,..., W are thoroughly tested on a number of hstorcal seres over several years; after that one of them, model, s accepted for producton w What f w works well n general but fals for some domans n some months? 15 U.S. BUREAU OF LABOR STATISTICS bls.gov
Model based screenng Use Bayesan posteror predctve dstrbuton p( y y, w) Compute p = Pr { y generated by } Declare dscoveres : domans wth small w p Analysts have to revew the lst and determne f extreme y s due to (1) defcency of the sample (e.g., nfluental sample unts, small sample, low response, etc) (2) falure of the model lnearty assumpton (analysts may have addtonal out-of-sample nformaton to support ther judgement) 16 U.S. BUREAU OF LABOR STATISTICS bls.gov
17 U.S. BUREAU OF LABOR STATISTICS bls.gov
Summary We appled jont modelng of the pont estmates and varances to CES data. Jont modelng resulted n lower MAD than models wth plugged n fxed and known varances Ftted values for samplng varance are a useful by-product of the jont modelng We extended the models of Mat et al. (2014) and Sugasawa et al. (2017) by allowng the data to estmate a clusterng structure on estmated means and varances. Co-clusterng model provdes better estmates of samplng varances when there s a lot of varablty n varances Devsed an automated, fast computng testng procedure based on the Bayesan FDR to nomnate a small subset of domans for analyst revew on a tmely bass. Ths procedure could become a useful tool to mark unusual estmates before they are publshed 18 U.S. BUREAU OF LABOR STATISTICS bls.gov
Contact Informaton Jule Gershunskaya Mathematcal Statstcan SMS/OEUS 202-691-6360 Gershunskaya.Jule@bls.gov 19 U.S. BUREAU OF LABOR STATISTICS bls.gov