I am working with some classmates to try to re-estimate some first-stage results from a recent paper on the effects of school funding reforms, except using the new Callaway and Sant'anna estimators to account for the staggered rollout (and possible heterogeneous effects) of these reforms. However, we are running into some trouble when trying to produce the estimates, and getting similar errors to this previous statalist post (in that we get the same "Panel is not balanced
Will use observations with Pair balanced (observed at t0 and t1)" warning message), but following the guidance there didn't seem to resolve our concerns. The concern is that many of the comparisons end up as 0s with a note that they are omitted, and we can figure out why. First, our sense is that we do not have a sparse number of observations for our comparisons, as each treated state in each period have many control states to be compared to, but perhaps our sense of the scale of what counts as very few observations is off. Second, when we use the method(reg) option we get more (but not all) estimates, but also understand that this may be a suboptimal way to do the estimation. I have put below the code we use (with a reference to the data file that should work for anyone with internet access), so it should run fully, so you can see the results we see. I'm also including the results as we see them.
HTML Code:
*Recreating and Extending School Finance Estimates clear all set more off set rmsg on *Import import delimited "https://raw.githubusercontent.com/clibassi/lrs_school_funding_replication/master/lrs_workingdata.csv", clear keep if (mainevent_t_slope_1p==copy | eventyr==.) xtset fipst fy replace eventyr = 0 if eventyr==. csdid q1q5_totrev_1990, time(fy) gvar(eventyr) notyet ivar(fipst) seed(48103) // method(reg)
HTML Code:
csdid q1q5_totrev_1990, time(fy) gvar(eventyr) notyet ivar(fipst) seed(48103) // method(reg)
Units always treated found. These will be excluded
Panel is not balanced
Will use observations with Pair balanced (observed at t0 and t1)
..................................................
..................................................
.....xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxx
Difference-in-difference with Multiple Time Periods
Number of obs = 921
Outcome model : weighted least squares
Treatment model: inverse probability tilting
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
g1992 |
t_1990_1992 | 1095.932 89.54615 12.24 0.000 920.4249 1271.439
t_1990_1993 | 1913.346 99.95304 19.14 0.000 1717.442 2109.25
t_1990_1994 | 76.75684 109.1579 0.70 0.482 -137.1888 290.7024
t_1990_1995 | 12.60415 112.1677 0.11 0.911 -207.2405 232.4488
t_1990_1996 | 403.4126 130.5466 3.09 0.002 147.5459 659.2793
t_1990_1997 | 580.2966 141.1035 4.11 0.000 303.7388 856.8544
t_1990_1998 | 692.7757 141.1157 4.91 0.000 416.1939 969.3575
t_1990_1999 | 279.1626 173.827 1.61 0.108 -61.53213 619.8573
t_1990_2000 | 271.0873 188.6188 1.44 0.151 -98.59876 640.7735
t_1990_2001 | 22.66297 191.2285 0.12 0.906 -352.1379 397.4639
t_1990_2002 | -149.3262 190.0864 -0.79 0.432 -521.8888 223.2364
t_1990_2003 | -92.152 163.7472 -0.56 0.574 -413.0906 228.7866
t_1990_2004 | 204.203 186.7843 1.09 0.274 -161.8876 570.2936
t_1990_2005 | -132.5554 164.4124 -0.81 0.420 -454.7978 189.6869
t_1990_2006 | 195.608 191.9067 1.02 0.308 -180.5223 571.7382
t_1990_2007 | 334.3379 197.7691 1.69 0.091 -53.28244 721.9581
t_1990_2008 | 709.842 218.7362 3.25 0.001 281.127 1138.557
t_1990_2009 | -3.237676 200.264 -0.02 0.987 -395.7479 389.2726
t_1990_2010 | 269.05 242.4948 1.11 0.267 -206.2311 744.3312
t_1990_2011 | 50.77995 218.9124 0.23 0.817 -378.2804 479.8403
t_1990_2012 | -387.431 206.0568 -1.88 0.060 -791.2948 16.43289
-------------+----------------------------------------------------------------
g1993 |
t_1990_1992 | -234.2644 232.0528 -1.01 0.313 -689.0796 220.5508
t_1992_1993 | -112.4388 165.6394 -0.68 0.497 -437.0861 212.2085
t_1992_1994 | -210.6332 298.5213 -0.71 0.480 -795.7241 374.4578
t_1992_1995 | 32.29282 424.6476 0.08 0.939 -800.0013 864.5869
t_1992_1996 | 619.3797 729.793 0.85 0.396 -810.9883 2049.748
t_1992_1997 | 544.1243 743.3894 0.73 0.464 -912.8921 2001.141
t_1992_1998 | 158.0276 555.0375 0.28 0.776 -929.8259 1245.881
t_1992_1999 | 46.61446 611.2941 0.08 0.939 -1151.5 1244.729
t_1992_2000 | 314.8592 633.1664 0.50 0.619 -926.1241 1555.842
t_1992_2001 | 232.3388 575.4518 0.40 0.686 -895.526 1360.204
t_1992_2002 | -581.6059 538.7071 -1.08 0.280 -1637.452 474.2407
t_1992_2003 | 312.7321 884.775 0.35 0.724 -1421.395 2046.859
t_1992_2004 | 376.9926 974.1248 0.39 0.699 -1532.257 2286.242
t_1992_2005 | -276.6181 759.4998 -0.36 0.716 -1765.21 1211.974
t_1992_2006 | 72.75047 920.5863 0.08 0.937 -1731.566 1877.066
t_1992_2007 | 509.4552 1217.457 0.42 0.676 -1876.716 2895.626
t_1992_2008 | -26.34066 982.0585 -0.03 0.979 -1951.14 1898.459
t_1992_2009 | -820.6729 1032.017 -0.80 0.426 -2843.389 1202.043
t_1992_2010 | -288.5642 906.824 -0.32 0.750 -2065.907 1488.778
t_1992_2011 | -791.9607 998.4819 -0.79 0.428 -2748.949 1165.028
t_1992_2012 | -513.7482 1428.941 -0.36 0.719 -3314.42 2286.924
-------------+----------------------------------------------------------------
g1995 |
t_1990_1992 | 131.1017 324.2906 0.40 0.686 -504.4962 766.6996
t_1992_1993 | 197.4969 279.5664 0.71 0.480 -350.4432 745.437
t_1993_1994 | 4.803792 138.4703 0.03 0.972 -266.5931 276.2007
t_1994_1995 | -40.47483 109.2284 -0.37 0.711 -254.5585 173.6088
t_1994_1996 | -277.0623 247.3954 -1.12 0.263 -761.9484 207.8238
t_1994_1997 | -474.2755 160.969 -2.95 0.003 -789.769 -158.7821
t_1994_1998 | -174.9257 106.9709 -1.64 0.102 -384.5848 34.73345
t_1994_1999 | -401.6927 146.4669 -2.74 0.006 -688.7626 -114.6228
t_1994_2000 | -522.2962 143.7425 -3.63 0.000 -804.0262 -240.5661
t_1994_2001 | -950.2494 144.5798 -6.57 0.000 -1233.621 -666.8781
t_1994_2002 | -863.2062 244.524 -3.53 0.000 -1342.464 -383.9479
t_1994_2003 | -794.9905 188.7538 -4.21 0.000 -1164.941 -425.0398
t_1994_2004 | -838.1472 214.8789 -3.90 0.000 -1259.302 -416.9923
t_1994_2005 | -799.1366 254.3504 -3.14 0.002 -1297.654 -300.6191
t_1994_2006 | -640.7223 326.1213 -1.96 0.049 -1279.908 -1.536218
t_1994_2007 | -591.5312 233.5838 -2.53 0.011 -1049.347 -133.7154
t_1994_2008 | -616.1381 211.635 -2.91 0.004 -1030.935 -201.3412
t_1994_2009 | -934.1367 176.2305 -5.30 0.000 -1279.542 -588.7314
t_1994_2010 | -668.8988 251.8563 -2.66 0.008 -1162.528 -175.2696
t_1994_2011 | -842.8265 417.3514 -2.02 0.043 -1660.82 -24.83282
t_1994_2012 | -1279.18 215.7476 -5.93 0.000 -1702.038 -856.3225
-------------+----------------------------------------------------------------
g1997 |
t_1990_1992 | -53.91153 211.1144 -0.26 0.798 -467.6881 359.8651
t_1992_1993 | -224.9099 381.0661 -0.59 0.555 -971.7857 521.966
t_1993_1994 | 404.1119 281.1195 1.44 0.151 -146.8721 955.096
t_1994_1995 | -95.15187 114.3933 -0.83 0.406 -319.3587 129.055
t_1995_1996 | -208.0472 234.1382 -0.89 0.374 -666.9497 250.8553
t_1996_1997 | 392.4204 113.4184 3.46 0.001 170.1243 614.7164
t_1996_1998 | 568.2612 341.8933 1.66 0.096 -101.8373 1238.36
t_1996_1999 | 420.7706 161.4674 2.61 0.009 104.3003 737.241
t_1996_2000 | 1057.912 253.8138 4.17 0.000 560.4463 1555.378
t_1996_2001 | 1180.775 691.4492 1.71 0.088 -174.4404 2535.991
t_1996_2002 | 1242.031 737.2518 1.68 0.092 -202.9564 2687.018
t_1996_2003 | 378.3748 482.3617 0.78 0.433 -567.0367 1323.786
t_1996_2004 | 803.6591 472.9957 1.70 0.089 -123.3955 1730.714
t_1996_2005 | 527.8136 128.4182 4.11 0.000 276.1185 779.5087
t_1996_2006 | 1084.65 269.7417 4.02 0.000 555.9663 1613.334
t_1996_2007 | 1012.402 682.9889 1.48 0.138 -326.2313 2351.036
t_1996_2008 | 1590.829 273.8831 5.81 0.000 1054.028 2127.63
t_1996_2009 | -249.5455 1187.342 -0.21 0.834 -2576.693 2077.602
t_1996_2010 | -1544.866 2022.297 -0.76 0.445 -5508.496 2418.763
t_1996_2011 | 1498.779 239.4337 6.26 0.000 1029.497 1968.06
t_1996_2012 | 1175.659 306.7882 3.83 0.000 574.3649 1776.952
-------------+----------------------------------------------------------------
g1998 |
t_1990_1992 | 76.33176 242.4246 0.31 0.753 -398.8118 551.4753
t_1992_1993 | 666.2495 296.2277 2.25 0.025 85.65381 1246.845
t_1993_1994 | 678.7922 453.7682 1.50 0.135 -210.5772 1568.162
t_1994_1995 | -674.6014 376.7498 -1.79 0.073 -1413.017 63.81455
t_1995_1996 | -231.4732 145.0732 -1.60 0.111 -515.8114 52.86506
t_1996_1997 | 234.3133 79.30412 2.95 0.003 78.88007 389.7465
t_1997_1998 | 638.4182 374.6897 1.70 0.088 -95.96008 1372.796
t_1997_1999 | 961.7929 469.4229 2.05 0.040 41.741 1881.845
t_1997_2000 | 1331.461 867.428 1.53 0.125 -368.6671 3031.588
t_1997_2001 | 956.8368 1035.84 0.92 0.356 -1073.372 2987.045
t_1997_2002 | 1079.39 1015.736 1.06 0.288 -911.4156 3070.195
t_1997_2003 | 918.0218 892.1782 1.03 0.303 -830.6154 2666.659
t_1997_2004 | 1586.924 1473.879 1.08 0.282 -1301.826 4475.674
t_1997_2005 | 1714.025 1047.304 1.64 0.102 -338.6536 3766.704
t_1997_2006 | 2439.023 1063.503 2.29 0.022 354.5947 4523.451
t_1997_2007 | 2143.754 1069.571 2.00 0.045 47.4337 4240.075
t_1997_2008 | 2131.362 1198.191 1.78 0.075 -217.0492 4479.772
t_1997_2009 | 2433.357 1425.325 1.71 0.088 -360.2297 5226.943
t_1997_2010 | 1689.417 1194.237 1.41 0.157 -651.245 4030.08
t_1997_2011 | 387.4205 1014.033 0.38 0.702 -1600.048 2374.889
t_1997_2012 | 945.4962 1584.817 0.60 0.551 -2160.689 4051.681
-------------+----------------------------------------------------------------
g1999 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1998_2000 | 0 (omitted)
t_1998_2001 | 0 (omitted)
t_1998_2002 | 0 (omitted)
t_1998_2003 | 0 (omitted)
t_1998_2004 | 0 (omitted)
t_1998_2005 | 0 (omitted)
t_1998_2006 | 0 (omitted)
t_1998_2007 | 0 (omitted)
t_1998_2008 | 0 (omitted)
t_1998_2009 | 0 (omitted)
t_1998_2010 | 0 (omitted)
t_1998_2011 | 0 (omitted)
t_1998_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2000 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_1999_2001 | 0 (omitted)
t_1999_2002 | 0 (omitted)
t_1999_2003 | 0 (omitted)
t_1999_2004 | 0 (omitted)
t_1999_2005 | 0 (omitted)
t_1999_2006 | 0 (omitted)
t_1999_2007 | 0 (omitted)
t_1999_2008 | 0 (omitted)
t_1999_2009 | 0 (omitted)
t_1999_2010 | 0 (omitted)
t_1999_2011 | 0 (omitted)
t_1999_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2001 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2000_2002 | 0 (omitted)
t_2000_2003 | 0 (omitted)
t_2000_2004 | 0 (omitted)
t_2000_2005 | 0 (omitted)
t_2000_2006 | 0 (omitted)
t_2000_2007 | 0 (omitted)
t_2000_2008 | 0 (omitted)
t_2000_2009 | 0 (omitted)
t_2000_2010 | 0 (omitted)
t_2000_2011 | 0 (omitted)
t_2000_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2002 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2001_2003 | 0 (omitted)
t_2001_2004 | 0 (omitted)
t_2001_2005 | 0 (omitted)
t_2001_2006 | 0 (omitted)
t_2001_2007 | 0 (omitted)
t_2001_2008 | 0 (omitted)
t_2001_2009 | 0 (omitted)
t_2001_2010 | 0 (omitted)
t_2001_2011 | 0 (omitted)
t_2001_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2003 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2002_2004 | 0 (omitted)
t_2002_2005 | 0 (omitted)
t_2002_2006 | 0 (omitted)
t_2002_2007 | 0 (omitted)
t_2002_2008 | 0 (omitted)
t_2002_2009 | 0 (omitted)
t_2002_2010 | 0 (omitted)
t_2002_2011 | 0 (omitted)
t_2002_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2004 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2003_2005 | 0 (omitted)
t_2003_2006 | 0 (omitted)
t_2003_2007 | 0 (omitted)
t_2003_2008 | 0 (omitted)
t_2003_2009 | 0 (omitted)
t_2003_2010 | 0 (omitted)
t_2003_2011 | 0 (omitted)
t_2003_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2005 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2004_2006 | 0 (omitted)
t_2004_2007 | 0 (omitted)
t_2004_2008 | 0 (omitted)
t_2004_2009 | 0 (omitted)
t_2004_2010 | 0 (omitted)
t_2004_2011 | 0 (omitted)
t_2004_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2006 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2005_2006 | 0 (omitted)
t_2005_2007 | 0 (omitted)
t_2005_2008 | 0 (omitted)
t_2005_2009 | 0 (omitted)
t_2005_2010 | 0 (omitted)
t_2005_2011 | 0 (omitted)
t_2005_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2007 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2005_2006 | 0 (omitted)
t_2006_2007 | 0 (omitted)
t_2006_2008 | 0 (omitted)
t_2006_2009 | 0 (omitted)
t_2006_2010 | 0 (omitted)
t_2006_2011 | 0 (omitted)
t_2006_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2008 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2005_2006 | 0 (omitted)
t_2006_2007 | 0 (omitted)
t_2007_2008 | 0 (omitted)
t_2007_2009 | 0 (omitted)
t_2007_2010 | 0 (omitted)
t_2007_2011 | 0 (omitted)
t_2007_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2010 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2005_2006 | 0 (omitted)
t_2006_2007 | 0 (omitted)
t_2007_2008 | 0 (omitted)
t_2008_2009 | 0 (omitted)
t_2009_2010 | 0 (omitted)
t_2009_2011 | 0 (omitted)
t_2009_2012 | 0 (omitted)
-------------+----------------------------------------------------------------
g2011 |
t_1990_1992 | 0 (omitted)
t_1992_1993 | 0 (omitted)
t_1993_1994 | 0 (omitted)
t_1994_1995 | 0 (omitted)
t_1995_1996 | 0 (omitted)
t_1996_1997 | 0 (omitted)
t_1997_1998 | 0 (omitted)
t_1998_1999 | 0 (omitted)
t_1999_2000 | 0 (omitted)
t_2000_2001 | 0 (omitted)
t_2001_2002 | 0 (omitted)
t_2002_2003 | 0 (omitted)
t_2003_2004 | 0 (omitted)
t_2004_2005 | 0 (omitted)
t_2005_2006 | 0 (omitted)
t_2006_2007 | 0 (omitted)
t_2007_2008 | 0 (omitted)
t_2008_2009 | 0 (omitted)
t_2009_2010 | 0 (omitted)
t_2010_2011 | 0 (omitted)
t_2010_2012 | 0 (omitted)
------------------------------------------------------------------------------
Control: Not yet Treated
0 Response to Warning when Running CSDID and Many Estimates Not Computing and Hard to Understand Why
Post a Comment