Good evening,
I am working on my dissertation and have come to a wall. My dissertation is on the efficacy of postsecondary remediation for incoming freshman at North Dakota institutions. The following are my summary statistics.
summarize Math103Letter mathAct
Variable | Obs Mean Std. Dev. Min Max
-------------+---------------------------------------------------------
Math103Let~r | 6,793 3.98734 1.140518 1 5
mathAct | 6,793 23.66524 3.645204 13 36
I then did a histogram of my running variable.
Array
The cutoff for remediation in North Dakota is a 22, those above a 22 are good, those below 22 will have to take remediation. With the histogram I notice that it is not continuous - there is a large decrease in scores at 21. My opinion is that students who get a 21 will retest to try to do better and then they get a 22 or higher which puts them out of remediation. I do not have data on if they retested or how many times they retested. I think because of that I need to utilize a fuzzy design? or does that fact make RD not a path I want to take?
After the histogram, I continued on the path and completed an RD Manipulation test
rddensity mathAct, c(22)
Computing data-driven bandwidth selectors.
RD Manipulation Test using local polynomial density estimation.
Cutoff c = 22 | Left of c Right of c Number of obs = 6793
-------------------+---------------------- Model = unrestricted
Number of obs | 1227 5566 BW method = comb
Eff. Number of obs | 15 1598 Kernel = triangular
Order est. (p) | 2 2 VCE method = jackknife
Order bias (q) | 3 3
BW est. (h) | 1.034 1.172
Running variable: mathAct.
-----------------------------------------------
Method | T P>|T|
-------------------+---------------------------
Robust | 87.0309 0.0000
-----------------------------------------------
This also gave the warning: bandwidth h may be too low. I am not sure what I would adjust the bandwidth to, any help here would be great.
After that, I did a rdplot
rdplot Math103Letter mathAct, c(22)
RD Plot with evenly spaced mimicking variance number of bins using spacings estimators.
Cutoff c = 22 | Left of c Right of c Number of obs = 6793
----------------------+---------------------- Kernel = Uniform
Number of obs | 1227 5566
Eff. Number of obs | 1227 5566
Order poly. fit (p) | 4 4
BW poly. fit (h) | 9.000 14.000
Number of bins scale | 1.000 1.000
Outcome: Math103Letter. Running variable: mathAct.
---------------------------------------------
| Left of c Right of c
----------------------+----------------------
Bins selected | 73 391
Average bin length | 0.123 0.036
Median bin length | 0.123 0.036
----------------------+----------------------
IMSE-optimal bins | 6 25
Mimicking Var. bins | 73 391
----------------------+----------------------
Rel. to IMSE-optimal: |
Implied scale | 12.167 15.640
WIMSE var. weight | 0.001 0.000
WIMSE bias weight | 0.999 1.000
---------------------------------------------
Array
As you can see there shows a discontinuity in the data.
Then I conducted a rdbwselect to try to find the optimal bandwidth. This could not be completed, I receive the error below:
rdbwselect Math103Letter mathAct, c(22) all
Invertibility problem in the computation of preliminary bandwidth below the thresholdInvertibility problem in the computation of preliminary bandwidth ab
> ove the thresholdInvertibility problem in the computation of bias bandwidth (b) below the thresholdInvertibility problem in the computation of bias ban
> dwidth (b) above the thresholdInvertibility problem in the computation of loc. poly. bandwidth (h) below the thresholdInvertibility problem in the comp
> utation of loc. poly. bandwidth (h) above the threshold
Bandwidth estimators for sharp RD local polynomial regression.
Cutoff c = 22 | Left of c Right of c Number of obs = 6793
-------------------+---------------------- Kernel = Triangular
Number of obs | 1989 4804 VCE method = NN
Min of mathAct | 13.000 23.000
Max of mathAct | 22.000 36.000
Order est. (p) | 1 1
Order bias (q) | 2 2
Outcome: Math103Letter. Running variable: mathAct.
--------------------------------------------------------------------------------
| BW est. (h) | BW bias (b)
Method | Left of c Right of c | Left of c Right of c
-------------------+------------------------------+-----------------------------
mserd | . . | . .
msetwo | . . | . .
msesum | . . | . .
msecomb1 | . . | . .
msecomb2 | . . | . .
-------------------+------------------------------+-----------------------------
cerrd | . . | . .
certwo | . . | . .
cersum | . . | . .
cercomb1 | . . | . .
cercomb2 | . . | . .
--------------------------------------------------------------------------------
I did some reading and saw that the error could come because the running variable is not continuous at the cutoff? Any help here would be great as well. I did try the rdbwselect as a fuzzy RD and I received the same error.
I continued just to see what would happen. I did the rdrobust sharp design and I utilized bandwidth of 9 since that was what it showed in the RD plot, If I did not put a value for h the program did not run. Here are the results:
rdrobust Math103Letter mathAct, c(22) h(9)
Sharp RD estimates using local polynomial regression.
Cutoff c = 22 | Left of c Right of c Number of obs = 6793
-------------------+---------------------- BW type = Manual
Number of obs | 1227 5566 Kernel = Triangular
Eff. Number of obs | 1223 5416 VCE method = NN
Order est. (p) | 1 1
Order bias (q) | 2 2
BW est. (h) | 9.000 9.000
BW bias (b) | 9.000 9.000
rho (h/b) | 1.000 1.000
Outcome: Math103Letter. Running variable: mathAct.
--------------------------------------------------------------------------------
Method | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------+------------------------------------------------------------
Conventional | .07855 .1176 0.6679 0.504 -.151937 .309032
Robust | - - 1.0388 0.299 -.248302 .808343
--------------------------------------------------------------------------------
According to these results, I could not reject the null hypothesis which would be that remediation does NOT have an effect on the next credit-bearing course, in this case, College Algebra. So to me, that would indicate that math remediation does not have an effect on the grade a student receives in college algebra? Am I on the right track, can I use the bandwidth of 9 or should it be something different? Any help would be appreciated.
Thank you,
Russ Ziegler
Related Posts with Struggling EdD student trying to make sense of my RD design.
Trouble with metadta for diagnostic test accuracy systematic reviewHello, im relatively new to stata so please forgive me if there is something obvious im missing. Im …
How to Construct Daily Food Consumption ScoreDear Statalisters, I am working with a dataset that contains information on household consumption of…
Append regression results horizontallyHi, I am a new STATA user and found this forum has been incredibly helpful for me. I can't seem to …
Mothers literacyHello, I wish to generate mother's matriculation literacy dummy (edu_matri) and illterate dummy (ill…
IV RegressionHello everyone. May I please have some assistance on what command I should use for my IV regression?…
Subscribe to:
Post Comments (Atom)
0 Response to Struggling EdD student trying to make sense of my RD design.
Post a Comment