Hi. I am trying to run a DiD regression to analyze the effect of education policy implementation on avg. statewide test scores. My treatment is defined by a 1 to 5 index (based on how similar a state's prior education standards were to the new education policy implemented). So, on one end, I hypothesized that the states with very similar prior standards (1 on index) will have little/no treatment effect and on the other, I expect states with very different prior standards (5 on index) to have the most pronounced change in test scores. Rather than looking at implementers vs non-implementers of the policy, I look at only the implementer states and then have identification based on this 1 to 5 index -- I want to see if the treatment effect varies in magnitude based on the degree of similarity between the new policy and a state's prior standards.)

I have several questions about the set up of my DiD:

reg avg_scale_score i.time_period_implement i.state i.similar##i.post_implement

where:
avg_scale_score is the outcome of interest (test score on scale of 0 to 500 points)
i.time_period_implement is time fixed effects
i.state is state fixed effects
i.similar##i.post_implement is my DiD term, where similar my "treatment" is on the 1 to 5 scale and post_implement 0 or 1 depending on if the observation is in years before or years during/after implementation.

(1) I had to rescale my time from years to a "two year period" scale because states implemented the policy in different years (2012 thru 2015), but my outcome of interest (the test) is only administered every other year. So, I had only even yeared observations for some states (+6, +4, +2, 0, -2, -4, -6... years relative to implementation) and only odd yeared observations (+5,+3,+1,-1,-3,-5...) for other states. I thought time-fixed effects with this rescaled time period would account for structural differences between states in the even yeared and odd yeared group. I did this because I thought this would allow me to include both sets of states in my regression. Does this approach make sense? If not, how do I account for even vs. odd states?

(2) I assumed that post_implement would be dropped due to collinearity with my time fixed effects, but it still appears in my output. (I don't think it's a problem with how I defined time_period_implement, either, becasue the same thing happens when include i.year instead of i.time_period_implement.) Can anyone provide intuition or solution?

(3) My DiD term coefficients are statistically significant when I regress in this way, with state and time FEs. However, when I regress with the traditional DiD (that is, reg avg_scale_score i.similar i.post_implement i.similar#i.post_implement) the DiD term coefficients lose all statistical significance. Why is this? How should I interpret results taking this into consideration?

(4) I have never done Diff-in-Diff in which the treatment is not binary. My professor suggested I use the 1 to 5 approach because having more variation will buy me more identification. But, I'm not exactly sure how to interpret my DiD coefficients:

similar#post_implement
Coef. Std. Err. t P>|t| [95% Conf. Interval]
2 x 1 | -2.334968 .9005698 -2.59 0.010 -4.106284 -.5636516
3 x 1 | -2.23384 .7814284 -2.86 0.005 -3.770819 -.6968609
4 x 1 | -3.318883 .835981 -3.97 0.000 -4.96316 -1.674605
5 x 1 | -1.601172 .8263082 -1.94 0.053 -3.226424 .0240803

Does this mean impact of policy is a 2.3 reduction in avg score for states with similarity index 2, 2.2 reduction for states with similarity index 3, 3.3 reduction for states with similarity 4, and 1.6 point reduction for states with similarity index of 5, relative to states with a similarity index of 1??

Any help would be greatly appreciated. I want to make sure my approach is as sound as possible.