Hi Statalisters,

I have some questions about the bonferroni correction and False Discovery Rates.

What do I do: I have 4 different dependent variables, on which I run 4 seperate regressions. So it looks somewhat like this:

Code:
xtreg dependentvariable1 control1 control2 control3 control4, fe vce(cluster state)
xtreg dependentvariable1 control1 control2 control3 control4 control2*control3, fe vce(cluster state)
xtreg dependentvariable1 control1 control2 control3 control4 control1*control2*control3, fe vce(cluster state)
xtreg dependentvariable1 control1 control2 control3 control4 control1*control2*control3*control4, fe vce(cluster state)

xtreg dependentvariable2 control1 control2 control3 control4, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3*control4, fe vce(cluster state)

xtreg dependentvariable2 control1 control2 control3 control4, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3*control4, fe vce(cluster state)

xtreg dependentvariable2 control1 control2 control3 control4, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3, fe vce(cluster state)
xtreg dependentvariable2 control1 control2 control3 control4 control1*control2*control3*control4, fe vce(cluster state)
I now think that I have to adjust my p-values in order to address the inflated probability of commiting a Type 1 error. I consider 2 approaches:

1. The Bonferroni correction. Here, for every individual regression, I multiply all p-values by 4, as I am carrying out 4 regressions per dependent variable.
That is, for example I run the first regression
Code:
xtreg dependentvariable1 control1 control2 control3 control4, fe vce(cluster state)
and then multiply the individual p-values on control 1, control2 and so forth by 4. I do this for all 16 regressions, and then argue that I show statistical significance under Bonferroni-corrected p-values. Is this appraoch correct?

2. I consider employing the False Discovery Rate by Benjamini and Hochberg (1995). However, I am unsure how to exactly do this in my specific case.
I know that you are meant to order your p-values, divide them by the number of tests and then multiply with a specified rate.
However, which p-values do I order? Do I order together all individuals p-values from every set of 4 regression?
As an example, for dependentvariable1 that would be: 20 independent p-values, then order them and then calculate (i/20)*Rate, and then compare with p-values.
Is this the correct way of applying this method in this case?

I hope you are able to help me and thank you a lot in advance.

Kind regards,
Andreas