Hi,

I've run an experimental survey (this is social science), with two treatment variables, using logistic regression to analyse the results. One of the treatment variables is rather consistently significant across all dependent variables (yes, I have many).

However, when running sensitivity and specificity analysis (estat classification) the number for specificity is usually close to zero or even actually zero.

To be clear, r-squares are incredibly bad which I expected since there is a lot more that affects the dependent variable than just whether or not you were in either treatment group. My goal with this study was not so much to explain everything about what affects the dependent variable, but rather to see whether either treatment affected the dependent variable. I've run the regressions with and without control variables, sensitivity and specificity do not change much.

In this context, how bad is the low specificity since I'm really just trying to identify whether there's a treatment effect? What are the implications? I'm asking because I have to comment on these results.