Considering a Difference-in-Differences equation with a dummy variable:

yi=α+βDi+Xit+εi.

While Xit is a set of covariates. Today when I add more covariates, the p-value decrease. Is it normal? If it is normal, what does it mean then?

And if it is a problem, what should I deal with that?

Many thanks in advance.