Good afternoon,

Suppose that my task is to obtain the residual vector in a multiple regression of Y on X and Z. Let's call this residual vector R. X and Z are two groups of variables.

With experimentation where X and Z contain each only one variable, it seems to me that the following algorithm achieves the task:

1. Regress Y on X, predict the residual vector R1, say.

2. Regress R1 on Z, predict the residual vector R2.

3. Regress R2 on X, predict the residual vector R3.

4. Regress R3 on Z, predict the residual vector R4

... repeat the procedure until the Residual Sum of Squares does not change between this and next iteration.

I did this manually with the auto data where Y is price, X is mpg and Z is headroom.

With each next iteration the residual was getting closer and closer to the residual from the joint regression of price on mpg and headroom.

My questions are:

a) Is it just a coincidence that I obtained convergence, or such an algorithm exists?

b) What is the name of this algorithm?