As a retired garage owner, I was interested to read about driveline noise in a Subaru Impreza. I have a 5-year-old Subaru Forrester that began to make a strange intermittent noise about two years ago. None of my friends still in the business have been able to pinpoint it and as we use this car only for long trips with the noise making no difference at all except it's getting progressively louder when cold and you can now hear it at low speed we haven't been too concerned about it. I have isolated it to the right-rear driveshaft and we may take the shaft out and have a look as the bloke who bought my business thinks he has heard a Ford Fiesta with the same noise. The shaft looks normal even when the boots are pulled back.
Noises like this can be hard to trace, but you're obviously on the right track and will ultimately establish the cause. Taking the shaft out seems like a good course of action having worked out that it is possibly at the heart of the problem. Good luck.