Turns out there are some plastic clips that hold the windows in place. Over time they can snap and leave the window out of position when all the way down.
Broken clip:

New clip:

Beats me why such an integral part would be made from cheap plastic but hey ho. Part cost £1.50 and did the job.
Window now closes and no more rattles when closing the door.
Thought I would share just in case anyone else has this issue, worth checking before doing the whole regulator.
Edit: apologies for lack of part number, will look it up.