I also break my rifles down to clean them every time. I just feel better about it and I want to clean that trigger too. Maybe I am lucky but I have not had a problem with my point of impact changing much.
I just use a damp rag to wipe my laminate stocks off.