Well it happens, you will have to convince yourself that people also need commitment.
And you should be there for them as a support system.
Otherwise if you keep drifting away you will have no one in the long run.
Try to give equal amount of effort all throughout.