When you say you cannot do something, this is all you have to say. If you offer an explanation of what other thing you are doing, then you are creating an opportunity for the other person to argue for whether or not that other thing is more or less important than the thing that you’re doing, and there is never a situation where you want to have that conversation.
That said, sometimes the person you’re talking with a jerk will press. In that situation, you may explain that you had a “prior commitment”, and if pressed further, you may simply say “it’s a personal matter”. At this point, the other person has pushed well beyond the bounds of politeness, and you are entitled to end the conversation. However, since the other person may be someone with some power over you and can apply pressure that you do this, then the correct closer (at any point) is to propose an alternate time and move the onus back on them.
Never Not Learning 