理亏心虚

lǐkuī-xīnxū

[feel apprehensive because one is not on solid ground; be not on solid ground; have a guility conscience because justice is not on one’s side] 因理由不充分而担心