This is a fairly well known bug, and it can be quite annoying. There's no way around it, you just have to deal with this unwanted copilot. It's generally considered to be the ghost of another captain in the match you're in. They should go away after the match, though.