It is one of the most annoying match changers that can happen, especially since in the heat of battle you don't constantly watch who is joining and leaving. Yesterday we were one short for a full crew and we lost the moment the AI engineer was replaced ALWAYS. Unless another engineer joined, which happened very rarely. This is especially frustrating when you have people join as gunners and then just idle around unless something gets in their sights.
We had a fun situation where the captain had to leave the wheel because the gunner was too fine to do some legwork. Or... anything... We were streaming at the time and even the viewers - none of which had played GoIO yet - filled the chat with very active suggestions as to what to do with that guy whose name shall not be revealed.
Given, he was an extreme case. But 1 gunner, 2 engineers, 1 Captain is usually the best composition for a crew.
I think possible solutions would be
...for the replacement to only be able to replace the function the AI is currently handling
...for the captain to put the desired role(s) before the battle starts (Who knows? Incontinent captain might need replacement?
...to be able to lock the AI in place and deactivate the option to join. It should probably make at least 3 crew members mandatory. The option's goal would not be to never meet new players and have 1 Captain ships everywhere. But to save the crew from unpleasant surprises