I'd say the penalty is requiring an engineer to peel off from whatever else they're doing (if there's a gunner) or dismounting their own gun (if they're a gungineer) to stand hitting the gun every 20-30 seconds for 10 seconds (don't know the exact figures) is penalty enough. More over, they've sacrificed some rebuilding/repairing power or the ability to extinguish to do this as well. I think there's plenty of penalities there. Sure you can prep a lot of components before engagements and preload ammo but during a fight all these things add up to make a tool that's pretty specific and fraught with danger if you mess your priorities up.
Edit: And yeah, I know you said 'to the gun' but it's all connected.
Umm, if you were a gunner you wouldn't be able to do all of that either, the concept of the Buffgineer on a gun is that he is pretty much a Gunner, and again, he still has more utility when it comes to repairs even if you decided to give away extinguishing power or some repair power (and you still get the buff and extinguisher, while the gunner would only have the wrench.)
You don't waste much time hitting the gun if you do it right. This thread seems pointless to me now though, since the new patch I see gunners, despite the advantages a Buff-gungineer has, as much more useful and practically essential for multi-range ships and many more guns benefit much more from multiple ammo types than just having a gungineer, the flexibility of a gunner is realy starting to shine a lot in contrast to the specialization of the Buff-gungineer.