As per the controls, shifting
ballast or
ballonets would easily do this. An airplane uses control surfaces to do this. An airship (or naval ship for that matter) just has to shift its weight.
While you are using this tool, your engines shut off, so you lose all standard controls. Turing the tool off would reset your Throttle control to Zero as your ship leveled itself. If you reengage the engines before the ship has fully leveled (in about 2 seconds), your ship will be pushed up or down a bit as well as the normal forward and back (more with kero/moonshine). So, yes, you would move in the direction you are pitched. Since this is only 8-12 degrees for less than two seconds, the effect would not be extreme. This is why it would have more of an effect on high acceleration ships. Roll would have no effect on normal throttle.
'Feathering' the tool with kero or moonshine to get a sustained dive would be possible, though sort of a pain due to the self-leveling and throttle resets. Could likely get a sustained dive of 4-6 degrees. Definitely a higher pilot skill to learn.