If I adjust the volume with the -/+ buttons while Smart Volume Control is active (meaning, the Smart Volume Control feature has currently increased the volume due to background noise), the current volume drops back to "normal", and is changed as I press the buttons, and it stays at this level until the background noise drops below the Smart Volume Control threshold.

This is a problem. The "normal" volume is now at an undesired level.