I had big problems with my Mighty mouse, which uses its whole top area as scroll bar. When my finger touches the mouse, Scribus often thinks i am using the scroll bar, and the numbers run in the selector where my mouse happened to be in.
I found an easy solution, when I took my old mechanical wheel mouse from my drawer. It is not so beautiful, but i works!
PS. Thanks for many fine tips; I have worried quite a long time, why I can't adjust the space between text box and text. Now I can! (It is in Properties > Text > Columns and Text Distances)