Make the bells and whistles part of the problems you want them to solve.
This is actually a great situation, since your students seem to be really interested in those things and show some intrinsic motivation on working on them. Just think about this, they are actually spending time doing extra work that you didn't even ask them to do! Now you just need to make sure that they still understand the mandatory stuff.
Ask yourself, why is it more interesting for a student to fiddle with those boring visual things instead of the exciting stuff that you are trying to teach? Because to them, it's the opposite. What you teach is the boring stuff, but oooh, look at this shiny little button I just created! And now I changed its color to orange, how fancy is that!
The good news for you is that with this boring programming logic you are trying to teach, you can create far more interesting visual effects than what they can come up with through their tinkering.
Want to teach them loops? Have them draw some fancy Logo flowers:
They will probably still spend most of the lesson tweaking the colors, but soon they will realize that they can achieve the most interesting effects by playing around with the loop variables. Nesting loops gives super cool results here. And the playing around can actually give them a great understanding in how loops work.
Similarly, one of the strongest motivations for the adoption of object-oriented programming techniques came from problems in GUI programming. Hand-tweaking a GUI to look nice becomes a nightmare once your number of widgets gets larger. Add resizable windows to that and your students will quickly come to the end of their tinkering abilities. This is where your boring programming techniques come to the rescue! Show them how OOP allows splitting the problem of layouting the different widgets into distinct sub-problems which can then be easily composed later.
You can find similar examples for almost any problem in computer science. Instead of stopping your students from what they are doing, embrace the fact that they are showing interest and shape your teaching examples in a way that it makes use of that interest.
The downside is that coming up with nice examples here is not easy and can be quite an effort. IfI've never had a course where every single exercise was exciting and fun. But the good courses made sure they did one of them every once in a while to remind us that all this hard work pays off in the end.