I'll vote with your colleagues here. I don't know anything about you or your teaching experience, but if you are new at it you should realize that a common error made by new teachers is that their students are just about like themselves and learn just about the same way that they themselves do. In reality this is wildly false. Every student is unique and almost none of them are very similar to their professors until you get to doctoral level education.
Your students are not like you and moreover every one of your students learns a bit differently. Learning Modalities is the term of art here. Some learn by hearing, some by doing, some by seeing, etc. Repetition is a big plus. Almost none of your students will learn anything told/shown/demonstrated/... to them just once.
Analogy and metaphor are big helps with any new topic with novices as long as you are aware of the limitations of any given metaphor. A set is like a coral of horses might get you off the ground, as long as you also know, and help your students know that a set is also unlike a coral of horses. The metaphor works. A picture helps. Some informal discussion works. Some question and answer works. Showing something that is NOT a set in some subtle way. Every bit adds to the student understanding.
And then, BANG, you can hit them with the formalism but you can also point out how the formalism relates to the less formal introductory remarks about the topic.
But if you teach strictly with formalism and your students are not yet used to dealing with formal definitions and arguments you will be speaking to only a tiny subset of your students.
So, it isn't just metaphor that works. It is attacking an idea from a variety of angles, but analogy/metaphor is a good way to start, though not a good way to finish up.
I have some experience that the most elegant, precise, and concise definitions, proofs, etc. is pretty much guaranteed to reach the smallest set of students possible.
Many professors have had the experience of feeling like a failure in a lecture because they made a mistake or got confused and had to muddle through, finally reaching the correct result. BUT then being thanked by the students for a wonderful lecture that enlightened them so much.
Thinking can be messy. It doesn't help to pretend otherwise.
Note, moreover, that formal definitions of things are not the way they arose originally. Professionals muddle through to an idea, often discussing it with colleagues, trying to pin down the true, deep, meaning. Only once they have done that can they begin to think in terms of a formalism. Set theory, in particular has bedeviled philosophers, logicians, and mathematicians for millennia.
In fact, your "definition" isn't quite as formal as you might think:
A set is an unordered, well defined collection of elements, in which repetition doesn't matter.
The reason is that it is based on English words and natural language is, itself, very messy. If I have a set of natural numbers, they are inherently ordered, but that isn't part of "set-ness". You need to define "well defined". The word "collection" is also messy and hard to define in a not-circular way. What do you mean by "element" (other than, circularly in relation to a set? Finally, I don't understand what you might mean by "repetition doesn't matter".
This is where analogy and examples and poking around the edges can help. For example why isn't "The set of all black horses" a valid thing? It can get very subtle.
Note that I've said similar things about teaching and reaching your students in answer to other questions on this site.