There’s a saying in the arts “Know the rules before you break the rules.” Master the classical conventions of your field before you violate them. Break the rules deliberately and not accidentally, skillfully and not due to a lack of skill. There’s a world of difference between a beginning musician playing a wrong note and a professional musician making an unusual note choice.
Does this apply to math? You don’t think of math as being somewhere you break rules, but it does apply, and it applies differently in pure and applied math.
Often “breaking the rules” is shorthand for following a more complex set of rules, one that includes more context . Skilled artists who “break the rules” are often not being iconoclastic but simply following a more sophisticated set of rules.
In pure math, you might create new rules, such as developing a new geometry where Euclid’s parallel postulate is replaced with another postulate. Non-Euclidean geometry is not lawless; it just follows a slightly different set of laws. And these variant laws are not capricious but rather are motivated by the physical universe .
In applied math, you have no choice but to break the rules. The world breaks the rules for you. You have to judge whether these deviations from the ideal are significant and if so how to mitigate them. For example, there has never been an infinite capacitor in the real world, and yet we model capacitors as being infinite all the time. Except sometime you can’t. The hard part is knowing when.
The hypotheses of theorems in mathematical statistics never entirely hold, and yet the theorems are very useful. When a statistician applies a theorem about infinite data sets to a set with 100 measurements, he isn’t “expressing” himself by flaunting some oppressive rule. He’s making a professional judgment that in context the infinite is an adequate guide.
 This reminds me of “deep magic from the dawn of time” and “deeper magic from before the dawn of time” in The Lion, the Witch, and the Wardrobe.”
 How can the universe be Euclidean and non-Euclidean? You can chose to describe different aspects of universe in different ways. For example, when someone asks how far Tel Aviv is from Houston, they might have in mind the distance between the cities as a pair of points in three dimensional space, or they might have in mind the distance as two points on the surface of a sphere. The question isn’t whether the geometry of our corner of the universe is Euclidean but whether Euclidean distance is most appropriate for a particular use.