The words signify and signal come from the Latin word signus for sign. The phrase “statistically significant” was coined to indicate the existence of a statistical signal that an effect was present. The word significant in this context is not meant to imply that an effect is large or important. David Salsburg elaborates on this in his book The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century (ISBN 0805071342).
The word was used in its late-nineteenth-century meaning, which is simply that the computation signified or showed something. As the English language entered the twentieth century, the word significant began to take on other meanings, until it took on its current meaning, implying something very important. … Unfortunately, those who use statistical analysis often treat a significant test statistic as implying something closer to the modern meaning of the word.
An effect is statistically significant if it is considered unlikely to be a coincidence. An effect can be highly significant (very unlikely to be a coincidence) and yet be very small. Also, calling an effect “statistically significant” does not imply that anyone necessarily thinks the effect is consequential.
Here’s an example from astronomy. Newtonian mechanics describes the motions of the planets pretty well, but not perfectly. There is a measurable contribution from relativistic effects when studying the orbit of Mercury. The effect of relativity in this case would highly significant statistically in an experiment conducted carefully enough to measure the effect. I don’t know whether the effect is significant in the sense that NASA would need to take it into consideration when sending a probe to Mercury; it may not be. I imagine the contributions of relativity to describing the orbits of the rest of the planets are not significant to NASA engineers though they would be statistically significant if they could be measured.
I’ve heard that GPS needs to take relativity into account to get the right answer; if so relativistic effects are meaningful quite close to home! Okay, okay, make that “macro-scale”.
I’ve had the “statistically significant but clinically insignificant” conversation with non-statisticians many times over the years. It is one of the most important (IMO) lessons one could hope to retain from an introductory course.
I’ve heard the same thing about relativity and GPS. On the other hand, I’ve heard that the constant in Newton’s law of gravity is only known to 4 significant figures. Apparently you don’t need to know the equations of motion to extraordinary precision to send a probe to the outer planets. There must be a large amount of in-flight correction, Kalman filters and all that.