Run automatically without any human participation beyond launching the test suite
And maybe not even that: Cruise Control, Continuum, etc.
Results should be pass/fail
Test failures need to be blindingly obvious
Test must be reproducible and consistent