You can operate a compressor in surge for a limited time (a couple seconds at a time), and achieve a relatively lengthy lifespan. However, repeated and extended operation in surge will result in a high cycle fatigue failure of the impeller blades. At the risk of sounding like a text book, surge is the phenominon of backflow through the compressor. This rapid reversal of flow makes the fluid in the compressor highly unstable resulting in vibration and high stresses in the blades. The way a compressor operates, the higher the backpressure imposed on the impeller, the lower the massflow and higher the pressure rise. When you release the throttle and throttle body closes this creates an increase in backpressure on the impeller, but a decrease in massflow since the flow follows the path of least resistance back through the compressor. By definition, this pushes the compressor towards surge. Every time the compressor surges you rack up a certain number of fatigue life cycles in the material, and eventually you will reach the fatigue limit (high cycle fatigue) and the material will fail. If you have a wheel spinning at 60k+ rpm, you can accrue millions of life cycles in a matter of minutes. The newer ball bearings are certainly more stout than journal bearings, but that doesn't not make the turbocharger indestructible. You may be able to operate sans bypass valve for a while, but it will prematurely fail the impeller or destroy the bearings.
Surge is the worse possible operating condition for a rotating wheel. Try being in the same building as an industrial gas turbine engine when it surges, it feels like an earthquake.
Now decide for yourself if you should run a blow off valve or not.