It must have been tough on those poor horses. A good profit margin for the owner demanded the horse be on the job every single day, so many were worked very hard and one wonders at the amount and quality of rest and maintenance they received when their (very hard) working day was done.
Also, I read somewhere constantly walking on cobbles is very hard on a horse, and eventually leads to hoof problems and hairline fractures in their lower legs, which could also account for the high loss rate. If you've ever read or seen "Black Beauty," mention is made of this even in this childrens' book. Very sad in some parts.