I don't think you're going to find any place that has the selection Amazon has for non-specialized stuff. I purchase CDs from all sorts of places: local shops, Bandcamp, eBay, Discogs, genre-specialty stores, independent record shops that have websites, etc.
I have had a few issues here and there with Amazon but typically they are good about correcting things with something goes wrong, even if it involves a third party seller. What was your issue if you don't mind me asking?