Given the centrality of arguments from vicious infinite regress to our philosophical reasoning, it is little wonder that they should also appear on the catalogue of arguments offered in defense of theses that pertain to the fundamental structure of reality. In particular, the metaphysical foundationalist will argue that, on pain of vicious infinite regress, there must be something fundamental. But why think that infinite regresses of grounds are vicious? I explore existing proposed accounts of viciousness cast in terms of contradictions, dependence, failed reductive theories and parsimony. I argue that no one of these accounts adequately captures the conditions under which an infinite regress—any infinite regress—is vicious as opposed to benign. In their place, I suggest an account of viciousness in terms of explanatory failure. If this account is correct, infinite grounding regresses are not necessarily vicious; and we must be much more careful employing such arguments to the conclusion that there has to be something fundamental.