When social scientists began employing evolutionary game theory (EGT) in their disciplines, the question arose what the appropriate interpretation of the formal EGT framework would be. Social scientists have given different answer, of which I distinguish three basic kinds. I then proceed to uncover the conceptual tension between the formal framework of EGT, its application in the social sciences, and these three interpretations. First, I argue that EGT under the biological interpretation has a limited application in the social sciences, chiefly because strategy replication often cannot be sensibly interpreted as strategy bearer reproduction in this domain. Second, I show that alternative replication mechanisms imply interpersonal comparability of strategy payoffs. Giving a meaningful interpretation to such comparisons is not an easy task for many social situations, and thus limits the applicability of EGT in this domain. Third, I argue that giving a new interpretation both to strategy replication and selection solves the issue of interpersonal comparability, but at the costs of making the new interpretation incompatible with natural selection interpretations of EGT. To the extent that social scientists seek such a natural selection interpretation, they face a dilemma: either face the challenge that interpersonal comparisons pose, or give up on the natural selection interpretation. By identifying these tensions, my analysis pleas for greater awareness of the specific purposes of EGT modelling in the social sciences, and for greater sensitivity to the underlying microstructure on which the evolutionary dynamics and other EGT solution concepts supervene.