Family history of first and second-degree relatives is known to increase the risk for breast cancer. Less data are available on the risks between defined multiple affected close and distant relatives for which the reliability of data may be an issue. Data on affected males are sparse. These questions and the probable genetic models were addressed in this study by means of a nationwide Swedish Family-Cancer Database. We estimated the effect of family history of breast cancer by Poisson regression for women of at least 30 years of age after adjusting for age, period, region, socioeconomic status, number of children, and age at first birth. The results of the study showed that relative risk (RR) for breast cancer was associated with a first degree as well as second-degree family history. Having at least two female affected first-degree relatives increased the RR at least to 2.8, favoring an additive interaction. The risk was increased around ten times in women with both parents affected. When either a father or a mother was affected, the RRs were nearly identical (RR = 1.73 and 1.74, respectively). The RR for a woman increased more when a brother was affected (RR = 2.48) compared to when a sister was affected (RR = 1.87). Having an affected grandmother showed lower familial excess risks than having an affected half sister (RR = 1.27, and 1.26; and RR = 1.39, and 1.50; respectively, for maternal and paternal relatives). We concluded that when both parents were diagnosed with breast cancer, the risk for the daughter was increased tenfold. Having an affected brother showed a somewhat higher risk than having an affected sister. The data suggest that male breast cancer has a higher genetic basis than female breast cancer, which invites further search of the underlying mechanisms.