Biological conversion of biomass into fuels and chemicals requires hydrolysis of the polysaccharide fraction into monomeric sugars prior to fermentation. Hydrolysis can be performed enzymatically or with mineral acids. In this study, dilute sulfuric acid was used as a catalyst for the pretreatment of rapeseed straw. The purpose of this study is to optimize the pretreatment process in a 15-mL bomb tube reactor and investigate the effects of the acid concentration, temperature, and reaction time. These parameters influence hemicellulose removal and production of sugars (xylose, glucose, and arabinose) in the hydrolyzate as well as the formation of by-products (furfural, 5-hydroxymethylfurfural, and acetic acid). Statistical analysis was based on a model composition corresponding to a 33 orthogonal factorial design and employed the response surface methodology to optimize the pretreatment conditions, aiming to attain maximum xylan, mannan, and galactan (XMG) extraction from hemicellulose of rapeseed straw. The obtained optimum conditions were: H2SO4 concentration of 1.76% and temperature of 152.6 °C with a reaction time of 21 min. Under these optimal conditions, 85.5% of the total sugar was recovered after acid hydrolysis (78.9% XMG and 6.6% glucan). The hydrolyzate contained 1.60 g/L glucose, 0.61 g/L arabinose, 10.49 g/L xylose, mannose, and galactose, 0.39 g/L cellobiose, 0.94 g/L fructose, 0.02 g/L 1,6-anhydro-glucose, 1.17 g/L formic acid, 2.94 g/L acetic acid, 0.04 g/L levulinic acid, 0.04 g/L 5-hydroxymethylfurfural, and 0.98 g/L furfural.