The construction industry is a major economic sector, but it is plagued with inefficiencies and low productivity. Robotics and automated systems have the potential to address these shortcomings; however, the level of adoption in the construction industry is very low. This paper presents an investigation into the industry-specific factors that limit the adoption in the construction industry. A mixed research method was employed combining literature review, qualitative and quantitative data collection and analysis. Three focus groups with 28 experts and an online questionnaire were conducted. Principal component and correlation analyses were conducted to group the identified factors and find hidden correlations. The main identified challenges were grouped into four categories and ranked in order of importance: contractor-side economic factors, client-side economic factors, technical and work-culture factors, and weak business case factors. No strong correlation was found among factors. This study will help stakeholders to understand the main industry-specific factors limiting the adoption of robotics and automated systems in the construction industry. The presented findings will support stakeholders to devise mitigation strategies.