One of the most critical tasks for startups is to validate their business model. Therefore, entrepreneurs try to collect information such as feedback from other actors to assess the validity of their assumptions and make decisions. However, previous work on decisional guidance for business model validation provides no solution for the highly uncertain and complex context of early-stage startups. The purpose of this paper is, thus, to develop design principles for a Hybrid Intelligence decision support system (HI-DSS) that combines the complementary capabilities of human and machine intelligence. We follow a design science research approach to design a prototype artifact and a set of design principles. Our study provides prescriptive knowledge for HI-DSS and contributes to previous work on decision support for business models, the applications of complementary strengths of humans and machines for making decisions, and support systems for extremely uncertain decision-making problems.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
For further details of the problem identification, demonstration, and evaluation see Appendix.
For a comprehensive review of design features for decisional guidance, see Morana et al. (2017).
For further details of the problem identification, demonstration, and evaluation phases, see Appendix.
For further information please see www.vencortex.com
Al-Debei, M. M., & Avison, D. (2010). Developing a unified framework of the business model concept. European Journal of Information Systems, 19(3), 359–376.
Alvarez, S. A., & Barney, J. B. (2007). Discovery and creation: Alternative theories of entrepreneurial action. Strategic Entrepreneurship Journal, 1(1–2), 11–26.
Alvarez, S. A., Barney, J. B., & Anderson, P. (2013). Forming and exploiting opportunities: The implications of discovery and creation processes for entrepreneurial and organizational research. Organization Science, 24(1), 301–317.
Andries, P., & Debackere, K. (2007). Adaptation and performance in new businesses: Understanding the moderating effects of independence and industry. Small Business Economics, 29(1), 81–99.
Armstrong, J. S. (2001). Principles of forecasting: A handbook for researchers and practitioners: Springer science and business media.
Arnold, V., Collier, P. A., Leech, S. A., & Sutton, S. G. (2004). Impact of intelligent decision aids on expert and novice decision-makers’ judgments. Accounting and Finance, 44(1), 1–26.
Attenberg, J., Ipeirotis, P. G., & Provost, F. J. (2011). Beat the machine: Challenging workers to find the unknown unknowns. Human Computation, 11(11).
Baer, J., & McKool, S. S. (2014). The gold standard for assessing creativity. International Journal of Quality Assurance in Engineering and Technology Education (IJQAETE), 3(1), 81–93.
Bailey, D. E., Leonardi, P. M., & Barley, S. R. (2012). The lure of the virtual. Organization Science, 23(5), 1485–1504.
Baker, J., Jones, D. R., & Burkman, J. (2009). Using visual representations of data to enhance sensemaking in data exploration tasks. Journal of the Association for Information Systems, 10(7), 533.
Bazerman, M. H., Moore, D. A. (2012). Judgment in managerial decision making.
Benbasat, I., Dexter, A. S. , Todd, P. (1986). An experimental program investigating color-enhanced and graphical information presentation: An integration of the findings. Communications of the ACM, 29(11), 1094–1105.
Bharadwaj, A., El Sawy, O. A., Pavlou, P. A. , Venkatraman, N. V. (2013). Digital business strategy: Toward a next generation of insights.
Blank, S. (2013). Why the lean start-up changes everything. Harvard Business Review, 91(5), 63–72.
Blattberg, R. C., & Hoch, S. J. (1990). Database models and managerial intuition: 50% model+ 50% manager. Management Science, 36(8), 887–899.
Blohm, I., Riedl, C., Füller, J., & Leimeister, J. M. (2016). Rate or trade? Identifying winning ideas in open idea sourcing. Information Systems Research, 27(1), 27–48.
Bouwman, H., Zhengjia, M., van der Duin, P., & Limonard, S. (2008). A business model for IPTV service: A dynamic framework. info, 10(3), 22–38.
Brynjolfsson, E., Geva, T., & Reichman, S. (2016). Crowd-squared: Amplifying the predictive POWER of SEARCH trend data. MIS Quarterly, 40(4), 941–961.
Carlile, P. R. (2002). A pragmatic view of knowledge and boundaries: Boundary objects in new product development. Organization Science, 13(4), 442–455.
Cavalcante, S., Kesting, P., & Ulhøi, J. (2011). Business model dynamics and innovation:(re) establishing the missing linkages. Management decision, 49(8), 1327–1342.
Cheng, J., Bernstein, M. S. (2015). Flock: Hybrid crowd-machine learning classifiers. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing, 600–611.
Cohen, S., & Hochberg, Y. (2014). Accelerating startups: The seed accelerator phenomenon.
Colton, S., Wiggins, G. A. (Ed.) 2012. Computational creativity: The final frontier?: IOS Press.
Creamer, G. G., Ren, Y., Sakamoto, Y., & Nickerson, J. V. (2016). A textual analysis algorithm for the equity market: The European case. The Journal of Investing, 25(3), 105–116.
Daas, D., Hurkmans, T., Overbeek, S., & Bouwman, H. (2013). Developing a decision support system for business model design. Electronic Markets, 23(3), 251–265.
Dellermann, D., Lipusch, N., Ebel, P. (2017a). Developing design principles for a crowd-based business model validation system. En A. Maedche, J. Vom Brocke A. Hevner (Eds.). Designing the digital transformation: 12th international conference, DESRIST 2017, Karlsruhe, Germany, may 30 – June 1, 2017, proceedings (pp. 163–178). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-59144-5_10.
Dellermann, D., Lipusch, N., & Ebel, P. (Eds.). (2018). Heading for new shores: Crowdsourcing for entrepreneurial opportunity creation.
Dellermann, D., Lipusch, N., Ebel, P. A., Popp, K. M. , Leimeister, J. M. (2017b). Finding the Unicorn: Predicting Early Stage Startup Success through a Hybrid Intelligence Method. Proceedings ICIS 2017, Seoul.
Demil, B., & Lecocq, X. (2010). Business model evolution: In Search of dynamic consistency. Business Models, 43(2), 227–246. https://doi.org/10.1016/j.lrp.2010.02.004.
Demil, B., Lecocq, X., Ricart, J. E., & Zott, C. (2015). Introduction to the SEJ special issue on business models: Business models within the domain of strategic entrepreneurship. Strategic Entrepreneurship Journal, 9(1), 1–11.
Dul, J., Hak, T. (2007). Case study methodology in business research: Routledge.
Ebel, P., Bretschneider, U., & Leimeister, J. M. (2016). Leveraging virtual business model innovation: A framework for designing business model development tools. Information Systems Journal, 26(5), 519–550.
Einhorn, H. J. (1972). Expert measurement and mechanical combination. Organizational Behavior and Human Performance, 7(1), 86–106.
Einhorn, H. J. (1974). Expert judgment: Some necessary conditions and an example. Journal of Applied Psychology, 59(5), 562.
Euchner, J., & Ganguly, A. (2014). Business model innovation in practice. Research-Technology Management, 57(6), 33–39.
Gönül, M. S., Önkal, D., & Lawrence, M. (2006). The effects of structural characteristics of explanations on use of a DSS. Decision Support Systems, 42(3), 1481–1493.
Gordijn, J., & Akkermans, H. (2007). Business models for distributed generation in a liberalized market environment. Electric Power Systems Research, 77(9), 1178–1188.
Gordijn, J., Akkermans, H., & Van Vliet, J. (2001). Designing and evaluating e-business models. IEEE Intelligent Systems, 16(4), 11–17.
Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS Quarterly, 497–530.
Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for maximum impact. MIS Quarterly, 37(2).
Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Association for Information Systems, 8(5), 312.
Haaker, T., Bouwman, H., Janssen, W., & de Reuver, M. (2017). Business model stress testing: A practical approach to test the robustness of a business model. Futures, 89, 14–25.
Hevner, A. R. (2007). A three-cycle view of design science research. Scandinavian Journal of Information Systems, 19(2), 4.
Hevner, A. , Chatterjee, S. (2010). Design research in information systems: Theory and practice: Springer science and business media.
Hochberg, Y. V. (2016). Accelerating Entrepreneurs and Ecosystems: The Seed Accelerator Model. Innovation Policy and the Economy, 16(1), 25–51.
Huang, L., & Pearce, J. L. (2015). Managing the unknowable: The effectiveness of early-stage investor gut feel in entrepreneurial investment decisions. Administrative Science Quarterly, 60(4), 634–670.
John, T. (2016). Supporting business model idea generation through machine-generated ideas: A design theory. Proceedings of the International Conference on Information Systems,
John, T. , Kundisch, D. (2015). Why fit leads to surprise: An extension of cognitive fit theory to creative problems. Proceedings of the International Conference on Information Systems,
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260.
Kahneman, D. (2011). Thinking, fast and slow: Macmillan.
Kamar, E. (2016). Directions in hybrid intelligence: Complementing AI systems with human intelligence. IJCAI, 4070–4073.
Keuschnigg, M., & Ganser, C. (2016). Crowd wisdom relies on agents’ ability in small groups with a voting aggregation rule. Management Science, 63(3), 818–828.
Klein, M., & Garcia, A. C. B. (2015). High-speed idea filtering with the bag of lemons. Decision Support Systems, 78, 39–50.
Larrick, R. P., & Feiler, D. C. (2015). Expertise in decision making. The Wiley Blackwell handbook of judgment and decision making, 696–721.
Larrick, R. P., Mannes, A. E., Soll, J. B., & Krueger, J. I. (2011). The social psychology of the wisdom of crowds. Social psychology and decision making, 227–242.
Leimeister, J. M., Huber, M., Bretschneider, U., & Krcmar, H. (2009). Leveraging crowdsourcing: Activation-supporting components for IT-based ideas competition. Journal of Management Information Systems, 26(1), 197–224.
Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R news, 2(3), 18–22.
Limayem, M., & DeSanctis, G. (2000). Providing decisional guidance for multicriteria decision making in groups. Information Systems Research, 11(4), 386–401.
Magnusson, P. R., Wästlund, E., & Netz, J. (2016). Exploring users' appropriateness as a proxy for experts when screening new product/service ideas. Journal of Product Innovation Management, 33(1), 4–18.
Mahoney, L. S., Roush, P. B., & Bandy, D. (2003). An investigation of the effects of decisional guidance and cognitive ability on decision-making involving uncertainty data. Information and Organization, 13(2), 85–110.
Malone, T. W., Laubacher, R. , Dellarocas, C. (2009). Harnessing crowds: Mapping the genome of collective intelligence.
March, J. G. (1978). Bounded rationality, ambiguity, and the engineering of choice. The Bell Journal of Economics, 587–608.
Massey, A. P., & Wallace, W. A. (1991). Focus groups as a knowledge elicitation technique: An exploratory study. IEEE Transactions on Knowledge and Data Engineering, 3(2), 193–200.
McCormack, J., d’Inverno, M. (Ed.) (2014). On the future of computers and creativity.
Moellers, T., Bansemir, B., Pretzl, M. , Gassmann, O. (2017). Design and evaluation of a system dynamics based business model evaluation method. En a. Maedche, J. Vom Brocke, A. Hevner (Eds.). Designing the digital transformation: 12th international conference, DESRIST 2017, Karlsruhe, Germany, may 30 – June 1, 2017, proceedings (pp. 125–144). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-59144-5_8.
Montazemi, A. R., Wang, F., Nainar, S. K., & Bart, C. K. (1996). On the effectiveness of decisional guidance. Decision Support Systems, 18(2), 181–198.
Morana, S., Schacht, S., Scherp, A., & Maedche, A. (2017). A review of the nature and effects of guidance design features. Decision Support Systems, 97, 31–42.
Morris, M., Schindehutte, M., & Allen, J. (2005). The entrepreneur's business model: Toward a unified perspective. Journal of Business Research, 58(6), 726–735.
Nagar, Y. , Malone, T. (2011). Making business predictions by combining human and machine intelligence in prediction markets.
Nielsen, J. (1997). The use and misuse of focus groups. IEEE Software, 14(1), 94–95.
Nonaka, I., & von Krogh, G. (2009). Perspective—Tacit knowledge and knowledge conversion: Controversy and advancement in organizational knowledge creation theory. Organization science, 20(3), 635–652.
Ojala, A. (2016). Business models and opportunity creation: How IT entrepreneurs create and develop business models under uncertainty. Information Systems Journal, 26(5), 451–476.
Olson, D. L., Delen, D., & Meng, Y. (2012). Comparative analysis of data mining methods for bankruptcy prediction. Decision Support Systems, 52(2), 464–473.
Osterwalder, A., & Pigneur, Y. (2013). Designing business models and similar strategic objects: The contribution of IS. Journal of the Association for Information Systems, 14(5), 237–244.
Osterwalder, A., Pigneur, Y., & Tucci, C. L. (2005). Clarifying business models: Origins, present. and future of the concept. Communications of the association for Information Systems, 16(1), 1.
Ozer, M. (2009). The roles of product lead-users and product experts in new product evaluation. Research Policy, 38(8), 1340–1349.
Pagani, M. (2009). Roadmapping 3G mobile TV: Strategic thinking and scenario planning through repeated cross-impact handling. Technological Forecasting and Social Change, 76(3), 382–395.
Parikh, M., Fazlollahi, B., & Verma, S. (2001). The effectiveness of decisional guidance: An empirical evaluation. Decision Sciences, 32(2), 303–332.
Patel, N. (2015) 90% of startups fail: Here's what you need to know about the 10%. Fortune. Recuperado de https://www.forbes.com/sites/neilpatel/2015/01/16/90-of-startups-will-fail-heres-what-you-need-to-know-about-the-10/#175654b96679
Pauwels, C., Clarysse, B., Wright, M., & van Hove, J. (2016). Understanding a new generation incubation model: The accelerator. Technovation, 50, 13–24.
Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77.
Petersen, M. A. (2004). Information: Hard and soft.
Riedl, C., Blohm, I., Leimeister, J. M., & Krcmar, H. (2013). The effect of rating scales on decision quality and user attitudes in online innovation communities. International Journal of Electronic Commerce, 17(3), 7–36.
Schneckenberg, D., Velamuri, V. K., Comberg, C., & Spieth, P. (2017). Business model innovation and decision making: Uncovering mechanisms for coping with uncertainty. RandD Management, 47(3), 404–419.
Sengupta, K., & Abdel-Hamid, T. K. (1993). Alternative conceptions of feedback in dynamic decision environments: An experimental investigation. Management Science, 39(4), 411–428.
Shepherd, D. A. (2015). Party on! A call for entrepreneurship research that is more interactive, activity based, cognitively hot, compassionate, and prosocial: Elsevier.
Shepherd, D. A., Williams, T. A., & Patzelt, H. (2015). Thinking about entrepreneurial decision making: Review and research agenda. Journal of Management, 41(1), 11–46.
Silver, M. S. (1991). Decisional guidance for computer-based decision support. MIS Quarterly, 105–122.
Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118.
Song, M., Podoynitsyna, K., van der Bij, H., & Im Halman, J. (2008). Success factors in new ventures: A meta-analysis. Journal of Product Innovation Management, 25(1), 7–27.
Sonnenberg, C., & Vom Brocke, J. (2012). Evaluations in the science of the artificial–reconsidering the build-evaluate pattern in design science research. Design science research in information systems. Advances in. Theory Into Practice, 381–397.
Sosna, M., Trevinyo-Rodríguez, R. N., & Velamuri, S. R. (2010). Business model innovation through trial-and-error learning: The Naturhouse case. Long Range Planning, 43(2), 383–407.
Soukhoroukova, A., Spann, M., & Skiera, B. (2012). Sourcing, filtering, and evaluating new product ideas: An empirical exploration of the performance of idea markets. Journal of Product Innovation Management, 29(1), 100–112.
Speier, C., & Morris, M. G. (2003). The influence of query interface design on decision-making performance. MIS Quarterly, 397–423.
Spiegel, O., Abbassi, P., Zylka, M. P., Schlagwein, D., Fischbach, K., & Schoder, D. (2016). Business model development, founders' social capital and the success of early stage internet start-ups: A mixed-method study. Information Systems Journal, 26(5), 421–449.
Strauss, A., and Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures and techniques: Sage Publications, Inc.
Surowiecki, J. (2005). The wisdom of crowds: Anchor.
Teece, D. J. (2010). Business models, business strategy and innovation. Long range planning, 43(2–3), 172–194.
Timmers, P. (1998). Business models for electronic markets. Electronic Markets, 8(2), 3–8.
Thaler, R., & Sunstein, C. (2008). Nudge: The gentle power of choice architecture. New Haven: Yale.
Tremblay, M. C., Hevner, A. R., & Berndt, D. J. (2010). Focus groups for artifact refinement and evaluation in design research. Communications of the Association for Information Systems (CAIS), 26(27), 599–618.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.
Veit, D., Clemons, E., Benlian, A., Buxmann, P., Hess, T., Kundisch, D., et al. (2014). Business models. Business and Information Systems Engineering, 6(1), 45–53.
Venable, J., Pries-Heje, J., & Baskerville, R. (2016). FEDS: A framework for evaluation in design science research. European Journal of Information Systems, 25(1), 77–89.
Vessey, I. (1991). Cognitive fit: A theory-based analysis of the graphs versus tables literature. Decision Sciences, 22(2), 219–240.
Vessey, I., & Galletta, D. (1991). Cognitive fit: An empirical study of information acquisition. Information Systems Research, 2(1), 63–84.
Yin, R. K. (2013). Case study research: Design and methods: Sage publications.
Yuan, H., Lau, R. Y. K., & Xu, W. (2016). The determinants of crowdfunding success: A semantic text analytics approach. Decision Support Systems, 91, 67–76.
Zhang, S. X., & Cueto, J. (2017). The study of bias in entrepreneurship. Entrepreneurship Theory and Practice, 41(3), 419–454.
Zoric, J. (2011). Connecting business models with service platform designs: Exploiting potential of scenario modeling. Telematics and Informatics, 28(1), 40–54.
Zott, C., Amit, R., & Massa, L. (2011). The business model: Recent developments and future research. Journal of Management, 37(4), 1019–1042.
Responsible Editor: Alexander Mädche
Electronic supplementary material
About this article
Cite this article
Dellermann, D., Lipusch, N., Ebel, P. et al. Design principles for a hybrid intelligence decision support system for business model validation. Electron Markets 29, 423–441 (2019). https://doi.org/10.1007/s12525-018-0309-2
- Collective intelligence
- Machine learning
- Decision support system
- Hybrid intelligence
- Business model
- Decision making