1 Introduction

This paper describes a community-engaged research project (Harvard Catalyst 2013) which aimed to aid particular organizations by using operations research (OR) techniques, while also learning about the organizations’ problems and about the process of applying the techniques within organizations. Within the framework of community-based operations research (CBOR, Johnson 2011a), we used decision modeling methods, particularly value-focused thinking (VFT, Keeney 1992, 1996).

We collaborated with community development corporations (CDCs) in several lower-income Massachusetts communities that focused on local housing development, especially acquisition and redevelopment of foreclosed housing. The importance of these organizations became increasingly apparent in the wake of the recent housing crisis in the US, which has led to unprecedented numbers of foreclosures and had large negative economic and social impacts in many locations.

We are motivated by research questions at two levels. Within the particular domain of CDCs, decision-making is usually qualitative, although some economic and financial considerations are quantified. The housing literature lacks structured definitions of objectives that allow for systematic analysis of trade-offs in decision-making. Therefore, by constructing objectives hierarchies using value modeling techniques for a set of CDCs, we explore which decision objectives are important across CDCs, which ones are important to some CDCs, and how does a CDC’s set of objectives depend on the characteristics of the organization and its situation.

More generally, as CBOR emerges as an area of practice, it is important to understand how community-based organizations (CBOs), as opposed to other more heavily studied organizations, respond to OR and to potential problem structuring methods (PSMs) including VFT. Therefore, in this study, we explore this response and how it depends on characteristics of the organizations and their situations. Our modeling efforts are organized within a multi-site, multi-method case study to generate results for these research questions.

The remainder of this paper is structured as follows. In Sect. 2, we review several relevant literatures relating to the intersection of OR techniques, CDCs and CBOs. Section 3 presents conceptual frameworks within which we position our research questions and describe the methods and plan for our study. In Sect. 4, we describe modeling efforts for three CDCs. These efforts are analyzed in Sect. 5 to address the research questions. In Sect. 6, we discuss how the results may extrapolate to broader questions of CBOR.

2 Literature review

2.1 Organizations

2.1.1 Community development corporations and organizational decision-making

Community development corporations in the US are central to revitalization of local areas that are economically and socially disadvantaged (Federal Reserve Bank of San Francisco and Low Income Investment Fund 2012). They pursue short-term and longer-term activities for local change as measured by economic, physical and social conditions. CDCs in particular usually operate through housing and economic development and community organizing (Nye and Glickman 2000).

Community development corporations are subject to a combination of economic, social and political forces (Gittell and Wilder 1999; Frisch and Servon 2006). They must meet the needs of multiple stakeholders (residents, businesses, other community organizations, government) and must balance multiple mission and operational objectives. However, there is a lack of scholarly literature on the means by which CDCs identify values, use values to generate objectives, and choose activities to optimize these objectives.Footnote 1 CDCs have been shown to focus more on process and structure than substantive decision modeling or decision-making in strategy design (Stoutland 1999) and tend not to think strategically or critically about their organizational mission and development objectives (NeighborWorks 2006). Moreover, the emphasis from funders and public officials on evaluation and outcome measurement has turned CDC attention away from such fundamental activities, which tend to be taken for granted (Bryson 1988; NeighborWorks 2006). It is often assumed that organizational values are not only understood by non-profit staff, but also agreed upon by other stakeholders. Even NeighborWorks America, the non-profit trade organization and support provider to CDCs, does not have among their many reports and best practice studies a discussion of how to identify and structure organizational values and development decisions, and only 3 % of the hundreds of courses and trainings offered by NeighborWorks relate to any aspect of strategic planning (Bratt 2009). Thus there is an apparent gap in CDC practice that could be filled by innovations related to decision modeling and decision-making.

2.1.2 Community-based organizations

Community development corporations are a type of community-based organization. CBOs, which may be public or private, incorporated for tax purposes or not, secular as well as religious, are non-profit organizations that tend to “[meet] human, educational, environmental or public community needs” (National Network of Libraries of Medicine 2013) in regions that are typically considered disadvantaged. CBOs explicitly reflect social welfare concerns of equity, inequality and local control in their missions (PBWorks.com 2013). Another way to view non-profit organizations is through their size and as measured by annual budget: The Boston Foundation (2008) classifies NPOs as ‘grassroots’ (budget of $250,000 or less), ‘safety net’ (budget within $250,000 and $50 million), and ‘economic engine’ (budget of $50 million or more) organizations. For this paper, we define CBOs as grassroots or safety net non-profit organizations that share the social policy and social justice characteristics listed above. It is plausible, then, that research on decision models for CDCs may be applicable to the larger universe of CBOs of similar size whose missions are related to housing, social and economic development or community capacity building.

2.2 Problem structuring

2.2.1 Problem structuring methods, soft OR and value-focused thinking

The field of operations research and management science (OR/MS; “OR” henceforth) typically consists of formal quantitative analysis on individual and organizational decision-making and problem solving, with varying levels of connections to real-world problems through practical methods. In the past two decades or so, particular classes of qualitative methods to engage real-world stakeholders and solve challenging problems have received growing attention under the labels of problem structuring methods (PSM) and “soft OR”. In the discussion which follows we provide details on the range of quantitative analysis that motivates and frames the present analysis, and argue that the qualitative methods that are the focus of this paper are properly viewed through the lens of PSM and soft OR.

Decision modeling for housing and community development encompasses both mathematical programming approaches and decision-analytic approaches (Johnson 2011b). In the former, the choice set can be very large, and prior knowledge of decision maker preferences is less important than the solution of models comprised of objective functions and constraints to generate decision policies (see e.g., Bayram et al. 2014; Johnson et al. 2010a). In the latter approach, however, the primary focus is modeling uncertainty, structuring preferences, objectives and attributes to help individual decision makers choose among alternatives (Dyer and Keisler 2012); examples of relevant applications include Armacost et al. (1994) and Johnson (2005). While recent published work on decision modeling for housing has focused on elaboration of specific social objectives arising from engagement with community partners (Johnson et al. 2012, 2013), there has been less attention paid to the way in which certain objectives are chosen to incorporate into prescriptive models, nor to the relationship these objectives may have with underlying organizational values. It is this approach in which we position our present inquiry.

Decision analysis (DA), a subfield of operations research, is typically used to (1) identify alternatives, uncertainties, and objectives, (2) elicit subjective probabilities and preferences over associated outcomes that are inputs to models for quantitative analysis, and (3) derive from these models a recommended course of action. DA is also used for a variety of other ways, such as sensitivity analysis. Value-focused thinking is an outgrowth of decision analysis which emphasizes the identification and structuring of objectives into value hierarchies and means-ends networks as valuable on its own, as well as serving as a prelude to traditional quantitative analysis.

While most traditional decision science relies on sophisticated mathematical models with observable and calculable inputs that yield precise answers, soft OR places less emphasis on engineering mechanistic solutions and more on value judgments and participation in its model development (e.g., Ackoff 1979; Ackermann 2012). These alternative approaches are generally appropriate when the problem presented to analysts is vaguely defined, involves multiple stakeholders (often with competing interests), includes a high degree of uncertainty, and involves data that are sparse and/or difficult to accurately measure (Mingers 2011). Accordingly, the methods used in soft OR applications are generally more interpretivist and less positivist in nature than traditional OR, with the process holding as much (and sometimes more) importance than the outcome (Mehrotra 2009; Mingers 2009).

Part of the soft OR process involves clear identification of the problems to be solved through elicitation of both the decisions to be made and objectives to be achieved. Facilitated modeling (Franco and Montibeller 2010) approaches such as problem structuring methods (PSMs) are thus sets of tools used to frame an issue within its relevant context (Mingers and Rosenhead 2001). The process involved in PSMs varies somewhat with the particular kind of method employed, but generally all PSMs have some common traits (Mingers and Rosenhead 2004): (1) integrating multiple perspectives, (2) being cognitively simple for participants to understand and engage with, (3) operating iteratively, and (4) allowing micro-level approaches. Some common types of PSMs include soft systems methodologies (Checkland and Scholes 1990; Ackermann 2012), drama theory (Bryant 2007) cognitive and causal mapping (Bryson et al. 2004) and strategic choice approach (Friend and Hickling 2004).

Soft OR, in addition to methods, constitutes engagement between people—OR experts and clients—aimed at bringing the client to insight, understanding, and ultimately action (Eden and Ackermann 2004). While soft OR involves activities such as problem identification and modeling, it also involves flexible and often facilitated conversations as a key means of advancing client thinking, while explicit modeling is used only when it will help accomplish that purpose. In fact, soft OR engagements often create the greatest value by bringing disagreements and uncomfortable issues to the fore so that they can be explicitly considered as part of planning and decision-making.

In its conception and application, VFT is consistent with the motivations and practice of soft OR. It was originally developed by Keeney (1992) as tools for applying multi-attribute utility by formulating decision-analytic problems so as to better reflect the subjective and sometimes conflicting values of decision makers and to identify decision opportunities that are aligned with decision makers’ fundamental values. Current VFT practice (e.g., as reviewed in Parnell et al. 2013) includes a wide range of techniques to surface, articulate and determine stakeholders’ objectives in a given context, as well as identify the relation of those objectives to each other and to the ways in which they may be achieved. VFT can be used flexibly both during initial conversations about a problem and later on as stakeholders process their situation. While it can be the basis for formal quantitative analysis, it often supports non-quantitative ends, such as generating alternatives and organizing discussion.

The emphasis on identifying organizational values is what makes VFT distinct within soft OR methods and from OR methods generally (Keeney 1992). Often, organizations will have a general idea or statement of their fundamental values or mission, but less clarity on how their decisions directly and indirectly achieve those values. Keeney distinguishes between values-driven and alternatives-driven decision processes, noting that the latter, while more common in traditional OR practice, lead decision makers toward solutions that may be operationally practical and efficient but that may not achieve their desired objectives. Focusing on values, Keeney argues, not only broadens the decision space and alternatives to be considered, but also fosters more strategic thinking about how such decisions may influence the achievement of multiple objectives. More recently, Keeney has made explicit the distinction between VFT and decision analysis through ‘value models’, which include utility functions and value functions as special cases and that rely on VFT, as traditionally understood, to ensure that these models reflect the actual values and fundamental concerns of multiple stakeholders (Keeney and von Winterfeldt 2008). We will refer to the primary method that generates values structures and trade-offs as ‘value-focused thinking’ and ‘values structuring’ interchangeably henceforth.

Value-focused thinking is similar to PSMs in that it may explore core organizational issues within their context, through a hands-on and iterative process in which decision makers identify successive levels of objectives. The levels range from those that are directly measured and influenced by organizational decisions to the more conceptual and mission-driven goals that define the organizations’ purpose. VFT goes beyond most PSMs, however, by linking objectives to specific actions with potentially measurable outcomes (Keisler 2012). While some overlap between VFT, PSMs, and other soft OR methods is apparent (Mingers 2011), we note that the former is rarely discussed as an option for conducting analyses of the latter. This may be because, as Mingers (2011) notes, VFT occupies a fuzzy middle ground between soft OR and traditional OR methods, as it combines qualitative data collection with quantitative analytical techniques for solving identified problems. PSMs, meanwhile, are not fundamentally focused on solving the problem, but instead in representing it in novel ways. Moreover, VFT can serve as both an intervention methodology and as a means to generate data about organizations and problem contexts. The notion of VFT as a multi-dimensional method that bridges diverse decision modeling approaches is central to the case analysis we present below.

2.2.2 Value modeling in the not-for-profit sector

Value-focused thinking has been found to be useful in a variety of organizations with a public mission by integrating different perspectives through objectives structures that define the dimensions across which organizations may make trade-offs, e.g., between needs of different constituents (Arvai et al. 2001; Chambal et al. 2003; Gregory et al. 2001). Making such trade-offs explicit also helps gain buy-in from these constituents (McDaniels et al. 1999).

Another feature of community and non-profit issues that lends them to VFT applications is the need for critical inspection of the problems such organizations face. Often, in these cases, the problem on the surface can mask deeper concerns that are not obvious to even the practitioners themselves (Nye and Glickman 2000; Frisch and Servon 2006). The introspective and iterative processes that are a hallmark of such applications help to get at root issues that may require a more critical lens to identify means-ends objectives that can effectively address such concerns (McDaniels et al. 1999).

Finally, because VFT approaches are designed specifically to address complex problems with a high degree of uncertainty (Keeney 1992), they have the potential to be useful in community-based applications operating in a dynamic social, economic, and political environment. CDCs are often subject to many constraints beyond their control, including funding and resource availability, municipal intervention, community engagement, policy, and internal organizational issues (Gittell and Wilder 1999; Nye and Glickman 2000; Glickman and Servon 1998). A flexible model that gets at fundamental objectives of the organization is thus required to help such organizations better operate within an ever-changing environment. VFT is then a promising candidate for analytic method to apply to the study of CDC (and NPO) decision modeling and decision-making.

2.2.3 Community-oriented and empirical decision modeling

Research on the practice of OR is concerned with how it is used within organizations, which may depend on the type of organization. Operations research and management science have had a long history of applications to not-for-profit organizations: government, military, and non-governmental organizations (see e.g., Pollock et al. 1994). A recent strand of research has emphasized the special nature of methods designed to meet the needs of community-based organizations. ‘Community operational research’ (Midgley and Ochoa-Arias 2004), which has its origins in the UK, is a mixed-method approach to problem identification, structuring, solving and implementation that is intimately linked to the perspectives and resources of local non-profit organizations. It tends to privilege qualitative and inductive approaches to decision modeling in local contexts. Operations management is the point of origin for a stream of research called ‘empirical operations management’, in which special attention is paid to real-world experiences of productions and operations management professionals in quantifying and solving decision problems (Scudder and Hill 1998; Wacker 1998). Inasmuch as case study and qualitative methods are embraced to generate and evaluate solutions in a practice context, this field is also a promising antecedent to the present study.

A newer initiative called community-based operations research (Johnson 2011a) adapts the innovations and sector focus of community OR and the emphasis on relevance of decision modeling in the field representative of empirical OM to a general perspective one might refer to as US-style OR. OR/MS researchers and professionals in the US-dominated perspective tend to rely on quantitative, prescriptive theory and methods that often reflect a logistics-systems view of problem domains, and an efficiency-based perspective on problem solving. CBOR is inspired by the values, missions, constraints and problem contexts of community-based organizations. It is designed to support the use of qualitative as well as quantitative models and methods, separately and together across particular applications, in an iterative, inductive way that may incorporate traditional mathematically based approaches. It aspires to embody the ethos of ‘appropriate technology’ for community-based decision modeling. The analysis to follow seeks to contribute to the study of CBOR, as well as its immediate antecedents.

2.3 Synthesis: values, decision modeling, CDCs and CBOs

As noted, there is little formal work on how CDCs should make decisions, either in terms of process or objectives. Our study is motivated by a desire to help these organizations identify novel and useful solutions to the intertwined problems of their communities. Multiple streams of thought in the decision sciences may provide useful and practical problem-solving guidance to organizations such as CDCs. Some of these streams represent (groups of) particular analytic methods, such as decision analysis. Others represent approaches to problem solving that tend to be particularly applicable to certain sectors, such as soft OR, community operations research and community-based operations research. The latter are young areas, and there is still much to learn about which exact approaches in these sectors are potentially most applicable and how to deploy them effectively.

3 Research plan

3.1 Questions

Our first set of questions concerns the conflicting objectives of CDCs with respect to their purchasing decisions and possibly to other activities in the community. Our conceptual framework for understanding the objectives of the client organizations (Fig. 1) is based on the literature of VFT in other organizations, the literature on CBOs, and the literature on CDCs, and our own experience and judgment.

Fig. 1
figure 1

Conceptual framework for CDC objectives

Full size image

Specifically, we propose that CDC objectives are determined by the problem domain (housing), characteristics of the CDC itself (e.g., its structure and its particular mission), and characteristics of the community it serves. While the literature touches upon a variety of considerations, there is nothing approaching a comprehensive listing of objectives that could serve as a basis for the decisions of a CDC. We do not have initial hypotheses about what these objectives will be. We have several conjectures: CDCs will share some objectives associated with improving neighborhoods through housing intervention; CDCs will not all have the same objectives; and the complexity of their means-ends objectives hierarchies will be related to the complexity of their mission and the number of stakeholders they have. We distill our conjectures to following research questions:

  1. 1.

    Which objectives do CDCs identify as means and ends?

  2. 2.

    Which of these objectives are common across CDCs?

  3. 3.

    Which of these objectives are contingent on the CDC’s characteristics and those of its target community?

  4. 4.

    How is the degree of complexity in value hierarchies structuring of value hierarchies contingent on the CDCs characteristics and those of its target community?

    Our second set of questions concerns CBO acceptance of OR-related methods. Our conceptual framework for these questions (Fig. 2) builds on the original technology acceptance model (TAM, Davis et al. 1989), which has been used to study decision support technologies as well as empirical operations research (Scudder and Hill 1998). The basic version of this model states that acceptance of a technology will depend on the technology’s target users perceptions of its usefulness and its ease of use. This framework generally refers to individual users, but in our case, with small organizations, there is not much difference between the decision makers and the users, so here we intend the framework to refer to the organization’s overall response. Appended to the basic framework are antecedents. Specifically, while TAM states that the perceptions of a method’s impact drive its acceptance, we hypothesize that these perceptions depend on the problem characteristics, the organization’s characteristics (e.g., its skills, mission), and the focus of the method. In Fig. 2, the arrows between perceived ease, perceived usefulness and acceptance indicate a positive association. The arrows from method, problem and organizational characteristics are placeholders indicating that we hope to find some such characteristics that have positive or negative association with the ease and usefulness.

    Fig. 2
    figure 2

    Conceptual framework for acceptance of OR methods

    Full size image

    The OR, soft OR and PSM literatures discuss various experiences, cases and guidelines for practice in a range of organization types, and there is a general understanding that different methods are appropriate for different situations. Rosenhead (2006, p. 764) observes from a review of reports that “the potential advantage of PSM’s transparent and accessible methods is evident” for grassroots organizations that have limited analytic expertise and non-hierarchical decision-making. However, there is little knowledge about which specific methods will be appropriate for CBOs. Given the holistic goals and community base associated with CBOs, and the many constraints they face, we started with the general hypothesis that CBOs would have high acceptance of value-focused thinking, and we formulate the following research questions:

  5. 5.

    To what degree will quantitative OR methods be accepted by CBOs?

  6. 6.

    To what degree are CBOs amenable to use of VFT?

  7. 7.

    How does CBO acceptance to various OR methods depend on organizational characteristics?

  8. 8.

    How amenable are CBOs to VFT alone and in conjunction with optimization-oriented methods?

  9. 9.

    How can CBOs use VFT as part of their broader decision and strategy processes?

3.2 Methods

As we noted, this effort is framed as community-engaged research, and some details of the interventions are customized to the needs of the client organizations. In the spirit of Eden and Ackermann (2004), the interventions are allowed to evolve in response to those needs. Within this frame, the effort is structured as a multi-case study to identify CDC objectives and understand their similarities and differences across organizations and to explore factors affecting CBO acceptance of use of problem structuring methods, using CDCs as an instance. VFT is the data generation method for the primary data for the first case study goal, i.e., the value hierarchies constructed are the artifact of interest, to be analyzed within and across the organizations, and to be arrayed against the secondary data.

Our data for the second goal are generated in the spirit of action research (Eden and Huxham 1996; Checkland and Holwell 1998) to improve practice, particularly action research of an inductive nature (Montibeller 2007). Specifically, our data are various observations and documentation about what happened in the course of helping the organizations, when and why in the engagement which analytic efforts were undertaken, with what motivation and what reception. To evaluate PSMs, White (2006, p. 849) suggests “collecting data through [a] pragmatic combination of approaches”. For our purposes, these data are mostly the record associated with conducting the efforts themselves, supplemented by our field notes and contemporaneous reflections with the clients. These data are also arrayed against our understanding of the organizations and their communities.

3.2.1 VFT as method

Our implementation of VFT relied on in-person discussions with staff of community partners and research-only analysis sessions, as well as multiple rounds of discussion of drafts of objectives hierarchies. Our single in-person discussion with community partners was divided into two parts comprising about 4 h. We took notes and taped the sessions. During the initial discussion of organizational goals and values, we asked a number of probing questions, such as: What do you want to achieve in real estate development? What are other high-level goals or objectives in housing and community development, including foreclosed housing? What benefits do you see to your stakeholders of your housing-related activities apart from those specifically tied to the presence of housing? What would you think of as ‘success’ in housing and community development? After a break, the researchers and community partners re-convened to clarify fundamental objectives and means-ends objectives. This involved exploring concepts such as: organizational goals; social/community goals; an over-riding theme versus subsidiary themes; attributes associated with means/ends goals, and clarifying concepts specific to different community partners.

After the initial meeting with community partners, we reviewed our in-person notes, transcribed the audio files and created the first of multiple drafts of objectives hierarchies. We shared each draft with the executive director of the CDC or with an analyst tasked with completing the structuring process, and we were especially sensitive to concerns of clarity, completeness and consistency with organizational values.

When it appeared that the community partner and researchers were at a stopping point regarding the objectives hierarchies, we asked ourselves how these structures might support specific decisions and decision processes. That is, the values structures served not only to help our partners clarify their understanding of the preferences, goals and constraints that characterized their housing and community development activities, but also to provide tangible guidance regarding decision alternatives. As explained below, two case sites felt that it would be useful to evaluate a range of alternative decision strategies with our values structures. We did this by developing weights for the various objectives, quantifying a collection of decision alternatives to create strategies, and ranking the strategies. We performed sensitivity analysis by altering objective weights to reproduce the preferences of different community stakeholders in addition to the CDC. Another case site felt that the holistic and community-engaged nature of their mission made such a conventional use of values structures less relevant; they preferred to consider how values structures we created could be used as a jumping-off point to suggest a wide range of potential community development actions. These actions could be classified according to specific means objectives, thus allowing the CDC to understand what portions of their mission might be fulfilled by pursuing one activity versus another.

3.2.2 Selection of cases

Our CDC case sites were selected a year before we decided to engage in a VFT process with them, as partner organizations on a larger research project in decision modeling for foreclosed housing acquisition and redevelopment (Johnson et al. 2010b; Turcotte et al. 2014). These organizations were filtered from an initial list of 24 prospective organizations. The requirements for organizations to be considered as potential partners on our research project included:

  • organizations based on Massachusetts and serving Massachusetts communities hard hit by the foreclosure crisis;

  • organizations actively engaged in the acquisition and redevelopment of foreclosed properties for the purposes of neighborhood stabilization;

  • organizations engaged in purchasing properties with some form of Federal funding for neighborhood-level foreclosure remediation, or in adherence to Federal guidelines for the purchase of foreclosed properties.

In addition, for the cases to provide material for comparisons, it was desirable for the set of organizations selected to be neither too similar nor too dissimilar along dimensions such as size, focus, and community problems.

We identified two community partners: a large CDC serving the Dorchester neighborhood of Boston, and a smaller CDC based on Fitchburg, MA, about an hour northwest of Boston. We also decided to include a third case based on Lowell, MA (henceforth “Simulated”), about an hour north of Boston and used in a previous pilot study (Turcotte et al. 2014). Characteristics of three organizations and their communities are summarized in Table 1. All three have comparable budgets and serve distressed and relatively low-income communities. Dorchester is more urban than the other two communities, which is reflected in dimensions such as its diversity, its greater variation in property values, its higher rents, as well its density. These reflect its connection to the larger city around it, and consequently the Dorchester CDC (henceforth “Urban”) is somewhat larger with more non-resident and professional staff, and it coordinates with other local organizations. The Lowell CDC is funded to serve almost as an arm of local government focused on housing, while the Fitchburg CDC (henceforth “Smalltown”) plays a much wider role in a community that has fewer organizational players.

Table 1 Case study sites
Full size table

3.2.3 Selection of primary method

For our study of community development corporations undertaking foreclosure acquisition and redevelopment for neighborhood stabilization, we sought to develop models that would support the decision-making of such organizations when evaluating different strategies and individual properties. We were aware that the organizations with which we were working faced some particular challenges, including limited financial resources, multiple stakeholders to consider, competition from private investors, and policy constraints. Our approach, therefore, required methods that would be flexible enough to accommodate these considerations and uncertainties and held the prospect of being used in daily practice by resource-limited organizations. Having identified a need for methods that are conducive to community-based OR applications, i.e., are inductive, iterative, client-focused, and able to use subjective data and assessments (Johnson 2011a), we turned to soft OR methods early in our process, and to PSMs in particular.

Starting with a number of well-known PSMs as potential methods to employ with our partner CDCs, we ruled out methods that emphasized strategic bargaining/game theory or were too focused on the integration of competing viewpoints; while our partners do need to balance the needs and preferences of multiple stakeholders, neither had identified this as a primary concern in their decision-making around foreclosure acquisition activities, which tend to be made internally rather than through a collaborative process. We also found that methods based on a systems approach to problem identification were not very relevant, since these tend to focus on long-term macro issues beyond the scope of our project. This narrowed down our options to a handful of methods, among them decision conferencing (Phillips 2007) and robustness analysis (Mingers and Rosenhead 2004). The former method deals with multi-dimensional consequences in the presence of uncertainty, largely through collaborative workshops in which participants develop the model themselves, with the goal of achieving a shared understanding of organizational purpose. The latter method emphasizes flexibility in organizational planning and the development of strategic activities that can handle uncertainty. Yet neither represented a strong match with our partners’ needs. Our partners wanted decision-making processes that were not just more efficient, but also could provide straightforward guidance on strategy design for housing and community development in the short- and medium-term under certainty.

For the reasons described in Sect. 2, we then we turned to VFT as an option to develop better understandings of organizational activities and objectives as well as generating actionable strategies. We presented VFT as an option to our community partners alongside two other customized traditional OR approaches we had developed: a stochastic optimization models for allocating limited resources given market uncertainties and incorporating a small set of objectives (Bayram et al. 2014); and a dynamic programming model for bidding on properties (Solak and Bayram 2013). Both organizations preferred to pursue a VFT-type modeling approach.

Within VFT, we considered all of the main techniques as plausible. Structuring of objectives hierarchies or means-ends networks is a necessary step whether or not the other techniques are used. Thus, our first set of research questions emerges as feasible, along with the use of objectives hierarchies as data for them. Following structuring, possible techniques from VFT include development of linear or non-linear utility functions (by defining ranges, eliciting weights, etc.), scoring alternatives using such functions, and generating alternatives and, ultimately, strategies using fundamental and other objectives as a starting point.

4 Case descriptions

4.1 Simulated CDC

4.1.1 Background

One of us (Turcotte), who had previously served on the board of a community development corporation, played the role of an executive director at Simulated CDC. The executive director defined the primary mission of this organization as to improve the quality of life in the community served by the CDC, primarily though improvements in the economic and personal opportunities of residents of the neighborhood. The emphasis on resident quality of life was reflected in the types of objectives he described as important to the CDC’s mission, such as: empowering residents, increasing affordable rental and homeownership opportunities, reducing crime, increasing amenities, and improving outcomes for children. Specific to the issue of foreclosures, the CDC would likely identify additional objectives related to improving the stability of the neighborhood and the quality of the foreclosed housing stock: these objectives eventually became the means to the fundamental objectives specified in the VFT structure.

As discussed in Sect. 3, Simulated CDC serves a community that is larger than the other two actual CDCs, though one that has faced the same dynamics of long-term economic decline as the Smalltown CDC. According to multiple descriptors of resident demographics, housing market strength and organization capacity, it lies approximately in the middle of the urban and Smalltown CDCs.

4.1.2 Results: values structure

The VFT structure created for Simulated CDC was a simple hierarchy, with all higher-level objectives linked to lower-level means objectives and decisions, and no relationships defined between objectives at the same level. This structure is shown in Fig. 3.

Fig. 3
figure 3

Values structure, simulated CDC

Full size image

Three fundamental objectives all influence the core objective of maximizing the quality of the neighborhood: maximizing resident outcomes, maximizing neighborhood character, and maximizing quality of the housing market. A set of lesser/ends objectives all contribute to these fundamental objectives, which in turn are affected by two means objectives: maximize residential stability and maximize quality of the housing stock. To achieve these, we separately and subsequently identified several actionable decisions that the CDC could make with respect to the housing market, including acquisition and redevelopment of vacant units, market rate rental units, affordable rental units, homeownership opportunities, and other commercial/amenity properties.

Following the development of the objectives hierarchy, we developed a scoring model by assessing trade-off weights for the objectives at each level, and compared a range of alternatives. The details of this part of our engagement may be of interest to CBOR practitioners but do not relate to our core research questions. Therefore, this is described in section 1 of the electronic supplement.

4.1.3 Reflections on process

The purpose of engagement with Simulated CDC was to validate process issues of value-focused thinking that we would test in the field with two actual CDCs. Therefore, the authors, with research assistants, attempted to recreate as closely as possible the environment of a values elicitation question, starting with broad-based questions (“what goals are most important to your organization?”) and continuing with more specific questions intended to distinguish between means and ends objectives and between higher-level and lower-level means objectives (“for the objective just mentioned, in what ways could it be better achieved?”) Because there was only a single stakeholder for this case, we found the process to be deceptively smooth. We were able to complete our values elicitation in a single 3-h session. Revising and clarifying the values structure took place over multiple weeks after the values elicitation session. Elicitation swing weights required multiple tries as we adapted textbook-style methods to the expertise of a practitioner not experienced with quantitative analysis.

4.2 Urban CDC

4.2.1 Background

During our VFT exercise with Urban CDC, the staff (executive director, real estate director, two project managers, and the lead community organizer) also identified quality of life in the neighborhood as the primary mission of their organization. The real estate development group described actions to achieve this mission as consisting of interventions in the housing market and improvement of the housing stock, including actions meant to address affordability of quality housing, reduce blight, and deal with foreclosures (other groups within the CDC place more emphasis on improving resident engagement and outcomes).

Urban CDC serves a community that, though disadvantaged according to usual measures of income, crime and educational attainment, and disproportionately minority and renter occupied as compared to the city in which it is located, has a relatively strong housing market, as compared to the other CDCs in our study. This is a consequence of Urban CDC’s location in an economically vibrant city, and in a community that is not far from transit nodes which connect to more-affluent neighborhoods to the east and south, and to the downtown to the north.

4.2.2 Results: values structure

The VFT structure for Urban CDC was not a simple hierarchy. Some objectives within the same conceptual level were linked, but the structure (Fig. 4) still formed a directed graph and was straightforward to implement in a calculation model.

Fig. 4
figure 4

Values structure, Urban CDC

Full size image

A clear pattern to the means-ends and fundamental objectives is apparent: there are two fundamental objectives contributing to the core objective of maximizing quality of life in the neighborhood: maximizing quality of individual lives, and maximizing quality of the neighborhood. Interestingly, despite the focus of the VFT session on the real estate development activities of the organization, the objectives that are identified in the structure relate strongly to resident outcomes, including socioeconomic diversity, sense of belonging, and safety. Indeed, only one of the five ends objectives specified in the structure reflects housing stock/market interventions: beautification of the neighborhood (the fifth objective is community economic well-being). The means through which these objectives are sought are also largely resident-specific, and include asset building, advocacy/control, displacement, and affordability. The actionable decisions made by the CDC, however, are all stock-specific, and include acquisitions and redevelopment of different property types and conditions (as well as green space).

As for Simulated CDC, we developed scoring models using judgments from the Urban CDC staff weighting trade-offs among objectives. We developed several alternative strategies with limited input from that CDC. The resulting model and analysis are described in section 2 of the electronic supplement.

4.2.3 Reflections on process

Urban CDC staff initially appeared skeptical of the purpose and importance of the values elicitation process. This may be due to the CDCs extensive experience with all manner of strategic planning and community engagement sessions. Since VFT requires explicit consideration of fundamental values, different types of objectives, and of graphical connections within and between classes of objectives, and trade-offs between objectives, and because there is, ultimately, an operations orientation embedded in this analysis, we found that the CDC staff became enthusiastic with the values elicitation process. They made many suggestions to draw connections between different objectives, and emphasized the relatively small role that foreclosed housing development, the motivating purpose of our research project, actually played in their organization’s planning and operations. Our scenario analysis efforts were done with an analyst with master’s level training in planning who seemed favorably disposed to challenging quantitative assessments of swing weights and realistic alternative development scenarios. Even with this favorable analysis environment, the scenario analysis tasks required multiple face-to-face and phone meetings to clarify different options and to validate our policy and planning insights.

4.3 Smalltown CDC

4.3.1 Background

Smalltown CDC was distinct from the first two in some important respects. The number of CDC staff and diversity of roles represented was greater, which encouraged a broader conception of the organization’s overall mission, including community organizing, community development and resident capacity building, rather than objectives specific to real estate development activities. Indeed, the session was characterized by one member of the research team as continually “going up to connect to higher-level objectives, and require holistic scoring methods”, which made it harder to decouple real estate from other activities.

The unique characteristics of the Smalltown CDC and of the values elicitation process it engaged in may reflect the nature of the community it serves: about an hour west from Boston and consequently less affected by Boston’s economic strengths and less reflective of that region’s socioeconomic diversity. Informants at this CDC described a ‘broken’ housing market in which housing values were low, high-quality CDC-developed housing might wait for months for qualified buyers, and there was a sense of malaise and lack of engagement among community members.

4.3.2 Results: values structure

The VFT structure developed for Smalltown CDC is closer to a simple hierarchy than that of Urban CDC; all objectives are linked only to those on a higher level, though some links are across two levels (Fig. 5).

Fig. 5
figure 5

Values structure, Smalltown CDC

Full size image

Reflecting the broader organizational focus of the VFT session with this CDC, the structure also includes a greater number of means-ends and fundamental objectives, and an additional intermediate level of objectives that connect the means-ends and higher-level fundamental objectives. The fundamental objectives themselves contain more salient temporal characteristics, with two (quality of CDC-owned properties and quality of area proximate to CDC-owned properties) medium-term objectives (i.e., 1–3 years, or roughly one development cycle for the CDC) and a third (sustainability of neighborhood quality) that is longer term (i.e., observed after the CDC development is complete). The types of actionable decisions linked to these objectives, meanwhile, are less specific, and they include interventions in both residents’ lives (e.g., behavioral interventions, tenant screenings) and the housing market. Due to the holistic view that CDC staffers had of their mission, we chose not to articulate a set of specific operational decisions as we had done for the previous two CDCs.

An additional objective added between meetings was the maximization of community financial resources (both for residents and for public/private investment). Sub-objectives were not added, but could have been. We also added a second fundamental objective for sustainability of the organization, which is influenced by the achievement of quality in the neighborhood and thus by the development decisions of the CDC, though indirectly.

In contrast with the first two cases, the value structure developed for Smalltown CDC did not lend itself to a similar style of scoring model. The macro/organizational focus of the objectives identified and the lack of directly observable/measurable inputs (though measureable in theory) to these objectives both suggest this structure would be better suited to adaptation as a strategic development model. Such a model would help the CDC identify efficient allocations of time and resources to such organizational strategies broadly, rather than with respect to specific actionable development decisions. The results of these strategy design efforts are described in section 3 of the electronic supplement.

4.3.3 Reflections on process

Our experience with Smalltown CDC was different from that of Urban CDC and Simulated CDC. The executive director of Smalltown CDC seemed quite enthusiastic about the values structuring process, and encouraged a wide range of employees to participate with the researchers. The CDC participants willingly shared many experiences of their community engagement efforts and their assessment of the spirit and values of community residents, both positive and negative. Though we agreed near the end of the first engagement session that a quantitative scenario analysis would not fit the skills and preferences of the CDC participants, they communicated a willingness to continue the engagement process. As a result, when the researchers developed the notion of policy/planning analysis through strategy tables, Smalltown CDC participants again willingly engaged the researchers in a wide-ranging discussion of current and potential community initiatives related to our values structure.

5 Analysis and interpretation

We now review the results of the three cases. In the first part of this section, we consider the research questions relating to the value structures of CDCs. In the second section, we consider the questions relating to the use of OR methods.

5.1 CDC values

Research question 1 simply asked which means and ends objectives CDCs identify. For this, we can take the union of the objectives at the different levels as presented in Figs. 3, 4 and 5 in the previous section. Of course, a wider set of cases would produce additional objectives associated with varying local issues, missions and mechanisms of action found within the range of CDCs. We observe that in each case, there were five levels in the objectives hierarchy. The bottom level in each case consisted of the direct result of expenditures or actions taken by CDCs, and the top level in each case represents overall utility. The intermediate levels represent the CDCs views about how they can affect their communities and what aspects of their communities they wish to affect.

To gain insight into the remaining values-related questions, we then identify relations between the objectives, and (subjectively) encode these relations in several steps:

  1. 1.

    Table 2 first lists the objectives from each case. For each objective in each case, we indicate whether the same objective (or essentially the same objective) appears in the objectives of the other cases. In the two columns to the right of each objective, a score of 1 is entered if the objective (or in our judgment objectives that are essentially the same) appears in the case associated with the column, 0.5 if the objective seems to overlap with objectives in that case, and 0 if it does not arise that case at all. The next column sums these scores for an overall commonality score, ranging from 0 to 2.

    Table 2 Similarity of objectives across sites
    Full size table
  2. 2.

    Table 3 compiles the unique objectives across cases, sorted in the first column by their commonality scores in the second column. The next two columns contain researcher judgments about why the objective listed in one case is not present in other cases; the first column is more specific and derived from our knowledge of the cases themselves, and the second column encodes the analysis another level of abstraction up from the primary data.

    Table 3 Reasons for Commonality and Contingency of CDC Objectives
    Full size table

The entries in Table 2 address research question 2––the shared objectives are those with higher commonality scores. These entries also provide part of the answer to research question 3––the contingent objectives are those with lower commonality scores. The entries in Table 3 point toward the rest of the answer––why those objectives are contingent. We refrain from more formal analysis, but add the following qualitative observations about the quantitative results in the tables.

  1. 1.

    Where objectives are not shared, it tends to be due to different foci of activity within realm of CDCs and to a lesser degree due to different missions within the realm of CDCs.

  2. 2.

    Where objectives are shared, it tends to be due to common elements of missions of all CDCs, and also of many CBOs, and to a lesser degree because the objectives relate to activities common to CDCs and their ability to have impact.

To address research question 4 on the complexity of objectives hierarchies, Table 4 tallies statistics on their structures. For each case, we note the number of objectives at each level and the number of observed links between objectives at pair of adjacent levels. The number of potential links between adjacent levels is simply the product of the number of objectives at the lower and higher levels. We define the level of saturation at a pair of adjacent levels as the ratio of observed links to potential links. In two of the cases, there were also several “odd” links between non-adjacent levels.Footnote 2 We calculate a total level of saturation for each hierarchy as the ratio of the number of observed adjacent links summed across levels to the ratio of potential adjacent links summed across levels, and also calculate a saturation level which adds the number of odd links to the numerator.

Table 4 Indicators of complexity of CDC objectives hierarchies
Full size table

We again refrain from formal analysis, and instead offer qualitative observations and speculation on the meaning of the results summarized in Table 4.

The overall number of links is comparable across the three cases, although this may result from the comparable amounts of time spent on their construction, and cognitive limitations involved in working on an entire hierarchy at one time. In two of the cases, Simulated and Smalltown CDCs, the hierarchies were wider in the middle, while in the Urban CDC case, the hierarchy was widest at the lowest levels. If we treat saturation as a proxy for complexity, the high value for Simulated CDC, where each level considers most of the objectives at the level below, may derive from its mission as quasigovernmental and supplementing other city management activities. The intermediate value for Urban CDC may derive from its being embedded in a dense community within a larger city, interacting with various other organizations. If we discount the five-to-three links for Smalltown CDC discussed above, the number of odd links may suggest somewhat higher complexity in Urban CDC––especially as their odd links describe multiple ways performances on different objectives affect each other. The lower value for Smalltown CDC in spite of its wider range of activities may derive from its role in a community where there are fewer major actors and where their actions have more direct effects. If so, complexity in the objectives hierarchies is due more to the degree of connections of the organization with other organizations and of the community with other communities, rather than due to the complexity of its activities.

5.2 CBOR questions

To shed light on questions 5–9, we consider the response to proposed methods and results at each of four decision points (meetings) for Urban and Smalltown CDCs, as described in Table 5. At the first meeting, both organizations elected to work with us to develop an objectives hierarchy, and were not interested in optimization efforts. Urban CDC saw some benefit in optimization, and Smalltown CDC did not. Our impression is that both saw it as hard to use. As we discussed the VFT process, both CDCs acknowledged that it would take commitment to some effort on their part, but also felt confident enough that it would provide sufficient benefits (such as improving internal understanding and consensus about their mission) that they committed to moving forward. This supports our initial hypothesis that value-based methods would appear attractive to CDCs (research question 6). Both organizations were also pleased with the results, finding them interesting, credible, and useful, and the process worthwhile. Hence, both were receptive to considering next steps.

Table 5 CDC responses and decisions on methods
Full size table

At this point, Urban CDC chose to work with us to develop a scoring model, by defining scales and trade-off weights for us to incorporate in a spreadsheet model (after they were shown a mockup of this model). They hoped this model would be useful in their annual planning. They felt that their alternatives for housing acquisition and management were fairly straightforward and could be considered independently of other neighborhood activities where they were only one organization contributing to collective efforts. Smalltown, in contrast, saw the mockup spreadsheet model as more complex than what they were already using, and hence difficult. However, they were eager to build on the initial work and started to suggest ways to use the values to generate alternatives even as we explained this idea. Thus, the differences in organizations skills and scopes of action support the proposed TAM antecedents.

Together with the first (Urban) CDC’s analyst, we presented our scoring model at a next meeting. Staff asked good questions and indicated that they understood it and felt it was credible. At this point, presented with possible next steps including again generating alternatives (about which they remained unenthusiastic), they were open to our analyzing alternatives. They perceived potential benefit––that it might produce insights––but at a modest level. We agreed it would require limited analyst time and no time from senior leadership. The fact that the researchers would carry out most of the effort may have reduced their concerns about ease of use to the point that they were willing to proceed.

Individuals at Smalltown, including the head, were pleased and even excited with initial generation of alternatives. We presented two possible paths forward: (1) develop a scoring model to evaluate alternatives, and (2) take the objective-specific improvement alternatives as raw material for development of broader strategies. Specifically, we proposed to combine these alternatives with other alternatives relating to the specific resources or asset classes they commonly considered. In our prior consulting lives, the trick of combining VFT and strategy tables had produced interesting results. Smalltown felt confident (to the point of highly engaged participation) and ready to have a conversation about strategy and they chose the second option.

After we presented the analysis associated with several potential alternative strategies to Urban CDC, they commented that it was interesting. But they showed no further interest in using such analysis for actual decisions or in pursuing other analytic efforts, as there were many other demands on their time.

In contrast, even as strategies were being developed, the Smalltown CDC was starting to implement parts of them. After we had refined the strategy tables offline and shared them with the head of Smalltown, he remained interested in future efforts with us, although not necessarily as a direct outgrowth of this work. Again, there was not interest in using quantitative models to help select among strategies, and instead, the organization moved ahead on its own toward implementation. This sheds light on question 9, about uses of VFT for CDC strategy.

Overall, the VFT efforts had much more impact on Smalltown than on Urban CDC. In each case when a softer method was used, the results were positive and led to continued enthusiasm. In each case when more quantitative methods were used, results were accepted but enthusiasm was declined. Our research questions thus should be thought of in terms of perceptions prior to experience as well as the impact of that experience on perceptions. While the willingness to try methods was consistent with our hypothesized TAM antecedents (that is, in answer to question 7, will skills and other characteristics affect acceptance prior to exposure), the evaluation of those methods was not contingent (that is, the opposite conclusion for question 7 in terms of ultimate acceptance). From both the initial presentation as well as the later decision points, the answer to question 8 in these cases was that even in conjunction with VFT, the quantitative approaches were not so attractive. In answer to question 5, the qualitative application of VFT had anticipated positive impacts (for articulating and discussing the organizational mission and for formulating strategy) and quantitative analysis did not.

6 Discussion

We now consider this work in perspective, first reflecting upon our experience with VFT in the field and considering links to other applications and disciplines, second considering some caveats to our methods and findings, and finally discussing future extensions of this work.

6.1 Reflection and extrapolation

We infer that for small, mission-driven non-profits, a rigid approach of decision modeling and solution design is not likely appropriate. This suggests that analytics and OR methods most likely to add value to CBOs would be designed using action research and community-engaged research principles. This appears to constitute evidence in support of CBOR as a useful approach in local contexts.

The decision objectives that do not relate specifically to CDC activities may be found in other CBOs, and we hypothesize that within each type of CBOs there will be some common elements associated with their core activities, along with some elements contingent on organizational and community factors that are similar or analogous to those found in CDCs. CBOs may focus on issues such as housing, community capacity, or human services (all of which CDCs may address), as well as arts and culture, education, health, youth, and philanthropy (Boston Foundation 2008). Certain aspects of CBOs or their environments may correlate strongly with type. These aspects include whether they operate in a competitive environment, have missions defined by local residents, have high technical expertise, rely on private and/or government funding, require rapid judgments, or focus on individuals. In extrapolating our results––about both values and process––the degree to which a CBO matches CDCs in these aspects may be critical.

We expect VFT to be useful (and perceived as useful) for most other CBOs and grassroots organizations, as most of their organizational values are influenced by stakeholders in their local communities to whom they aim to be responsive and responsible. There is, however, more variation among CBOs on these dimensions than there was in our study, so this remains speculative. While we did not test a range of PSMs, VFT itself seemed well suited to CDCs. Other PSMs might also be perceived as beneficial in other CBOs, e.g., those with more need for successful prediction or more systematic and less opportunistic allocation of resources, and at least some of VFT does not require high technical skill, although for other parts we expect ease of use to vary among organizations. We expect the better reception of non-quantitative aspects of the PSM to translate in other CBOs, as ease of use is likely to be an issue in organizations with varied and limited quantitative modeling skill.

The results here were consistent with and somewhat supportive of the substantial literature about soft OR for various non-profits and NGOs. For example, Kotiadis and Mingers (2006) list several possible challenges in moving an organization from using soft OR to hard OR methods, including (as we found) practical problems associated with the additional time required, as well as cognitive cultural styles in play. Similarly, the results support positions such as those of Ackoff (1979) and Rosenhead (2006) that soft and participative methods are more likely to be well received in some organizations. The results enrich such arguments by relating use of more specific methods to more specific organizational characteristics.

6.2 Limitations and assessment

This study involved a small number of organizations, one simulated. Not all stakeholders of the organizations participated. For both these reasons, there are likely other CDC objectives to be discovered. The results that are specific to housing would, obviously, not extend to directly to other CBOs. Given these limitations, the client engagements incorporated here for case-based research (Yin 2003) should provide usable data. The search for objectives and discussion of options and perceptions with the CDCs was extensive and included frequent checks for understanding, disconfirming evidence, and alternative sources of information and explanations. The structuring of objectives hierarchies at each CDC was preceded by roughly the same preparation and provision of information for participants, and took roughly the same amount of time, conducted by the same analysts. The analytic measures to describe commonality and complexity, and the characterization of process decision points were developed as a means of triangulation, providing a test of whether the initial findings remain consistent when viewed from multiple angles.

With regard to acceptance of CBOR, we have presented an explanation consistent with the data. This explanation depends on characteristics that tend to hold for CDCs. We are confident in stating that CDCs are relatively receptive to VFT as a qualitative method and less receptive to complex quantitative approaches. Since many organizations are receptive to quantitative approaches, we certainly cannot extrapolate this finding to all organizations. To the extent that other CBOs share relevant characteristics with CDCs, we expect they would display similar patterns. In the paper, we speculate about which characteristics are relevant in this regard, but this study does not include variation that would allow us to identify them. We do not believe the results here would necessarily extend beyond CBOs.

For example, Airoldi et al. (2011) effort on behalf of a UK governmental organization (National Health Service) found high benefit from their accessible quantitative analysis. Study participants were citizens who had preferences about, but not expectation that they had direct influence upon, health care policies. Thus, the participants may have had more interest in understanding the impacts of trade-offs, rather than ensuring that considerations were on the table, while the NHS as a large agency had already defined objective for, in this case, quality of life. Thus, in an organization that is quite different from CDCs, successful use of OR was achieved in a different way.

6.3 Future research

This line of inquiry can be extended in several directions. Larger and more controlled studies would help to confirm our preliminary findings and validate measures. Within the CDC/housing domain, our future work will aim to reconcile the objectives identified here with quantifiable foreclosure acquisition value models from other paradigms, e.g., strategic value (Johnson et al. 2012) and property value impact (Johnson et al. 2013). As discussed above, adapting these results to other CBOs would require more detail, identifying additional objectives for other types of CBOs and enriching the technology acceptance story by accounting for their organizational characteristics and uses. In addition, drivers of CBO acceptance of other PSMs merit study. Finally, a more extensive study would allow comparison of CBOR to OR for other whole classes of organizations such as grassroots organizations, service delivery and humanitarian NGOs, and governments. Each of these has its own tendencies on dimensions such as of size, problem difficulty, expertise, data, local and social focus.

6.4 Summary

In our cases developing OR models for CDCs, we found that the organizations were more receptive to non-quantitative approaches than quantitative approaches––regardless of technical capability (contrary to our expectation). We found CDCs appropriate for and receptive to value-focused thinking. The objectives hierarchies for three CDCs provide a menu of core and contingent objectives for other CDCs to incorporate in their own work, and some of both types of objectives ought to be appropriate for other CBOs. More generally, the nature of these organizations and their communities affects their acceptance and use of different OR methods and the content produced by those methods.

The use of objectives hierarchies as data as well as measures used to analyze that data are a novel way to compare decision models. Also novel is the use of project decision points as data to map technology acceptance across multi-site community engagements. This paper supplements current literature on CDCs by bringing a new approach to articulating values for strategy, and by identifying objectives. It supplements the empirical literature on soft OR, PSMs and the practice of decision aiding by testing the degree to which soft and hard methods, as well as value-oriented methods, work in the little explored context of CBOs.