1 Introduction

By 2050, the United Nations projects that 68% of the world population will live in cities (UN DESA 2019). However, with continuous population growth, the number of people living in rural areas of many low- and middle-income countries (LMICs) will continue to rise. Two-thirds of the extreme poor live in rural areas (World Bank 2016), and the livelihoods of two to three billion rural people, often the most food insecure and vulnerable, still depend primarily on small farms (Laborde Debucquet et al. 2020; Woodhill et al. 2020).

There are various estimates of the number of small farms in the world, but they all suggest that these farms are numerous. Lowder et al. (2016) used agricultural census data from 167 countries to estimate that, of the total 570 millionFootnote 1 farms in the world, 475 million have less than 2 hectares (ha), dominating agriculture in most LMICs, where farm sizes continue to fall. Africa south of the Sahara has the highest rural population growth rate globally, and thus the number of small farms is expected to increase more than in other regions. Africa’s share of total world rural poverty is also expected to rise from 39.6% in 2015 to 58.1% in 2050 (Thurlow et al. 2019). Transforming Africa’s agriculture sector is thus a priority embodied in the Malabo Declaration on Accelerated Agricultural Growth and Transformation for Shared Prosperity and Improved Livelihoods (AU 2014). However, to meet the Malabo goals and achieve multiple SDGs in all LMICs by 2030, creating an enabling environment where small farms are included in and benefit from rapid growth and transformation of agrifood systems is urgent (Barrett et al. 2020a).

Small farms not only contribute to feeding the households that operate them, but also make two broader contributions. First, small farms are important to the overall food security of LMICs. Samberg et al. (2016) noted that farms less than 5 ha are responsible for 53% of the global production of food calories for human consumption. Herrero et al. (2017) reported that, in Africa and South and Southeast Asia, small farms with less than 2 ha produce around 30% of food and make valuable contributions to micronutrient-rich food production. Ricciardi et al. (2018) estimated that farms under 2 ha globally produce 30–34% of the food supply. Nonetheless, small farm households themselves are often unable to afford a nutritious diet.

Second, small farms contribute to the sustainability of agrifood systems by maintaining the genetic diversity of crops and livestock and supporting ecosystem services. Small farms have more crop diversity and harbor greater non-crop biodiversity at the farm and landscape scales than do larger farms (Ricciardi et al. 2021). Subsistence-oriented small farmers plant a greater diversity of traditional crops and maintain genetic resources by cultivating land races (Fifanou et al. 2011; McCord et al. 2015). Small fields have more edges than larger fields, creating a heterogeneous landscape and providing habitat for non-crop species (Ouin and Burel 2002). To the extent that small farms have more tree cover than larger farms, they provide above- and below-ground carbon storage, with global benefits for climate mitigation (Ritchie and Roser 2017). Trees on farms can also improve water infiltration, a hydrological service that benefits other water users in the landscape and downstream (Anache et al. 2019).

For small farms to be part of inclusive and sustainable agrifood system transformation, both innovative technology and market institutions are required to support LMICs’ diverse agroecological and socioeconomic contexts. Many debates on the future of small farms focus only on farm production, rather than the whole context of farm household livelihoods, which include off-farm activities, or the agrifood system on which farms depend for buying inputs and selling outputs (Reardon et al. 2019; Giller et al. 2020). The future of small farms should instead be assessed using a holistic livelihoods and agrifood system lens.

2 Who Will be Small Farmers in the Future?

More than 410 million farms are very small, with less than 1 ha of land, and another 70 million are between 1 and 2 ha (Lowder et al. 2016). However, discussions of farm size often ignore land quality considerations (Eastwood et al. 2010). For example, a 5 ha farm in a rainfed zone with poor quality soil may support less production than a 1 ha farm in an irrigated zone with good soil. Thus, mere farm size ranges tell us nothing about differences in agroecological land quality, or about the socioeconomic contexts in which they operate, such as market and infrastructural conditions (FAO 2014; Graeub et al. 2016). While the product mix of small farms varies depending on this context, many are diversifying that mix, driven by urbanization, consumers’ dietary preferences, technology, infrastructure development, and rural-urban links. Moreover, households that operate small farms tend to have diversified income sources, including non-farm activities, and that diversification is expected to increase over time, although at different rates among different sets of small farmers (Davis et al. 2017).

Despite the strong heterogeneity across small farms, they can be categorized in ways that make our analysis more tractable. Following Vorley (2002) and Hazell (2018), and based primarily on Hazell (2020), we classify small farmers in LMICs into three groups.

Commercial small farmers run their farms as businesses. While commercial agriculture is an important source of income for them, many also undertake rural non-farm employment (RNFE). Most commercial small farmers do not specialize in high-value crops or livestock, as many also produce food crops. Their product and activity mix are conditioned by agroecological circumstances, urban market proximity, rural infrastructure, and the agro-processors, logistics, exporters, and wholesale enterprise investment and density in their area. Climate change and economic transformation also condition their farm businesses and will create new challenges and opportunities even over the next 10 years. Some commercial small farms will continue to focus on today’s traditional export crops—for example, cocoa in Ghana, cotton in Mali, and coffee in Ethiopia—while increasing numbers will turn to products that cater to the diversifying diets of burgeoning domestic urban markets, including fruits, vegetables, fish, poultry, edible oils, milk, and feed grains such as soy. Non-cereal products are especially labor-demanding and often offer little or no economies of scale, allowing small farms to be competitive. Over time, we expect to see greater specialization in the farming of high-value products and a movement away from the combination of cash and staple crop farming, similar to what one sees among specialized vegetable farmers in the Shandong province of China (Huang et al. 2010) or specialized poultry and pig farmers near Yangon in Myanmar (Belton et al. 2020).

Small farmers in transition often depend heavily on RNFE while also maintaining small plots for home food consumption, plus some semi-commercialized food or non-food products. They tend to buy a substantial share of their food. These farmers are in zones where favorable non-farm opportunities exist locally or in near-by towns. With demand growing for high-value farm products in cities, some transitional farmers will commercialize their small farms while continuing their RNFE. However, others may exit agriculture or maintain just small food plots because access to food markets in their area is uncertain, or because the RNFE labor market itself is uncertain or limited (de Janvry and Sadoulet 2006). Thus, many small farmers in this group will continue to have one foot in farming and one foot in RNFE as their major sources of income, and their number is expected to remain large over the next decade.

Subsistence-oriented small farmers are marginalized for a variety of reasons, many of which will be difficult to change in the next decade, such as ethnic discrimination, sickness, age, or their farm’s location in a remote area with limited agricultural potential. We expect the number of these small farms to fall with economic transformation, but it is unrealistic to expect most will disappear in the next decade. These farm households tend to undertake some RNFE or farm wage labor (usually the domain of the poorest farmers or the landless), but many of the same factors that constrain their farming also prevent them from undertaking remunerative RNFE to become transition farmers. These subsistence-oriented farmers are typically net buyers of staple foods. While market and technology development will help them improve farm productivity, the above constraints limit even this. They need social protection policies and other public support beyond what the agrifood system and rural labor market can provide.

RNFE is an important income source for rural small farm households and, on average, occupies more of their working time than farming in many African and Asian LMICs (Dolislager et al. 2020). For commercialized and transition small farmers, who are often in places with favorable agroclimates and adequate infrastructure, RNFE helps fund farming by providing cash or collateral for credit to buy inputs and diversifying income risk from agriculture. This can incentivize experimentation with new production technologies and riskier products like vegetables, poultry, and fish that have higher values. Increases in local RNFE activities often lead to rising rural wages (Lanjouw and Murgai 2009), which can induce the adoption of mechanization (Wang et al. 2016). However, in less favorable agroclimatic zones or hinterland areas, where most subsistence-oriented small farmers are located, RNFE is used mainly to fund food purchases and competes with, but also compensates for, unprofitable farming (Davis et al. 2009).

3 Innovations for the Future of Small Farms

The future of small farms will depend on technological and institutional innovations that are now appearing in some developed and developing country contexts or have yet to be developed (Herrero et al. 2020, 2021). Technological innovations have the potential to benefit small farms in LMICs, but ensuring their appropriateness remains a challenge. High transaction costs, lack of collective action, and failures in production and marketing coordination all introduce risks for small farms and are commonly seen as barriers to adopting modern technologies and participating in value chains. Many subsistence farmers may be too remote from markets or lack the capacity to benefit from new technologies. Transition farmers can be disincentivized from adopting new technologies if they are labor-intensive and compete with their non-farm employment. Even for commercial small farmers, the adoption of new technologies requires enabling conditions from output and input supply chains. Small farmers’ adoption of new technologies and the cultivation of higher-value products thus requires that they have the proper profit incentives and market access, which are, in large part, a function of the broad market institutional context. Effective market institutions require improved infrastructure that facilitates input supply chains upstream from the farm and connects small farmers to cities downstream from their farms.

Downstream from the farm, output market conditions affect small farmers’ prices, risk, and transaction costs. Critical factors include urban market size and proximity; the density and quality of roads between farmers and markets; and the midstream (wholesalers, logistics firms, and processors) and downstream (retailers) accessibility to and conduct toward small farmers. Developments in these enabling conditions in LMICs are themselves local innovations, which often rapidly improve market access for small farmers, as in the examples from Ethiopia, Nigeria, and India discussed below. Changes in these conditions will continue to be the main factor affecting small farmers’ technology adoption, income growth, and inclusion in agrifood system transformation in the next decade. Some emerging technologies, such as e-commerce linked to digitalization, are also promising innovative market institutions that will impact the relationship between small farmers and markets in the next few decades.

The urban market now makes up the largest share of national food consumption in LMICs (Reardon et al. 2019, 2021a, b). Proximity to urban markets in primary and secondary cities and small towns asserts a strong influence on market conditions and the technology and product choices of small farmers (Vandercasteelen et al. 2018). Highways and rural roads connecting farmers to urban markets likewise are critical to small farmers’ access to these booming urban markets, suggesting the importance of public investment in rural infrastructure (Stifel et al. 2016).

The combination of growing urban markets, expanding road connections, and the development of wholesale markets provides favorable conditions for the spontaneous formation of clusters of wholesalers, cold storages, processors, and logistics enterprises that provide crucial services enabling small farmers to access urban markets. The emergence of clusters of small and medium enterprises (SMEs) offering potato cold storages in Bihar, India, is a good example; these have allowed small farmers to store their produce and wait for much higher prices in the off-season (Minten et al. 2014). In Ethiopia, the spontaneous development of a teff value chain connecting rural areas to Addis Ababa has been facilitated by the growth of midstream private SMEs utilizing public infrastructure and improvements in wholesale markets. Midstream market development also spurred the adoption of new technology and a new teff variety by small farmers (Minten et al. 2016). Many thousands of small chicken farmers in Nigeria, mostly women, benefited from the rapid growth of long north–south maize supply chains, operated by thousands of SME wholesalers and feed millers, to market their chicken and eggs in towns and secondary cities (Liverpool-Tasie et al. 2017). Spontaneous clusters of traders and input suppliers are also seen in aquaculture districts of Bangladesh and are a key determinant of small farmer technology adoption (Hu et al. 2019).

The relations of supply chain firms with small farmers are a critical determinant of small farmers’ participation in markets for high-value agricultural products. These firms not only buy from small farms, but also often provide resources and services that small farmers need to participate in the market, from inputs and credit allowing them to adopt new technologies that meet market requirements to services such as aggregating, sorting, and packing. This facilitation is offered through formal contract-farming arrangements with large processors and retailers (Swinnen and Kuijpers 2019), as well as through informal relationships with SME wholesalers and processors that reduce the price risk for small farms (Liverpool-Tasie et al. 2020). Relative to the “traditional” arrangement of spot markets, this facilitation can be broadly seen as a market institution innovation, especially in the poorer LMICs. We expect these relationships to expand over the next decade as the double-pronged food system revolution continues its rapid course, with both the proliferation of SMEs and of modern large-scale firms underpinning the growth of rural-urban supply chains (Reardon et al. 2019).

Despite still being in its infancy in LMICs, e-commerce (marketing online) and e-procurement (buying intermediate inputs online) are emerging rapidly. The diffusion of Internet access, mobile phones, and computers helps the spread of “delivery intermediaries,” whose expansion has been particularly rapid during the COVID-19 pandemic as consumers tried to avoid in-person shopping (Reardon and Swinnen 2020). COVID-19 accelerated e-commerce growth, for example, from 30% to 70% per year in India, 10% to 20% in China, and 20% to 50% in Nigeria (Vardhan 2020). The benefits of e-commerce for small farmers will depend on three conditions. First, widespread access to e-commerce will depend on mobile phone rates and Internet costs, which currently are particularly high in Africa (Torero 2019). Second, while e-commerce can make it easier for small farmers to sell to urban markets, their costs and product quality must still be competitive with medium and large farmers and importers. Small farmers linked to e-commerce may be better able to compete in more proximate niche markets. Third, e-commerce as digitalization per se only informs a buyer of a seller and a seller of a buyer; the final transaction still relies on delivery intermediaries, roads, and logistics, and the same high transaction costs that have constrained the development of non-digitized supply chains will constrain large numbers of small farmers from participating in e-commerce.

Encouragingly, there are interesting examples of e-commerce that are inclusive of small farmers with potential to spread in the future, depending on the three conditions noted above. In Indonesia, the Rumah Sayur Group, a vegetable farm co-op with 2500 farmers, sold to supermarkets, wet markets, and food-service businesses in Jakarta before the pandemic. During the pandemic, they turned to Alibaba’s Lazada to sell directly to consumers and retailers. In Malaysia, Lazada connected SME flower suppliers to online florists to gain a new customer base when COVID-19-related restrictions interrupted the traditional marketing system. In Africa, Facebook and other e-platforms have helped small farmers sell directly to consumers. Examples include Koop direk von boer (buy directly from the farmer), a Facebook group of farmers created in May 2020 that attracted 46,000 members across South Africa in just 2 weeks (Masiwa 2020).

Upstream from the farm, market conditions affect the input prices, risk, and transaction costs facing small farmers, just as the output market affects the profitability of adopting new farm technologies and the transition to higher-value products, as do input supply chains. Importantly, input market conditions are parallel to output market conditions, affected by many of the same policies and public investments discussed in the context of downstream factors. Again, the development of these conditions is a local innovation. Changes in these conditions can rapidly improve input market access for small farmers, spurring technology change at the farm level.

Some particularly interesting market institutions and technological innovations in agricultural service markets appear to be helping small farmers. We characterize them as the development of mobile “outsource” services. They include a wide range of services available to farmers on a fee basis. For an individual small farmer, the outlays of capital for machines required would not be affordable given their small scale and the large lump-sum fixed cost for machinery. In the early 1880s, such on-demand operational services emerged in the United States and European countries, where large farmers dominated. Small farmer demand for mechanization and agricultural operational services has risen in recent years in LMICs, first in Asia and Latin America and, more recently, in Africa. These services, perhaps especially as they are facilitated by communications innovations, appear to provide important support to small farming technological change. In general, mobile technology can help service supply and extension reach widely dispersed small farmers (Van Campenhout et al. 2021). For example, mobile mechanization services for land preparation, harvesting, and threshing are hired by many small farmers in South and Southeast Asia (Zhang et al. 2017; Paudel et al. 2019; Diao et al. 2020; Yagura 2020; Belton et al. 2021). They are increasingly accessible for small farmers in Africa (Berhane et al. 2016; Kahan et al. 2018; Takeshima 2018; Diao et al. 2020; Cabral 2021). Mobile phones are widely used for connecting service providers and small farmers, and new digital platforms appear to have potential to reach groups of small farmers. Examples include Hello Tractor in Nigeria, TroTro Tractor in Ghana, Rent to Own in Zambia, and EM3, Trringo, and farMart in India (Birner et al. 2021; Daum et al. 2020).

Moreover, other SME services are emerging in various agricultural operations traditionally carried out by small farmers themselves, such as for rice seeding and transplanting in southern China (Li et al. 2015; Gong et al. 2012); spraying, pruning, land preparation, harvesting, and marketing for mango farmers in Indonesia (Qanti et al. 2017); seed propagation, digging wells and ponds, spraying, and loading trucks for vegetable farmers in Ethiopia (Minten et al. 2020); and bee pollination services for vegetable and fruit growers throughout China (Altay News 2019). Many of these services have replaced labor-intensive farming activities with machines or specialized techniques, helping small farmers who lack the cash to invest in machines, the skills to use machines and other techniques, or simply the time to spend farming because of non-farm employment. These services also introduce small farmers to new technologies that they otherwise might have been unaware of had they not been provided as part of a package of services by SMEs, such as flower hormone use to extend the harvesting of mangoes in Indonesia (Qanti et al. 2017).

New institutional innovations can also benefit small farmers through contributions to sustainable land stewardship. Market-based institutions that incentivize farmers to maintain ecosystem services and biodiversity have been used for over a decade. With payments for ecosystem services (PES), the private or public sector pays land stewards (farmers) to protect watersheds, sequester carbon through tree planting, or conserve biodiversity (Milder et al. 2010). In the case of carbon, for example, the institution providing payments receives offset credits in the voluntary or regulatory carbon market. Another scheme involves certification of agricultural commodities, such as coffee, palm oil, and cacao. Certification schemes are generally implemented by non-governmental organizations (NGOs) and rely on consumers paying a premium for production practices that conform to sustainable social and environmental goals (Brandi et al. 2015; Giovannucci and Ponte 2005; Ruysschaert and Salles 2014). Smallholder farmers have benefited from these schemes only to a modest degree due to high transaction costs, low demand for ecosystem services, and poor access to information.

For carbon markets, smallholder participation is impeded by the required technical capacity, as well as the costs of monitoring and complex requirements for reporting (Brandi et al. 2015; Wells et al. 2017). With certification schemes, evidence indicates mixed success for environmental, social, and economic goals. The supply of certified products is generally larger than the demand (DeFries et al. 2017). Insecure land tenure, lack of credit, and insufficient profit to warrant the required investments hamper smallholder participation in both PES and certification schemes.

With rising recognition of the importance of land stewardship for climate mitigation and conservation of biodiversity, institutions to incentivize protection of ecosystem services and sustainability goals are likely to become more widespread in the coming decades. Carbon markets, which, to date, have largely been unable to stem land clearing and greenhouse-gas-emitting practices on agricultural land, will likely be a more significant driver of farmers’ decisions in the future. In combination with digital technology, institutional innovations have the potential to reduce transaction costs and enable participation by smallholders to maximize their ability to benefit from these schemes, both to boost their incomes and to contribute to society’s sustainability goals. Technology and training for smallholders to access and interpret satellite data, monitor their lands, and fulfill reporting requirements are needed if they are to benefit from a growing demand for ecosystem services.

4 Policies for Inclusive Small Farm Transformation Through Innovation

This chapter has sought to imagine the future of small farms and identify promising innovations in agrifood systems to improve their prospects over the next 10 years. Because small farms are heterogeneous and dynamic, we classed them into three groups: commercial, in-transition, and subsistence-oriented small farms. Each has its own set of challenges and opportunities, and policies and investments that prioritize inclusive small farm transformation must be differentiated to best target the needs of each group as agrifood systems evolve (Hazell 2020).

Commercial small farmers are the vanguard of agrifood transformation and best prepared to take advantage of the opportunities that growing market demand for agrifood products will create. They tend to be located in more favorable agroclimates, nearer to cities and towns, and in areas better served by infrastructure and midstream SMEs that facilitate input and output markets. These same market opportunities will incentivize some transitional farmers to invest in their small farms and become commercial farmers. To enhance small commercial and transitional farmers’ competitiveness to pursue these market opportunities, the following government policies and public investments are important:

  • Increase investments in infrastructure, including rural roads connecting to secondary and tertiary cities, that can create economies of agglomeration and a critical mass of proximate services such as wholesale, logistics, and farm input provision for small farmers in the surrounding rural areas, thus reducing transaction costs. Often, mobile agricultural services are clustered in towns and fan out to serve small farms in a hub-and-spoke model (Zhang et al. 2017). Many new digital technologies applied in e-commerce, information provision, and farm service businesses also depend on good infrastructure. While initial investments need to come from governments, they will serve to lure in private investments from both large companies and SMEs.

  • Promote education and training programs that target rural youths to develop the skills and knowledge required to support modern agriculture and marketing. These skills are necessary for both farm management and off-farm jobs in logistics, machinery maintenance/repair services, and broader RNFE.

  • Facilitate co-operatives and farmer groups that can collectively pursue emerging opportunities in urban markets and modern farm technology. Local networks can also be strengthened through village-level innovation platforms to link smallholder farmers with extension and research, such as China’s Science and Technology Backyard (Barrett et al. 2020a). These show promise for drawing together the wisdom of (small farmer) crowds and the knowledge of cutting-edge scientific researchers to accelerate discovery, adaptation, and diffusion (Nelson 2019; van Etten et al. 2019).

  • Support SMEs upstream and downstream from farms by reducing unnecessary regulations and informal restrictions that often discourage SME development. SMEs are more accessible to small farmers than larger enterprises, and small farmers value the mix of services that SMEs provide (Liverpool-Tasie et al. 2020).

RNFE is the main economic activity of transitional farmers and is increasingly the main source of income for most small farmers. RNFE provides small farmers with cash, both to purchase food and for farm investments to raise productivity, expand commercial activities, and produce higher-value products. RNFE is also important for some marginalized farmers, helping them reduce their reliance on risky, low-yield agriculture. For these farmers, RNFE development will directly improve food security in a way that marginally boosting agricultural production cannot (ZEF and FAO 2020; Frelat et al. 2016). Public investments and policies that facilitate growth of the agrifood system must pay more attention to creating enabling environments for the development of RNFE and strengthening the synergy between agriculture and RNFE in rural areas. In this regard, the following actions are promising for governments to actively promote agriculture–RNFE synergies for rural development and agrifood system transformation:

  • Pursue policies that have broad effects across economic activities in rural areas and do not limit interventions to farming alone. RNFE and farming are complementary, and both are needed for inclusive growth in rural areas.

  • Develop an enabling environment—including basic infrastructure, property rights, and legal systems with enforcement mechanisms—favorable to rural businesses that encourage and facilitate inclusive RNFE (Haggblade et al. 2007).

  • Identify engines of regional growth through consultation with the private sector and farmers, and conduct supply chain diagnostics for prioritization of strategic interventions (Haggblade et al. 2007). Emphasize differentiated strategies and flexible institutional coalitions for implementation appropriate to diverse rural areas.

This chapter emphasizes the importance of market institution innovations for achieving higher agricultural productivity and quality through small farm technology adoption and improving incomes for small farm households through participation in both farm and non-farm economic activities. In addition to the policy recommendations discussed above, some additional policy recommendations are listed here, although adapting and differentiating policies over heterogeneous contexts across LMICs requires context-specific research and consultation with stakeholders (Barrett et al. 2020b):

  • Support new technologies that reduce risk and are attractive to small farmers when viewed in a holistic way, taking into account farmers’ resource environment, as well as their livelihood strategies. Do not automatically assume labor-intensive innovations are appropriate for small farmers, who often want to reduce, not intensify, their farm labor use (Hazell 2020). For transitional farmers who depend on RNFE, proposing new labor-intensive farming activities could fail if they cut into the time farmers have available for RNFE livelihood strategies (Moser and Barrett 2006).

  • Ensure that agricultural interventions to support sustainable farming practices are economically viable for farmers and provide direct economic benefits. In the longer term, farmers are most strongly motivated to adopt and maintain sustainable practices when they perceive positive outcomes of these practices for their farm or the environment (Piñeiro et al. 2020).

  • Scale up productive social protection programs for subsistence farmers in hinterland areas who face barriers in accessing markets and other economic opportunities. Safety net programs ease liquidity constraints and increase tolerance for risk among small farms and, when integrated with measures to increase agricultural productivity, have the potential to make significant progress toward the eradication of hunger (Wouterse et al. 2020).