Software development with feature toggles: practices used by practitioners

Abstract

Background

Using feature toggles is a technique that allows developers to either turn a feature on or off with a variable in a conditional statement. Feature toggles are increasingly used by software companies to facilitate continuous integration and continuous delivery. However, using feature toggles inappropriately may cause problems which can have a severe impact, such as code complexity, dead code, and system failure. For example, the erroneous repurposing of an old feature toggle caused Knight Capital Group, an American global financial services firm, to go bankrupt due to the implications of the resultant incorrect system behavior.

Aim

The goal of this research project is to aid software practitioners in the use of practices to support software development with feature toggles through an empirical study of feature toggle practice usage by practitioners.

Method

We conducted a qualitative analysis of 99 artifacts from the grey literature and 10 peer-reviewed papers about feature toggles. We conducted a survey of practitioners from 38 companies.

Results

We identified 17 practices in 4 categories: Management practices, Initialization practices, Implementation practices, and Clean-up practices. We observed that all of the survey respondents use a dedicated tool to create and manage feature toggles in their code. Documenting feature toggle’s metadata, setting up the default value for feature toggles, and logging the changes made on feature toggles are also frequently-observed practices.

Conclusions

The feature toggle development practices discovered and enumerated in this work can help practitioners more effectively use feature toggles. This work can enable future mining of code repositories to automatically identify feature toggle practices.

Appendix

The survey questions are as follows:

• 1. What is your company name?

• 2. How long has your team used feature toggles?

• 3. What feature toggle management system is used by your team? (Check all that apply). Options: Closed source custom system maintained by the company; Open source custom system maintained by the company; Third party (e.g. LaunchDarkly), Open source but not maintained by the company; None; Other.

• 4. For what purpose(s) does your team use feature toggles? (Check all that apply). Options: Support CI of partially-completed features; Dark launches; A/B testing; Gradual rollout; Other.

• 5. Does your team make decision about using feature toggles for each feature? Options: Yes. The team checks to find if using a feature toggle is necessary for the new feature; No. The feature toggle is always added when a new feature is added; Other.

• 6. How often does your team do the following management practices? Options: Always, Mostly, About half of the time, Rarely, and Never.

  • Document feature toggle’s metadata (spreadsheet, etc) to manage data about feature toggles. (i.e. the owner of the toggle, the current value (on, off), the current status (to remove, keep) and the time of its creation).

  • Logging changes to toggle values/configurations (e.g. who changes which toggle and when, etc.).

  • Grouping toggles together in any way to simplify management or giving permissions (i.e. related toggles, other).

  • Allowing all team members (i.e. Q&A team) to have access to feature toggles and can make changes.

• 7. How often does your team do the following initialization practices? Options: Always, Mostly, About half of the time, Rarely and Never.

  • Determining the type (permission toggle, ops toggle, release toggle, experiment toggle, short-lived toggle, long-lived toggle) of the toggle at design step. (More information about types of toggles: https://goo.gl/4okG5Y)

  • Using naming conventions for toggles (similar to variable and function naming conventions).

  • Setting up a default value for toggle if toggle value is not found (i.e. toggle is off if its value is not found in the code).

• 8. How are the values of the toggles stored? (Check all that apply) Options: Configuration files; Databases; Other.

• 9. How are the values are assigned to the toggles in the system? (Check all that apply) Options: Assigned boolean values (True, False); Assigned multivariate values (e.g. Red, Yellow, Blue); Assigned string values (e.g. “disable-flash-3d”, “enabled-flash-3d”); Other.

• 10. How does a developer access the toggle value in the code? (Check all that apply) Options: Value is accessed by checking a primitive data type (e.g. enableMyFeature == true),; Value is accessed through an object representing a toggle (e.g. MyFeature.isActive()); Value is accessed through a toggle manager/mapping from key to value (e.g. Dictionary); Other.

• 11. How often does your team do the following clean-up practices? Options: Always, Mostly, About half of the time, Rarely and Never.

  • Limiting the number of existing toggles in the code.

  • Build or test failing if a toggle is not deleted by a specified date (Time bomb).

  • Automatic reminders near date to delete the toggle.

  • Using tasks/stories/cards for removing toggles.

  • Creating a clean-up branch for removing toggle points at the time of creation of the toggle.

  • Tracking unused toggles for removal.

  • Changing feature toggle to configuration setting to keep it in the code.