feat: migrate experiments config from array to bools by mwbrooks · Pull Request #398 · slackapi/slack-cli

mwbrooks · 2026-03-14T05:58:28Z

Changelog

N/A

Summary

This pull request migrates the experiments config format from an array to a map of boolean values. It includes backwards-compatible JSON unmarshaling that will auto-migrate config.json files.

Example

Before:

{
  "experiments": ["sandboxes"],
  "manifest": {
    "source": "local"
  },
  "project_id": "..."
}

After:

{
  "experiments": {
    "sandboxes": true,
    "charm" :false
  },
  "manifest": {
    "source": "local"
  },
  "project_id": "..."
}

Motivation

Why on earth are you doing this? 👨🏻‍🎨 🎨 🚲 🏠

The motivation is to unblock adding a slack config command to get and set system-level and project-level config files.

The config command will use a dot-notation to get and set key values:

$ slack config get manifest.source       # Getter
$ slack config set manifest.source local # Setter

When the <value> is a simple type, it works well, such as string, number, or boolean

$ slack config set manifest.source "local"  # String
$ slack config set some.timeout 10000       # Number
$ slack config set some.toggle false        # Boolean

When the <value> is a complex type, it's not intuitive or simple, such as arrays

# Which format do we support? It will require regex parsing in most cases
$ slack config set experiments experiment1, experiment2, experiment3
$ slack config set experiments ["experiment1", "experiment2", "experiment3"]
$ slack config set experiments ['experiment1', 'experiment2', 'experiment3']

# Does this append experiment1 to the array or replace the array with experiment1?
$ slack config set experiments experiment1

In order for setting [value] to be simple and intuitive, we should restrict the config.json files to use simple value types that can be expressed on the command-line (e.g. string, number, and boolean). Arrays are complex and error-prone to express (["value1", "value2"] or value1, value2) and all elements must be included each time.

This is why this pull request migrates the experiments to an object of key: boolean pairs.

An added bonus is that the new format naturally supports disabling experiments from the config file. 🧠

Requirements

I've read and understood the Contributing Guidelines and have done my best effort to follow them.
I've read and agree to the Code of Conduct.

Change the experiments field in SystemConfig, ProjectConfig, and Config from []experiment.Experiment to map[string]bool. This enables dot-notation access for the upcoming config command and allows explicit enable/disable of experiments. Includes backwards-compatible JSON unmarshaling that auto-converts the old array format (["charm"]) to the new map format ({"charm": true}). Updates help output to show all available experiments with ENABLED/DISABLED status for better discoverability.

Reverts the EXPERIMENTS section changes from the experiments migration commit to keep PR #1 focused on the array-to-map refactor. Help menu improvements will be done in a separate PR.

codecov · 2026-03-14T06:01:24Z

Codecov Report

❌ Patch coverage is 94.23077% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.91%. Comparing base (e47334c) to head (c500bde).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/config/experiments.go	92.85%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #398      +/-   ##
==========================================
+ Coverage   67.87%   67.91%   +0.03%     
==========================================
  Files         218      217       -1     
  Lines       18050    18044       -6     
==========================================
+ Hits        12252    12254       +2     
+ Misses       4642     4637       -5     
+ Partials     1156     1153       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

zimeg

@mwbrooks LGTM! This is an amazing change for improved configurations overall - too often I changed "bolt" to "bolts" for confident testing ⚡ 🏁

I'm leaving a few questions related to the order of detecting experiments as well as question about typings. I also don't think we should preserve backward compatibilities for experiments since that adds meaningful logic otherwise but no blockers 🎆

zimeg · 2026-03-16T19:11:41Z

internal/config/config.go


 	// Feature experiments
 	ExperimentsFlag []string
-	experiments     []experiment.Experiment


🪬 question: Can we continue to use strict types of experiment or is this not allowed as a key?

experiments map[experiment.Experiment]bool

@zimeg Great suggestion and I should have caught that myself. I think we can use map[experiment.Experiment]bool, so let me look into refactoring it 💻

Commit a70d3ee uses the strict types:

experiments map[experiment.Experiment]bool

A helper method had to be created to copy from the JSON format to the strict type.

zimeg · 2026-03-16T19:15:46Z

internal/config/experiments.go

 	// Load from flags
 	for _, flagValue := range c.ExperimentsFlag {
-		experiments = append(experiments, experiment.Experiment(flagValue))
+		experiments[flagValue] = true
 	}


👁️‍🗨️ thought: The order of experiment parsing might need to be reversed so that project and user configurations can be overridden with a flag if the lower settings use a "false" value?

@zimeg 🙇🏻 Very good catch! Commit f0c8bc3 updates the order to be:

Permanently enabled

System config

Project config

Flags

question: Not sure where Permanently enabled should be. I feel we would be better served to rename it to "Enabled by Default" and allow developers to opt-out in case there is a known bug. If we keep it as Permanently enabled, then we may want it to be after Flags to override all other settings.

zimeg · 2026-03-16T19:17:52Z

internal/config/experiments_test.go

-		jsonContents := []byte(fmt.Sprintf("{\"last_update_checked_at\":\"2023-05-11T15:41:07.799619-07:00\",\"experiments\":[\"%s\"]}", validExperiment))
+		jsonContents := []byte(fmt.Sprintf(`{"last_update_checked_at":"2023-05-11T15:41:07.799619-07:00","experiments":{"%s":true}}`, validExperiment))


⭐ praise: Thanks for improving both the string formatting and experiments formatting here!

zimeg · 2026-03-16T19:18:52Z

internal/config/experiments_test.go

+	t.Run("Correctly finds experiments from old array format in config.json", func(t *testing.T) {
+		// Setup
+		ctx, fs, _, config, pathToConfigJSON, teardown := setup(t)
+		defer teardown(t)
+		_, mockPrintDebug := setupMockPrintDebug()
+
+		// Write old array format
+		jsonContents := []byte(
+			fmt.Sprintf(
+				`{"last_update_checked_at":"2023-05-11T15:41:07.799619-07:00","experiments":["%s"]}`,
+				validExperiment,
+			),
+		)
+		_ = afero.WriteFile(fs, pathToConfigJSON, jsonContents, 0600)
+
+		config.LoadExperiments(ctx, mockPrintDebug)
+		experimentOn := config.WithExperimentOn(validExperiment)
+		assert.Equal(t, true, experimentOn)
+	})
+


Suggested change

t.Run("Correctly finds experiments from old array format in config.json", func(t *testing.T) {

// Setup

ctx, fs, _, config, pathToConfigJSON, teardown := setup(t)

defer teardown(t)

_, mockPrintDebug := setupMockPrintDebug()

// Write old array format

jsonContents := []byte(

fmt.Sprintf(

`{"last_update_checked_at":"2023-05-11T15:41:07.799619-07:00","experiments":["%s"]}`,

validExperiment,

),

)

_ = afero.WriteFile(fs, pathToConfigJSON, jsonContents, 0600)

config.LoadExperiments(ctx, mockPrintDebug)

experimentOn := config.WithExperimentOn(validExperiment)

assert.Equal(t, true, experimentOn)

})

🪓 quibble(non-blocking): We might not fear breaking experimental configuration setups since this can become burdensome to keep in mind ongoing?

thought: Good question. Here's what I've found from a little searching:

No documentation exists for configuring experiments with config.json.

No sample apps use experiments.

So, if a developer has an experiment enabled in the config.json then it's at their own risk.

My concern only concern is that removing the backwards compatibility will cause apps with config.json experiments to break on any CLI command:

Failed to parse contents of project-level config file (unable_to_parse_json) json: cannot unmarshal array into Go struct field ProjectConfig.experiments of type map[string]bool

This is a very poor experience, but we can be good hosts by displaying a remediation message:

💡 Suggestion Check that ~/path/to/my-app/.slack/config.json is valid JSON. If the "experiments" field is an array (e.g. ["exp1"]), update it to an object (e.g. {"exp1": true})

Commit c500bde does the following:

Removes internal/config/experiments_unmarshal.go

Removed UnmarshalJSON from internal/config/project.go

Removed UnmarshalJSON from internal/config/system.go

Displays the remediation hint about "experiments" format when there is a JSON parse error for config.json

mwbrooks

Whoops, like a dummy I forgot to publish my comments for the reviewers 🤦🏻

mwbrooks · 2026-03-16T17:21:07Z

internal/config/experiments_test.go


 		// Write our test script to the memory file system used by the mock
-		jsonContents := []byte(fmt.Sprintf("{\"last_update_checked_at\":\"2023-05-11T15:41:07.799619-07:00\",\"experiments\":[\"%s\"]}", validExperiment))
+		jsonContents := []byte(fmt.Sprintf(`{"last_update_checked_at":"2023-05-11T15:41:07.799619-07:00","experiments":{"%s":true}}`, validExperiment))


note: Just updating the formatting to make the tests more readable with literals and no escapes.

mwbrooks · 2026-03-16T18:29:18Z

internal/config/experiments_test.go

 		assert.Contains(t, mockOutput.String(), "active project experiments: []")
 		assert.Contains(t, mockOutput.String(), fmt.Sprintf("active permanently enabled experiments: [%s]", experiment.Placeholder))
-		assert.Equal(t, []experiment.Experiment{experiment.Placeholder}, config.experiments)
+		assert.ElementsMatch(t, []experiment.Experiment{experiment.Placeholder}, config.GetExperiments())


note: Ordering now varies because it's a map instead of slice.

Use the stricter experiment.Experiment type instead of string for the internal Config.experiments map key to improve type safety.

Reorder experiment source precedence so that higher-priority sources override lower ones: permanently enabled < system config < project config < flags.

Remove custom UnmarshalJSON methods and the unmarshalExperimentsField helper that supported the deprecated array format for experiments in config.json. The old format (["exp1"]) now produces a parse error with a remediation hint to update to the object format ({"exp1": true}).

mwbrooks added 2 commits March 13, 2026 21:23

revert: remove help menu experiment display changes

95d3cd1

Reverts the EXPERIMENTS section changes from the experiments migration commit to keep PR #1 focused on the array-to-map refactor. Help menu improvements will be done in a separate PR.

mwbrooks added this to the Next Release milestone Mar 14, 2026

mwbrooks self-assigned this Mar 14, 2026

mwbrooks added enhancement M-T: A feature request for new functionality experiment Experimental feature accessed behind the --experiment flag or toggle semver:patch Use on pull requests to describe the release version increment labels Mar 14, 2026

mwbrooks changed the title ~~feat: migrate experiments config from array to map~~ feat: migrate experiments config from array to bools Mar 14, 2026

mwbrooks added 3 commits March 16, 2026 11:33

test: tighten up the tests

f01e37f

test: tighten up the tests

b288fb4

Merge branch 'main' into mwbrooks-config-experiments-type

47f6207

mwbrooks marked this pull request as ready for review March 16, 2026 18:43

mwbrooks requested a review from a team as a code owner March 16, 2026 18:43

zimeg approved these changes Mar 16, 2026

View reviewed changes

mwbrooks commented Mar 17, 2026

View reviewed changes

mwbrooks added 3 commits March 17, 2026 09:23

refactor: use experiment.Experiment type as experiments map key

a70d3ee

Use the stricter experiment.Experiment type instead of string for the internal Config.experiments map key to improve type safety.

fix: reorder experiment loading so flags have highest priority

f0c8bc3

Reorder experiment source precedence so that higher-priority sources override lower ones: permanently enabled < system config < project config < flags.

		jsonContents := []byte(fmt.Sprintf("{\"last_update_checked_at\":\"2023-05-11T15:41:07.799619-07:00\",\"experiments\":[\"%s\"]}", validExperiment))
		jsonContents := []byte(fmt.Sprintf(`{"last_update_checked_at":"2023-05-11T15:41:07.799619-07:00","experiments":{"%s":true}}`, validExperiment))

Conversation

mwbrooks commented Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog

Summary

Example

Motivation

Requirements

Uh oh!

codecov bot commented Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zimeg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mwbrooks left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mwbrooks commented Mar 14, 2026 •

edited

Loading

codecov bot commented Mar 14, 2026 •

edited

Loading