Skip to content

docs(experiments): document evaluation contexts field in experiment creation #19811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
inkeep wants to merge 1 commit into
base: master
Choose a base branch
from
Open

docs(experiments): document evaluation contexts field in experiment creation #19811

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 35 additions & 15 deletions contents/docs/experiments/creating-an-experiment.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,25 @@ availability:
enterprise: full
---

import { ProductScreenshot } from 'components/ProductScreenshot'
export const CreateExperimentLight = "https://res.cloudinary.com/dmukukwp6/image/upload/q_auto,f_auto/wizard_step_1_light_c7e16a15d1.png"
export const CreateExperimentDark = "https://res.cloudinary.com/dmukukwp6/image/upload/q_auto,f_auto/wizard_step_1_dark_ddc4372ce6.png"
export const ExperimentGoalWarningLight = "https://res.cloudinary.com/dmukukwp6/image/upload/posthog.com/contents/images/features/experiments/experiment-goal-warning-light-mode.png"
export const ExperimentGoalWarningDark = "https://res.cloudinary.com/dmukukwp6/image/upload/posthog.com/contents/images/features/experiments/experiment-goal-warning-dark-mode.png"

import { ProductVideo} from 'components/ProductVideo'
export const SetExperimentLight = "https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/set-experiment-rollout-light-mode.mp4"
export const SetExperimentDark = "https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/set-experiment-rollout-dark-mode.mp4"
export const ImprovementCalculatorLight = "https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/improvement-calculator-light-mode.mp4"
export const ImprovementCalculatorDark = "https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/improvement-calculator-dark-mode.mp4"
import { ProductScreenshot } from "components/ProductScreenshot";
export const CreateExperimentLight =
"https://res.cloudinary.com/dmukukwp6/image/upload/q_auto,f_auto/wizard_step_1_light_c7e16a15d1.png";
export const CreateExperimentDark =
"https://res.cloudinary.com/dmukukwp6/image/upload/q_auto,f_auto/wizard_step_1_dark_ddc4372ce6.png";
export const ExperimentGoalWarningLight =
"https://res.cloudinary.com/dmukukwp6/image/upload/posthog.com/contents/images/features/experiments/experiment-goal-warning-light-mode.png";
export const ExperimentGoalWarningDark =
"https://res.cloudinary.com/dmukukwp6/image/upload/posthog.com/contents/images/features/experiments/experiment-goal-warning-dark-mode.png";

import { ProductVideo } from "components/ProductVideo";
export const SetExperimentLight =
"https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/set-experiment-rollout-light-mode.mp4";
export const SetExperimentDark =
"https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/set-experiment-rollout-dark-mode.mp4";
export const ImprovementCalculatorLight =
"https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/improvement-calculator-light-mode.mp4";
export const ImprovementCalculatorDark =
"https://res.cloudinary.com/dmukukwp6/video/upload/posthog.com/contents/images/features/experiments/improvement-calculator-dark-mode.mp4";

To create a new experiment, go to the [Experiments](https://app.posthog.com/experiments) section in the PostHog app, and click on the **New experiment** button in the top right.

Expand All @@ -33,10 +41,10 @@ A Guide panel on the right provides contextual tips for each step.
When you're done, click **Save as draft** on the final step. You can launch the experiment from its detail page after saving.

<ProductScreenshot
imageLight={CreateExperimentLight}
imageDark={CreateExperimentDark}
alt="How to create an experiment in PostHog"
classes="rounded"
imageLight={CreateExperimentLight}
imageDark={CreateExperimentDark}
alt="How to create an experiment in PostHog"
classes="rounded"
/>

## Step 1: Description
Expand All @@ -53,6 +61,16 @@ By default, the wizard creates a new feature flag for you automatically when the

> **Advanced:** It's possible to create experiments without using PostHog's feature flags (for example, if you're using a different third-party library for feature flags). For more info, read our docs on [implementing experiments without feature flags](/docs/experiments/running-experiments-without-feature-flags).

### Evaluation contexts

When creating a new flag for your experiment, you can specify **evaluation contexts** to control where the flag evaluates at runtime. This field only appears when the experiment creates a new flag — if you link an existing flag, it keeps its own contexts, which you can edit from the flag page.

Evaluation contexts are labels like `main-app`, `marketing-site`, or `checkout` that describe where in your application the flag should be active. You can select from existing contexts or type custom values.

If your project has default evaluation contexts configured, you can leave the field empty and the defaults are applied automatically. If your project requires evaluation contexts for new flags and no defaults are configured, you must select at least one context before saving.

For more on how evaluation contexts work, see the [feature flags evaluation contexts documentation](/docs/feature-flags/creating-feature-flags#evaluation-contexts-optional).

## Step 2: Variant rollout

This step controls who sees which variant. You configure the variants themselves, how traffic is split between them, and what percentage of your users are included.
Expand Down Expand Up @@ -82,9 +100,11 @@ To target on data that lives outside your product, like a plan tier in Stripe or
> **Note:** If you select a server-side event, you may see a warning that no feature flag information can be detected with the event. To resolve this issue, see [step 2 of adding your experiment code and how to submit feature flag information with your events](/docs/experiments/adding-experiment-code#step-2-server-side-only-add-the-feature-flag-property-to-your-events).
>
> <ProductScreenshot

imageLight={ExperimentGoalWarningLight}
imageDark={ExperimentGoalWarningDark}
alt="Experiment goal warning" classes="rounded"

/>

## Step 3: Analytics
Expand Down
Loading