Fork a run

Explore different parameters or models from a specific point in an experiment without impacting the original run.

The ability to fork a run is in active development and in private preview. Contact W&B Support at [e-mail] to request access to this feature.

You can explore different hyperparameters or models from a specific point in an experiment without impacting the original run. To do this, fork from an existing W&B run.

When you fork from a run, W&B creates a new run using the source run’s unique ID and a specified step. Summary metrics from the source run are copied to the forked run. The forked run shares all history and files from the source run up to the specified step.

After the fork step, you can log new data to the forked run independently of the original run.

View a live demo of a forked run produced by the code below.

• Forking a run requires wandb SDK version >= 0.16.5
• Forking a run requires monotonically increasing steps. You cannot fork from a run that uses non-monotonic steps defined with define_metric(). Non-monotonic steps break the chronological order of run history and system metrics.

Specify the source run’s unique run ID and the step you want to start the forked run from as arguments to fork_from in wandb.init().

Fork from a previously logged run

The following code snippet shows how to fork from a run that was previously logged to W&B.

First, obtain the run ID of the run you want to fork from. Next, specify the run ID and the step you want to fork from as arguments to fork_from in wandb.init().

Copy and paste the following code into a Python script or notebook cell. Replace <source-run-id>, <project>, and <entity> with your own values:

import wandb

# The unique ID of the source run to fork from
source_run_id = "<source-run-id>"

# Specify the step to fork from
fork_step = 200

# Fork the run
with wandb.init(
    project="<project>",
    entity="<entity>",
    fork_from=f"{source_run_id}?_step={fork_step}",
) as forked_run:
    pass

Fork from a run in the same script

The following code snippet shows how to create a run and fork from that run within the same script. This might occur if you want to fork from a run that you just created without having to look up the run ID in the W&B App.

First, initialize a run and log some data. Next, use the original run object’s id property to obtain the run ID of that run. Finally, initialize a new run and pass the original run’s ID and the step you want to fork from as arguments to fork_from in wandb.init().

import wandb

# Initialize a run
with wandb.init(
    project="<project>",
    entity="<entity>"
) as original_run:
    # ...training logic goes here ...
    pass

# Specify the step to fork from
fork_step = int("<num>")

# Use the original run's ID and specify the step to fork from
with wandb.init(
    project="<project>",
    entity="<entity>",
    fork_from=f"{original_run.id}?_step={fork_step}",
) as forked_run:
    # ...training logic goes here ...
    pass

Use the original_run.id property to obtain the unique run ID of the original run.

Example script

For example, the following code example shows how to first fork a run and then how to log metrics to the forked run starting from a training step of 200.

Copy and paste the following code into a Python script or notebook cell. Replace <project> and <entity> with your own values.

import wandb
import math

# Initialize the first run and log some metrics
with wandb.init(
    project="<project>",
    entity="<entity>"
) as run1:
    for i in range(300):
        run1.log({"metric": i})

# Fork from the first run at a specific step and log the
# metric starting from step 200
with wandb.init(
    project="<project>", 
    entity="<entity>", 
    fork_from=f"{run1.id}?_step=200"
) as run2:
    # Continue logging in the new run
    # For the first few steps, log the metric as is from run1
    # After step 250, start logging the spikey pattern
    for i in range(200, 300):
        if i < 250:
            # Continue logging from run1 without spikes
            metric_value = i
        else:
            # Introduce the spikey behavior starting from step 250
            metric_value = i + (2 * math.sin(i / 3.0))  # Apply a subtle spikey pattern

        # Log both metrics in a single call to ensure they're
        # logged at the same step
        run2.log({
            "metric": metric_value,
            "additional_metric": i * 1.1
        })

Rewind and forking compatibility

Forking complements a rewind by providing more flexibility in managing and experimenting with your runs.

When you fork from a run, W&B creates a new branch off a run at a specific point to try different parameters or models.

When you rewind a run, W&B lets you correct or modify the run history itself.