Frisky

Frisky is a Dask scheduler in Rust. Frisky is ...

• Fast: running at about 200x Dask scheduling performance
• Simple: focusing on only core APIs
• Agentic: with inspection APIs designed specifically for agents

Using Frisky

Frisky supports Dask APIs for flexible custom workflows like Dask Futures

from frisky import LocalCluster

def increment(x):
    return x + 1

with LocalCluster(n_workers=4) as cluster:
    with cluster.get_client() as client:
        futures = client.map(increment, range(10))
        total = client.submit(sum, futures)
        print(total.result())

And can operate on existing Dask collections libraries

import dask.array as da
import xarray as xr
from frisky import LocalCluster

data = xr.DataArray(
    da.random.random((365, 100, 100), chunks=(30, 50, 50)),
    dims=("time", "lat", "lon"),
    name="temperature",
)

with LocalCluster(n_workers=4) as cluster:
    with cluster.get_client() as client:
        annual_mean = client.compute(data.mean("time"))
        print(annual_mean.result())

Performance

The difference is that Frisky operates at ~200x the scheduling performance of Dask's internal scheduler. This means it can support larger graphs and more workers without slowing down.

Frisky adds performance in the following:

• Scheduling: we target 1-5us per task, supporting hundreds of thousands of tasks per second
• Network/Disk: throughput is greatly improved, running at raw metal speeds

Frisky however does not speed up the following:

• Dask graph construction: if you're using Dask collections (Xarray, Dask Array, Dask dataframe) then graph construction is still done in Python
• In-task code: Frisky can not reach into your tasks running normal Python code (numpy, pandas, polars, pytorch) and make it faster
• Dumb shit: you can still do dumb things (although Frisky works well with AI agents to avoid this)

Frisky is intentionally small. It is not a full replacement for every Dask feature; it focuses on scheduling Python tasks quickly and making running clusters easy to inspect.