Skip to content

Integration Guide

This page shows the typical pattern for wiring leitwerk into your own evaluation loop.

The intended call flow is sequential:

Flowchart

1. Define a Parameter Schema

~ Preflight Check ~

from dataclasses import dataclass

from leitwerk import parameter


@dataclass
class MyParams:
    attack_threshold: float = parameter()
    worker_limit: float = parameter(mean=66, scale=10, min=12)

parameter(...) defines the prior distribution for each value:

  • mean: initial best guess
  • scale: initial spread
  • min and max: optional bounds

Tip

Dictionary schemas are supported if you prefer string-based lookup: 

from leitwerk import Parameter

MyParams = {"a": Parameter(), "b": Parameter()}

Tip

Use nested schemas to group parameters together. leitwerk understands tree structures.

Note

Dataclass fields that are not parameters are passed through as constants.

2. Create the Optimizer

~ Engine Ignition ~

For automatic JSON persistence, use the OptimizerSession wrapper:

from leitwerk import OptimizerSession

opt = OptimizerSession("params.json", MyParams)

If the session file already exists, it is loaded and reconciled automatically:

>>> opt.restored
True
>>> opt.schema_diff
SchemaDiff(added=[], removed=[], changed=[], unchanged=['attack_threshold', 'worker_limit'])

Question

What happens when the schema changes?

For an in-memory run, use Optimizer directly:

from leitwerk import Optimizer

opt = Optimizer(MyParams)

If you are using other means of persistence, you can optionally restore it from state as well:

schema_diff = opt.load(state)

Both Optimizer and OptimizerSession accept additional constructor arguments:

from leitwerk import Optimizer

opt = Optimizer(MyParams, batch_size=10, seed=1234)

Available settings:

  • batch_size: number of samples per batch / optimizer step
  • seed: for reproducible runs

3. Sample a Candidate

~ Liftoff ~

params = opt.ask()

>>> params
MyParams(attack_threshold=-0.8312413125179872, worker_limit=59.407519238244)

For deterministic evaluation, use the optimized mean instead of sampling:

>>> opt.mean
MyParams(attack_threshold=0.0, worker_limit=66.0)

Optionally, provide a JSON-valued context for the current sample:

context = {"opponent_race": "Protoss"}  # optional
params = opt.ask(context)

Question

What context should I provide?

4. Report the Result

~ Landing ~

After evaluation, encode the outcome as one or more scalars:

result = +1 if win else 0
report = opt.tell(result)

Binary win/loss alone tends to have little gradient to learn from. Add smooth tie-breakers when possible:

report = opt.tell((result, get_efficiency()))

Result handling:

  • opt.tell((a, b, c)) ranks results lexicographically with higher = better
  • the first item is the main objective, later items are tie-breakers
  • maximization is the default, flip the sign for loss objectives

Question

How should I choose the objective?

>>> report
OptimizerReport(completed_batch=False, matched_context=False, status=<XNESStatus.OK: 1>, restarted=False)

When using OptimizerSession, the JSON file is updated atomically on tell. An Optimizer can be serialized with:

state = opt.save()