Scripts

Everything runnable lives in scripts/. Nothing under src/ executes on load.

The folder mirrors the package's three submodules, so a script sits next to the model it drives:

scripts/
├── zbenchmark/   drivers for the zonal (copper-plate) reconstruction
├── nbenchmark/   drivers for the network-resolved nodal dispatch
└── opffcas/      drivers for AC/DC OPF with FCAS co-optimisation

Every script takes its whole configuration from the command line or the environment — including which solver to use — and prints that configuration before doing any work, so a result is reproducible from the log above it.

How arguments work

The plumbing lives in NEMX.Scripting and is re-exported by NEMX, so a script needs only using NEMX. There is nothing to include.

Three mechanisms, in decreasing order of precedence:

FormExample
Named option--solver=ipopt, --data-dir=/mnt/nem, --forms=DCP,ACP
Bare flag--download, --with-lmp, --verbose-solver
Environment variableNEMX_SOLVER=ipopt, NEMX_DATA_DIR=/mnt/nem

An option's environment variable is its name upper-cased, with hyphens turned into underscores, prefixed NEMX_: --out-dir reads NEMX_OUT_DIR.

Bare flags are always off by default. A switch that should default to on is exposed as --no-something, so the default is visible on the command line rather than hidden in the file.

Positional arguments are accepted where a script documents them, and each can equally come from an environment variable.

NEMX.Scripting — Module
NEMX.Scripting

Command-line plumbing shared by every driver script: argument parsing, solver selection, output paths, and the configuration banner.

This lives in the package rather than in a file each script includes, so that a script needs only using NEMX to have all of it, and so that the behaviour is tested with the rest of the library instead of being duplicated seventeen times.

Why it exists

Scripts are where reproducibility usually goes wrong. A configuration edited in place leaves no record of what produced a result. A solver hard-coded into a script makes that script unusable to anyone without that solver's licence. And the same flag spelled three different ways across three scripts guarantees that one of them is stale.

So every driver in this package takes its configuration the same way, selects its solver the same way, and prints the whole configuration before doing any work.

How a script takes configuration

Three mechanisms, in decreasing order of precedence:

FormExample
Named option--solver=ipopt, --data-dir=/mnt/nem, --forms=DCP,ACP
Bare flag--download, --with-lmp, --verbose-solver
Environment variableNEMX_SOLVER=ipopt, NEMX_DATA_DIR=/mnt/nem

An option's environment variable is its name upper-cased, with hyphens turned into underscores, prefixed NEMX_. So --out-dir reads NEMX_OUT_DIR.

Bare flags are always OFF by default. A switch that ought to default to ON is exposed as --no-something, so that the default is visible on the command line rather than hidden in the file.

Typical use

using NEMX

const START  = script_datetime(script_positional(1, "NEMX_START", "2025-09-02T04:05"))
const N      = script_integer(script_positional(2, "NEMX_N", "288"))
const SOLVER = select_solver(script_option("solver", "highs"))

print_banner("Zonal benchmark sweep", "start" => START, "intervals" => N)
source
NEMX.Scripting.script_option — Function
script_option(name::AbstractString, default::AbstractString = "") -> String

Value of a --name=value command-line option.

Falls back to the backing environment variable (see environment_name), and then to default.

Arguments

  • name: the option name without its leading --.
  • default: value to use when neither the option nor the variable is set.

Returns

The option value, as a string. Convert it with script_integer, script_number, script_datetime or script_list.

Example

julia --project=. scripts/nbenchmark/run_network_day.jl --solver=ipopt
NEMX_SOLVER=ipopt julia --project=. scripts/nbenchmark/run_network_day.jl
source
NEMX.Scripting.script_flag — Function
script_flag(name::AbstractString) -> Bool

Whether the bare flag --name was passed on the command line, or its backing environment variable was set to "1".

Bare flags are always off by default. A switch that should default to ON is exposed as --no-something instead, so the default stays visible.

Arguments

  • name: the flag name without its leading --.

Returns

true if the flag was given, false otherwise.

source
NEMX.Scripting.script_positional — Function
script_positional(index::Integer, variable::AbstractString,
                  default::AbstractString) -> String

The index-th positional argument, ignoring anything beginning with --.

Falls back to the named environment variable, and then to default. The environment variable is named explicitly here rather than derived, because positional arguments have no option name to derive it from.

Arguments

  • index: 1-based position among the bare (non---) arguments.
  • variable: environment variable to consult when the argument is absent.
  • default: value to use when neither is present.

Returns

The argument value, as a string.

source
NEMX.Scripting.script_integer — Function
script_integer(text::AbstractString) -> Int

Parse an integer argument.

Throws

ErrorException quoting the offending text if it is not an integer.

source
NEMX.Scripting.script_number — Function
script_number(text::AbstractString) -> Float64

Parse a floating-point argument.

Throws

ErrorException quoting the offending text if it is not a number.

source
NEMX.Scripting.script_datetime — Function
script_datetime(text::AbstractString) -> DateTime

Parse an ISO-8601 dispatch interval, for example 2025-09-02T04:05.

Throws

ErrorException quoting the offending text and the expected format.

source
NEMX.Scripting.script_list — Function
script_list(text::AbstractString) -> Vector{String}

Split a comma-separated argument, trimming whitespace and dropping empties.

Example

script_list("DCP, ACP ,, LPACC")     # ["DCP", "ACP", "LPACC"]
source
NEMX.Scripting.resolve_output_dir — Function
resolve_output_dir(default::AbstractString) -> String

Directory for a script's output, from --out-dir= or NEMX_OUT_DIR, else default. The directory is created if it does not exist.

A script that writes a large result set should default to a LOCAL scratch path rather than a cloud-synced folder. A file rewritten every few minutes for hours is not reliably served back at its newest version by a sync client — a completed sweep has been observed leaving only its first two thirds on disk. The pattern to follow is: checkpoint to scratch, then copy into the project tree once, at the end.

Arguments

  • default: directory to use when neither the option nor the variable is set.

Returns

The resolved directory path, which now exists.

source
NEMX.Scripting.resolve_input_dir — Function
resolve_input_dir(default::AbstractString) -> String

Directory a script reads its inputs from, via --data-dir= or NEMX_DATA_DIR.

The directory is NOT created. A missing input directory is an error the script should report, not paper over.

Arguments

  • default: directory to use when neither the option nor the variable is set.

Returns

The resolved directory path, which may or may not exist.

source
NEMX.Scripting.print_banner — Function
print_banner(title::AbstractString, settings::Pair...)

Print a script's full configuration before it does any work.

Every result this package produces should be reproducible from the banner printed above it: the script, the package version, the resolved arguments, and the behavioural flags in force. Scripts call this once, at the top, after resolving their arguments and before the first solve.

Arguments

  • title: one line naming what is about to run.
  • settings: "label" => value pairs, printed one per line in the order given.

Example

print_banner("Network day sweep",
             "start" => START, "intervals" => N,
             "formulations" => join(FORMS, ","), "solver" => SOLVER_NAME)
source
NEMX.Scripting.print_flag_values — Function
print_flag_values(mod::Module, names::Symbol...)

Print the current value of the behavioural flags a script depends on.

Flags are Refs whose defaults reproduce a validated configuration. A script that changes one must say so in its output, or its results are not reproducible from the log. Call this after any flag assignment.

Arguments

  • mod: the module holding the flags, e.g. NEMX.ZBenchmark.
  • names: flag names. A name that does not exist in mod is skipped, so a script need not know which flags a given version defines.
source
NEMX.Scripting.print_progress — Function
print_progress(done::Integer, total::Integer, started::Float64,
               label::AbstractString = "")

One line of progress, with elapsed time and an estimate of what remains.

Long sweeps run for hours. Without this there is no way to tell a slow interval from a hung one.

Arguments

  • done: intervals completed so far.
  • total: intervals in the whole run.
  • started: the value time() returned when the run began.
  • label: short text identifying the current item, e.g. the interval.
source
NEMX.Scripting.environment_name — Function
environment_name(option_name::AbstractString) -> String

The environment variable that backs a given command-line option.

The rule is fixed and mechanical: upper-case the option name, replace hyphens with underscores, prefix NEMX_. So --out-dir is backed by NEMX_OUT_DIR.

Arguments

  • option_name: the option name WITHOUT its leading --.

Returns

The environment variable name.

source
NEMX.Scripting.package_version — Function
package_version() -> VersionNumber

The NEMX version, read from Project.toml.

Defined here rather than taken from the parent module, so that Scripting has no dependency on the load order of the rest of the package.

source

Choosing a solver

julia --project=. scripts/zbenchmark/run_zonal_benchmark.jl --solver=highs
julia --project=. scripts/opffcas/run_opffcas.jl --nlp-solver=ipopt --lp-solver=highs
NEMX.Scripting.SOLVER_CHOICES — Constant
SOLVER_CHOICES

Solver names accepted by select_solver, and what each is for.

NamePackageUse
highsHiGHSLinear and mixed-integer problems. The default wherever an LP is solved.
ipoptIpoptNon-linear problems: the AC formulations and the IV rectangular form.
scsSCSConic relaxations, where Ipopt is a poor fit.

All three are open source. No script in this package requires a licensed solver. A commercial solver can still be used by passing its optimizer straight to the library functions, which take an optimizer object rather than a name.

source
NEMX.Scripting.select_solver — Function
select_solver(name::AbstractString = "highs";
              silent::Bool = true,
              attributes::Vector{<:Pair} = Pair{String,Any}[])

Build a JuMP optimizer factory from a solver name.

Arguments

Keywords

  • silent: suppress the solver's own output. true by default, because these scripts print their own progress. Pass --verbose-solver in a script when diagnosing a bad solve.
  • attributes: extra solver attributes, appended AFTER the defaults so that they win. For example ["max_iter" => 20_000] for a stubborn AC interval.

Returns

An OptimizerWithAttributes, ready to hand to any library function.

Throws

ErrorException naming the valid choices, rather than a MethodError several frames deeper.

Example

solver = select_solver("ipopt"; attributes = ["max_iter" => 20_000])
source

All three solvers are open source; nothing here needs a licence. --verbose-solver lets the solver print its own progress, which is what you want when diagnosing a bad solve.

The network-day sweep is the exception. Each formulation carries its own solver in NEMX.NBenchmark.FORMULATIONS, because an LP formulation wants HiGHS and a non-linear one wants Ipopt, and mixing them is never what you want. Change it there rather than by flag.

scripts/zbenchmark/ — the zonal reconstruction

General drivers

ScriptPurpose
zonal_simple_example.jlTwo units, one region, no data required. Run this first.
run_zonal_benchmark.jlSweep intervals and score every price against AEMO's published ROP.
plot_zonal_benchmark.jlFigures and tables from that sweep.
export_interval_csvs.jlDump every market input for one interval.
run_bess_event_study.jlPer-unit local prices, offers and binding constraints across a window.
analyse_bess_event_study.jlThe gated investigation over those results, emitting LaTeX tables.
plot_bess_event_study.jlThe event figure, with a symmetric-log price axis.

Named runs

These fix the arguments of a study that is referred to by name, so that "run the July 2024 benchmark" is one command and always means the same thing. Each is a thin wrapper: it sets defaults and includes the general driver, so there is one implementation of the sweep rather than several. Anything given on the command line still wins.

ScriptWhat it fixes
run_historical_dispatch_jul2024.jlJuly 2024, 10 random intervals, tag 2024_07
run_historical_dispatch_sep2025.jlSeptember 2025, 10 random intervals, tag 2025_09
run_historical_dispatch_sep2025_3day.jlSeptember 2025, 864 consecutive intervals (three trading days), tag 2025_09_3day
run_bess_event_nov2025_2day.jl20–21 November 2025, 576 consecutive intervals (two days), NSW1
julia --project=. scripts/zbenchmark/zonal_simple_example.jl
julia --project=. scripts/zbenchmark/run_historical_dispatch_jul2024.jl --download
julia --project=. scripts/zbenchmark/run_historical_dispatch_sep2025_3day.jl
julia --project=. scripts/zbenchmark/export_interval_csvs.jl 2025-09-02T13:25

A random sample gives an unbiased error estimate but cannot show anything that depends on the sequence. The consecutive runs exist for ramping, storage state, fast-start mode, and price spikes with the recovery that follows them.

scripts/nbenchmark/ — the network model

ScriptPurpose
run_network_interval.jlOne interval, several formulations, printed side by side.
run_network_day.jlA full day; prices, decomposition and constraint ledger to CSV.
run_network_day_20250902.jlNamed run: the reference network day, 2025-09-02, 288 intervals, all formulations.
run_ac_recovery.jlWhat it costs to make a DC dispatch AC-feasible.
plot_network_day.jlFigures and tables from a network-day sweep.
fix_snem2000_case.jlOne-off repair of the raw 2000-bus case.
julia --project=. scripts/nbenchmark/run_network_interval.jl 2025-09-02T12:05 DCP,ACP
julia --project=. scripts/nbenchmark/run_network_day_20250902.jl
julia --project=. scripts/nbenchmark/run_ac_recovery.jl 2025-09-02T04:05 288

Validate a window with DCP before widening the formulation list: the AC formulations and the convex relaxations are 2000-bus non-linear programs taking tens of seconds each, so the full list over a day is an overnight job.

scripts/opffcas/ — OPF with FCAS

ScriptPurpose
run_opffcas.jlAC/DC OPF with FCAS across formulations, with regional price tables.
run_opffcas_arpst.jlThe same with phase-shifting transformers active.
compute_marginal_loss_factors.jlPer-bus MLFs by finite-differencing an AC power flow.
build_market_scenario.jlRaw AEMO bid CSVs into an OPFFCAS scenario.
julia --project=. scripts/opffcas/run_opffcas.jl --scenario=s1
julia --project=. scripts/opffcas/build_market_scenario.jl --scenario=s1 \
      --market-dir=/data/aemo/2025-09

compute_marginal_loss_factors.jl needs PowerModelsACDCsecurityconstrained, which is not in the General registry; see Installation.

Long runs

A 288-interval sweep across every formulation runs overnight. Three things make that survivable.

Checkpointing. Results are flushed every --checkpoint intervals, so an interrupted run leaves usable output. run_ac_recovery.jl goes further and resumes: intervals already in its CSV are kept and not re-solved, so re-running the same command continues where it stopped.

Isolation. Each interval is solved in its own try. One unsolvable or malformed interval costs that interval, not the run, and appears in the failure list at the end.

Scratch output. Scripts that write large result sets default --out-dir to a local scratch path and copy into the project tree once, at the end. A file rewritten every few minutes for hours is not reliably served back at its newest version by a cloud sync client — a completed sweep has been seen leaving only its first two thirds on disk. The final copy is also wrapped: if the data directory is read-only or unreachable, the failure is reported and the scratch copy is named, rather than an exception discarding hours of work.

Writing a new script

The skeleton, in the house style:

# =============================================================================
# my_script.jl
#
# PURPOSE
#   One paragraph on what this produces and why.
#
# ARGUMENTS
#   Positional    Env            Default            Meaning
#   ------------  -------------  -----------------  ------------------------
#   1             NEMX_START     2025-09-02T04:05   first interval
#
#   Options       Env            Default            Meaning
#   ------------  -------------  -----------------  ------------------------
#   --solver=     NEMX_SOLVER    highs              highs | ipopt | scs
#
# EXAMPLES
#   julia --project=. scripts/zbenchmark/my_script.jl
# =============================================================================

using NEMX
using Dates

const ZB = NEMX.ZBenchmark

const START  = script_datetime(script_positional(1, "NEMX_START", "2025-09-02T04:05"))
const SOLVER = script_option("solver", "highs")

ZB.SOLVER_FACTORY[] = select_solver(SOLVER; silent = !script_flag("verbose-solver"))

print_banner("What this run is", "start" => START, "solver" => SOLVER)
print_flag_values(ZB, :LOSS_MODEL_FROM_XML, :ZONAL_MLF_KEEP_SCALING)

# ... the work ...

Three conventions to keep:

  1. Document every argument in the header, with its environment variable and its default. The header is what someone reads; the body is what they read afterwards.
  2. Print the configuration before doing any work, including any behavioural flag the script changes.
  3. Take the solver as an argument. A hard-coded solver makes a script unusable to anyone without it.