Scripts
Everything runnable lives in scripts/. Nothing under src/ executes on load.
The folder mirrors the package's three submodules, so a script sits next to the model it drives:
scripts/
├── zbenchmark/ drivers for the zonal (copper-plate) reconstruction
├── nbenchmark/ drivers for the network-resolved nodal dispatch
└── opffcas/ drivers for AC/DC OPF with FCAS co-optimisationEvery script takes its whole configuration from the command line or the environment — including which solver to use — and prints that configuration before doing any work, so a result is reproducible from the log above it.
How arguments work
The plumbing lives in NEMX.Scripting and is re-exported by NEMX, so a script needs only using NEMX. There is nothing to include.
Three mechanisms, in decreasing order of precedence:
| Form | Example |
|---|---|
| Named option | --solver=ipopt, --data-dir=/mnt/nem, --forms=DCP,ACP |
| Bare flag | --download, --with-lmp, --verbose-solver |
| Environment variable | NEMX_SOLVER=ipopt, NEMX_DATA_DIR=/mnt/nem |
An option's environment variable is its name upper-cased, with hyphens turned into underscores, prefixed NEMX_: --out-dir reads NEMX_OUT_DIR.
Bare flags are always off by default. A switch that should default to on is exposed as --no-something, so the default is visible on the command line rather than hidden in the file.
Positional arguments are accepted where a script documents them, and each can equally come from an environment variable.
NEMX.Scripting — Module
NEMX.ScriptingCommand-line plumbing shared by every driver script: argument parsing, solver selection, output paths, and the configuration banner.
This lives in the package rather than in a file each script includes, so that a script needs only using NEMX to have all of it, and so that the behaviour is tested with the rest of the library instead of being duplicated seventeen times.
Why it exists
Scripts are where reproducibility usually goes wrong. A configuration edited in place leaves no record of what produced a result. A solver hard-coded into a script makes that script unusable to anyone without that solver's licence. And the same flag spelled three different ways across three scripts guarantees that one of them is stale.
So every driver in this package takes its configuration the same way, selects its solver the same way, and prints the whole configuration before doing any work.
How a script takes configuration
Three mechanisms, in decreasing order of precedence:
| Form | Example |
|---|---|
| Named option | --solver=ipopt, --data-dir=/mnt/nem, --forms=DCP,ACP |
| Bare flag | --download, --with-lmp, --verbose-solver |
| Environment variable | NEMX_SOLVER=ipopt, NEMX_DATA_DIR=/mnt/nem |
An option's environment variable is its name upper-cased, with hyphens turned into underscores, prefixed NEMX_. So --out-dir reads NEMX_OUT_DIR.
Bare flags are always OFF by default. A switch that ought to default to ON is exposed as --no-something, so that the default is visible on the command line rather than hidden in the file.
Typical use
using NEMX
const START = script_datetime(script_positional(1, "NEMX_START", "2025-09-02T04:05"))
const N = script_integer(script_positional(2, "NEMX_N", "288"))
const SOLVER = select_solver(script_option("solver", "highs"))
print_banner("Zonal benchmark sweep", "start" => START, "intervals" => N)NEMX.Scripting.script_option — Function
script_option(name::AbstractString, default::AbstractString = "") -> StringValue of a --name=value command-line option.
Falls back to the backing environment variable (see environment_name), and then to default.
Arguments
name: the option name without its leading--.default: value to use when neither the option nor the variable is set.
Returns
The option value, as a string. Convert it with script_integer, script_number, script_datetime or script_list.
Example
julia --project=. scripts/nbenchmark/run_network_day.jl --solver=ipopt
NEMX_SOLVER=ipopt julia --project=. scripts/nbenchmark/run_network_day.jlNEMX.Scripting.script_flag — Function
script_flag(name::AbstractString) -> BoolWhether the bare flag --name was passed on the command line, or its backing environment variable was set to "1".
Bare flags are always off by default. A switch that should default to ON is exposed as --no-something instead, so the default stays visible.
Arguments
name: the flag name without its leading--.
Returns
true if the flag was given, false otherwise.
NEMX.Scripting.script_positional — Function
script_positional(index::Integer, variable::AbstractString,
default::AbstractString) -> StringThe index-th positional argument, ignoring anything beginning with --.
Falls back to the named environment variable, and then to default. The environment variable is named explicitly here rather than derived, because positional arguments have no option name to derive it from.
Arguments
index: 1-based position among the bare (non---) arguments.variable: environment variable to consult when the argument is absent.default: value to use when neither is present.
Returns
The argument value, as a string.
NEMX.Scripting.script_integer — Function
script_integer(text::AbstractString) -> IntParse an integer argument.
Throws
ErrorException quoting the offending text if it is not an integer.
NEMX.Scripting.script_number — Function
script_number(text::AbstractString) -> Float64Parse a floating-point argument.
Throws
ErrorException quoting the offending text if it is not a number.
NEMX.Scripting.script_datetime — Function
script_datetime(text::AbstractString) -> DateTimeParse an ISO-8601 dispatch interval, for example 2025-09-02T04:05.
Throws
ErrorException quoting the offending text and the expected format.
NEMX.Scripting.script_list — Function
script_list(text::AbstractString) -> Vector{String}Split a comma-separated argument, trimming whitespace and dropping empties.
Example
script_list("DCP, ACP ,, LPACC") # ["DCP", "ACP", "LPACC"]NEMX.Scripting.resolve_output_dir — Function
resolve_output_dir(default::AbstractString) -> StringDirectory for a script's output, from --out-dir= or NEMX_OUT_DIR, else default. The directory is created if it does not exist.
A script that writes a large result set should default to a LOCAL scratch path rather than a cloud-synced folder. A file rewritten every few minutes for hours is not reliably served back at its newest version by a sync client — a completed sweep has been observed leaving only its first two thirds on disk. The pattern to follow is: checkpoint to scratch, then copy into the project tree once, at the end.
Arguments
default: directory to use when neither the option nor the variable is set.
Returns
The resolved directory path, which now exists.
NEMX.Scripting.resolve_input_dir — Function
resolve_input_dir(default::AbstractString) -> StringDirectory a script reads its inputs from, via --data-dir= or NEMX_DATA_DIR.
The directory is NOT created. A missing input directory is an error the script should report, not paper over.
Arguments
default: directory to use when neither the option nor the variable is set.
Returns
The resolved directory path, which may or may not exist.
NEMX.Scripting.print_banner — Function
print_banner(title::AbstractString, settings::Pair...)Print a script's full configuration before it does any work.
Every result this package produces should be reproducible from the banner printed above it: the script, the package version, the resolved arguments, and the behavioural flags in force. Scripts call this once, at the top, after resolving their arguments and before the first solve.
Arguments
title: one line naming what is about to run.settings:"label" => valuepairs, printed one per line in the order given.
Example
print_banner("Network day sweep",
"start" => START, "intervals" => N,
"formulations" => join(FORMS, ","), "solver" => SOLVER_NAME)NEMX.Scripting.print_flag_values — Function
print_flag_values(mod::Module, names::Symbol...)Print the current value of the behavioural flags a script depends on.
Flags are Refs whose defaults reproduce a validated configuration. A script that changes one must say so in its output, or its results are not reproducible from the log. Call this after any flag assignment.
Arguments
mod: the module holding the flags, e.g.NEMX.ZBenchmark.names: flag names. A name that does not exist inmodis skipped, so a script need not know which flags a given version defines.
NEMX.Scripting.print_progress — Function
print_progress(done::Integer, total::Integer, started::Float64,
label::AbstractString = "")One line of progress, with elapsed time and an estimate of what remains.
Long sweeps run for hours. Without this there is no way to tell a slow interval from a hung one.
Arguments
done: intervals completed so far.total: intervals in the whole run.started: the valuetime()returned when the run began.label: short text identifying the current item, e.g. the interval.
NEMX.Scripting.environment_name — Function
environment_name(option_name::AbstractString) -> StringThe environment variable that backs a given command-line option.
The rule is fixed and mechanical: upper-case the option name, replace hyphens with underscores, prefix NEMX_. So --out-dir is backed by NEMX_OUT_DIR.
Arguments
option_name: the option name WITHOUT its leading--.
Returns
The environment variable name.
NEMX.Scripting.package_version — Function
package_version() -> VersionNumberThe NEMX version, read from Project.toml.
Defined here rather than taken from the parent module, so that Scripting has no dependency on the load order of the rest of the package.
Choosing a solver
julia --project=. scripts/zbenchmark/run_zonal_benchmark.jl --solver=highs
julia --project=. scripts/opffcas/run_opffcas.jl --nlp-solver=ipopt --lp-solver=highsNEMX.Scripting.SOLVER_CHOICES — Constant
SOLVER_CHOICESSolver names accepted by select_solver, and what each is for.
| Name | Package | Use |
|---|---|---|
highs | HiGHS | Linear and mixed-integer problems. The default wherever an LP is solved. |
ipopt | Ipopt | Non-linear problems: the AC formulations and the IV rectangular form. |
scs | SCS | Conic relaxations, where Ipopt is a poor fit. |
All three are open source. No script in this package requires a licensed solver. A commercial solver can still be used by passing its optimizer straight to the library functions, which take an optimizer object rather than a name.
NEMX.Scripting.select_solver — Function
select_solver(name::AbstractString = "highs";
silent::Bool = true,
attributes::Vector{<:Pair} = Pair{String,Any}[])Build a JuMP optimizer factory from a solver name.
Arguments
name: one ofSOLVER_CHOICES. Case-insensitive.
Keywords
silent: suppress the solver's own output.trueby default, because these scripts print their own progress. Pass--verbose-solverin a script when diagnosing a bad solve.attributes: extra solver attributes, appended AFTER the defaults so that they win. For example["max_iter" => 20_000]for a stubborn AC interval.
Returns
An OptimizerWithAttributes, ready to hand to any library function.
Throws
ErrorException naming the valid choices, rather than a MethodError several frames deeper.
Example
solver = select_solver("ipopt"; attributes = ["max_iter" => 20_000])All three solvers are open source; nothing here needs a licence. --verbose-solver lets the solver print its own progress, which is what you want when diagnosing a bad solve.
The network-day sweep is the exception. Each formulation carries its own solver in NEMX.NBenchmark.FORMULATIONS, because an LP formulation wants HiGHS and a non-linear one wants Ipopt, and mixing them is never what you want. Change it there rather than by flag.
scripts/zbenchmark/ — the zonal reconstruction
General drivers
| Script | Purpose |
|---|---|
zonal_simple_example.jl | Two units, one region, no data required. Run this first. |
run_zonal_benchmark.jl | Sweep intervals and score every price against AEMO's published ROP. |
plot_zonal_benchmark.jl | Figures and tables from that sweep. |
export_interval_csvs.jl | Dump every market input for one interval. |
run_bess_event_study.jl | Per-unit local prices, offers and binding constraints across a window. |
analyse_bess_event_study.jl | The gated investigation over those results, emitting LaTeX tables. |
plot_bess_event_study.jl | The event figure, with a symmetric-log price axis. |
Named runs
These fix the arguments of a study that is referred to by name, so that "run the July 2024 benchmark" is one command and always means the same thing. Each is a thin wrapper: it sets defaults and includes the general driver, so there is one implementation of the sweep rather than several. Anything given on the command line still wins.
| Script | What it fixes |
|---|---|
run_historical_dispatch_jul2024.jl | July 2024, 10 random intervals, tag 2024_07 |
run_historical_dispatch_sep2025.jl | September 2025, 10 random intervals, tag 2025_09 |
run_historical_dispatch_sep2025_3day.jl | September 2025, 864 consecutive intervals (three trading days), tag 2025_09_3day |
run_bess_event_nov2025_2day.jl | 20–21 November 2025, 576 consecutive intervals (two days), NSW1 |
julia --project=. scripts/zbenchmark/zonal_simple_example.jl
julia --project=. scripts/zbenchmark/run_historical_dispatch_jul2024.jl --download
julia --project=. scripts/zbenchmark/run_historical_dispatch_sep2025_3day.jl
julia --project=. scripts/zbenchmark/export_interval_csvs.jl 2025-09-02T13:25A random sample gives an unbiased error estimate but cannot show anything that depends on the sequence. The consecutive runs exist for ramping, storage state, fast-start mode, and price spikes with the recovery that follows them.
scripts/nbenchmark/ — the network model
| Script | Purpose |
|---|---|
run_network_interval.jl | One interval, several formulations, printed side by side. |
run_network_day.jl | A full day; prices, decomposition and constraint ledger to CSV. |
run_network_day_20250902.jl | Named run: the reference network day, 2025-09-02, 288 intervals, all formulations. |
run_ac_recovery.jl | What it costs to make a DC dispatch AC-feasible. |
plot_network_day.jl | Figures and tables from a network-day sweep. |
fix_snem2000_case.jl | One-off repair of the raw 2000-bus case. |
julia --project=. scripts/nbenchmark/run_network_interval.jl 2025-09-02T12:05 DCP,ACP
julia --project=. scripts/nbenchmark/run_network_day_20250902.jl
julia --project=. scripts/nbenchmark/run_ac_recovery.jl 2025-09-02T04:05 288Validate a window with DCP before widening the formulation list: the AC formulations and the convex relaxations are 2000-bus non-linear programs taking tens of seconds each, so the full list over a day is an overnight job.
scripts/opffcas/ — OPF with FCAS
| Script | Purpose |
|---|---|
run_opffcas.jl | AC/DC OPF with FCAS across formulations, with regional price tables. |
run_opffcas_arpst.jl | The same with phase-shifting transformers active. |
compute_marginal_loss_factors.jl | Per-bus MLFs by finite-differencing an AC power flow. |
build_market_scenario.jl | Raw AEMO bid CSVs into an OPFFCAS scenario. |
julia --project=. scripts/opffcas/run_opffcas.jl --scenario=s1
julia --project=. scripts/opffcas/build_market_scenario.jl --scenario=s1 \
--market-dir=/data/aemo/2025-09compute_marginal_loss_factors.jl needs PowerModelsACDCsecurityconstrained, which is not in the General registry; see Installation.
Long runs
A 288-interval sweep across every formulation runs overnight. Three things make that survivable.
Checkpointing. Results are flushed every --checkpoint intervals, so an interrupted run leaves usable output. run_ac_recovery.jl goes further and resumes: intervals already in its CSV are kept and not re-solved, so re-running the same command continues where it stopped.
Isolation. Each interval is solved in its own try. One unsolvable or malformed interval costs that interval, not the run, and appears in the failure list at the end.
Scratch output. Scripts that write large result sets default --out-dir to a local scratch path and copy into the project tree once, at the end. A file rewritten every few minutes for hours is not reliably served back at its newest version by a cloud sync client — a completed sweep has been seen leaving only its first two thirds on disk. The final copy is also wrapped: if the data directory is read-only or unreachable, the failure is reported and the scratch copy is named, rather than an exception discarding hours of work.
Writing a new script
The skeleton, in the house style:
# =============================================================================
# my_script.jl
#
# PURPOSE
# One paragraph on what this produces and why.
#
# ARGUMENTS
# Positional Env Default Meaning
# ------------ ------------- ----------------- ------------------------
# 1 NEMX_START 2025-09-02T04:05 first interval
#
# Options Env Default Meaning
# ------------ ------------- ----------------- ------------------------
# --solver= NEMX_SOLVER highs highs | ipopt | scs
#
# EXAMPLES
# julia --project=. scripts/zbenchmark/my_script.jl
# =============================================================================
using NEMX
using Dates
const ZB = NEMX.ZBenchmark
const START = script_datetime(script_positional(1, "NEMX_START", "2025-09-02T04:05"))
const SOLVER = script_option("solver", "highs")
ZB.SOLVER_FACTORY[] = select_solver(SOLVER; silent = !script_flag("verbose-solver"))
print_banner("What this run is", "start" => START, "solver" => SOLVER)
print_flag_values(ZB, :LOSS_MODEL_FROM_XML, :ZONAL_MLF_KEEP_SCALING)
# ... the work ...Three conventions to keep:
- Document every argument in the header, with its environment variable and its default. The header is what someone reads; the body is what they read afterwards.
- Print the configuration before doing any work, including any behavioural flag the script changes.
- Take the solver as an argument. A hard-coded solver makes a script unusable to anyone without it.