PowerIO.jl

PowerIO

Julia bindings for the PowerIO Rust core: parse MATPOWER / PSS/E / PowerWorld / PowerModels JSON / egret JSON case files, convert between any pair (byte exact on a same-format round trip, maximal fidelity otherwise), and materialize an immutable Network, all through the powerio-capi C ABI.

Parse once with parse_file → Network, then read or transform it, all over the same C ABI:

• the rich, lossless element tables via the JSON transport (every field + extras, costs, storage, HVDC): the accessors and to_json.
• to_dense: the numeric tables as dense typed arrays for matrix assembly, straight from the C ABI extractors, no JSON.
• to_arrow: one table over the Arrow C Data Interface (owned columns by default; zero copy with copy=false).

to_normalized derives a per-unit / radian / filtered copy that preserves source bus ids, and to_matpower / convert_file serialize back out.

read_gridfm / read_gridfm_scenarios read a gridfm-datakit Parquet dataset back into a Network (the ML→classical return leg; lossy but power-flow-complete, needs powerio-capi built --features gridfm).

At first use the binding checks the library's ABI version (pio_abi_version) against the version it targets (PIO_ABI_VERSION) and refuses a stale or mismatched library with an error stating both versions.

The C library resolves automatically: the bundled lazy artifact, or a sibling powerio build during development. Point at a custom build with set_library! or the POWERIO_CAPI environment variable.

ArrowColumn{T} <: AbstractVector{T}

One zero-copy column of to_arrow(...; copy=false): a view into the producer's buffer that roots the shared ArrowBuffers owner, so the column alone keeps the memory alive — extracting it from its ArrowTable is safe. collect it for a plain owned Vector.

ArrowTable

The zero-copy result of to_arrow(...; copy=false): a NamedTuple of ArrowColumn views into the producer's buffers, behind property access (t.id, t.from, ...). Every property name resolves to a column — including t.columns, which would look up a column called columns — so the NamedTuple itself comes from the unexported accessor PowerIO.columns(t). The columns and the table each root the shared buffer owner, which frees the buffers once none of them is reachable; a column extracted from the table is safe on its own. close(t) frees the buffers eagerly instead of waiting for GC. (The default copy=true returns a plain NamedTuple of owned Vectors instead, no ArrowTable involved.)

Network

An immutable view of a parsed case, materialized from the C ABI's JSON transport. Raw MATPOWER units and 1-based bus ids, mirroring powerio's Network: buses, loads, shunts, branches, generators, storage, and hvdc tables plus base_mva, name, and source_format (the format it was read from). For now the tables are the parsed JSON (net.data).

A Network from parse_file also keeps its live Rust NetworkHandle (net.handle), so the to_* transforms (to_normalized, to_dense, to_matpower, to_arrow) work straight off it. The handle's finalizer frees the Rust case once the Network is unreachable. A Network constructed from a bare JSON3.Object has handle === nothing; the accessors and to_json work on it, but the handle-only transforms error.

NetworkHandle

Opaque handle to a parsed network inside the Rust core. Freed by its finalizer; you normally go straight to parse_file, which returns a [Network].

close(t::ArrowTable)

Release the producer's buffers now instead of at GC. Every ArrowColumn of t is invalid afterwards; reading one is undefined behavior. Idempotent (the release callbacks NULL themselves).

abi_version() -> UInt32

The ABI version the resolved C library was built with (see pio_abi_version). Compared against PIO_ABI_VERSION, the version this binding targets.

arrow_available() -> Bool

True if the resolved C library exports pio_export_arrow (built --features arrow).

Branches (lines and transformers), in source order.

bus_type_code(kind) -> Int

Map the canonical bus-type string ("PQ", "PV", "REF", "ISOLATED") to the MATPOWER code (1, 2, 3, 4). The strings are the Rust core's BusType::as_str values.

Buses, in source order (1-based ids preserved). See the accessor-surface note.

convert_file(path, to; from=nothing) -> (text, warnings)

Convert path to format to. All five formats read and write, so any pair converts. A same-format conversion is byte exact; a cross-format one is maximal fidelity and reports whatever the target can't carry in warnings. Tokens (case-insensitive): "matpower"/"m", "powermodels-json"/"powermodels"/"pm", "egret-json"/"egret", "psse"/"raw", "powerworld"/"aux". from overrides extension inference (needed to tell egret and PowerModels .json apart).

from_json(text) -> Network

Rebuild a live Network from the JSON transport produced by to_json. The result has a Rust handle, so to_* transforms work on it.

from_powermodels(data) -> Network

Build a PowerIO Network from PowerModels network data. data may be a Julia dictionary / NamedTuple or a JSON string.

Generators, one per machine (bus repeats when a bus has several).

gridfm_available() -> Bool

True if the resolved C library exports pio_read_gridfm (built --features gridfm).

Two-terminal HVDC lines (MATPOWER dcline); empty unless the source carries them.

is_radial(net) -> Bool

Whether the in-service topology is radial (a forest), as the C ABI computes it (pio_is_radial). The same quantity as to_dense(net).is_radial, without building the dense view. Needs net's live Rust handle (from parse_file).

library_available() -> Bool

True if the C ABI library resolves and is ABI compatible with this binding (see abi_version).

library_version() -> String

The powerio-capi crate version string the resolved library reports (e.g. "0.0.1"). Informational; abi_version is the compatibility check.

First-class loads (PSS/E and PowerModels keep several per bus; MATPOWER splits its bus row).

n_components(net) -> Int

Number of connected components of the in-service topology, as the C ABI computes it (pio_n_components). The same quantity as to_dense(net).n_components, without building the dense view. Needs net's live Rust handle (from parse_file).

n_gens(net) -> Int

Number of generator rows (one per machine; bus repeats). Matches pio_n_gens: every row, not in-service-filtered.

network_name(net) -> String

The case name carried through from the source file.

parse_ac_power_data(input; from=nothing, filtered=true, T=Float64) -> NamedTuple

Return the NamedTuple shape consumed by ExaModelsPower's build_polar_opf, build_rect_opf, and build_dcopf. input may be a Network or a path.

parse_file(path; from=nothing) -> Network
parse_file(io::IO, format::AbstractString) -> Network

Parse a case into a [Network].

From a file path the format is inferred from the extension unless from is given (needed to tell egret and PowerModels .json apart). From an io stream the format is required (there is no extension); parse in-memory text by wrapping it, parse_file(IOBuffer(text), "matpower").

Accepted format tokens (case-insensitive): "matpower"/"m", "powermodels-json"/ "powermodels"/"pm", "egret-json"/"egret", "psse"/"raw", "powerworld"/"aux".

parse_str(text, format="matpower") -> Network

Parse in-memory case text into a Network. This is the string sibling of parse_file(io, format), matching the Rust, Python, and C interfaces.

read_gridfm(dir; scenario=0) -> (; network::Network, scenario::Int, warnings::Vector{String})

Read one scenario of a gridfm-datakit Parquet dataset back into a Network — the inverse of the gridfm writer. dir resolves leniently: the raw/ directory holding the parquet files, a <case>/ directory with a raw/ child, or a parent with one */raw/ child. scenario selects one snapshot from a batch (0, the base case, by default).

The read is lossy but power-flow-complete: it recovers bus types, voltages and limits, nodal load and shunt totals, generator dispatch and bounds, branch r/x/b/tap/shift/rate_a/angle-limits, and base_mva — enough to write a runnable case — but not original bus ids (synthesized 1..n), per-element load/shunt granularity, piecewise/cubic costs, or HVDC/storage. What it can't recover is listed in warnings.

The returned network carries a live Rust handle, so the to_* transforms work on it. Needs powerio-capi built --features gridfm; see gridfm_available. For every scenario in a batch use read_gridfm_scenarios.

read_gridfm_scenarios(dir) -> Vector

Read every scenario of a gridfm dataset, one read_gridfm result per scenario id (ascending) over the shared topology — the read side of a scenario batch. Each scenario is rebuilt independently, so two may differ in branch status, bus types, and reference bus. See read_gridfm for the lenient directory resolution and the fidelity contract.

reference_bus_id(net) -> Union{Int,Nothing}

The 1-based id of the reference (slack) bus, or nothing unless exactly one bus has kind == "REF". This mirrors the "exactly one" rule of the C ABI's pio_reference_bus (which returns a dense 0-based index, not an id), but returns the 1-based id space the other accessors use.

reference_bus_indices(net) -> Vector{Int}

The dense [0, n) indices of every reference (slack) bus, in dense bus order. Unlike reference_bus_id — which returns a single 1-based id and only when exactly one bus is REF — this returns all of them (zero, one, or many) as dense indices. Map an index back to a 1-based id with to_dense(net).bus_ids. Needs net's live Rust handle (from parse_file).

set_library!(path)

Point PowerIO at a locally built libpowerio_capi (cargo build -p powerio-capi --release in the PowerIO Rust tree → target/release/libpowerio_capi.{dylib,so}). A development override that wins over the bundled artifact.

First-class bus shunts.

source_format(net) -> String

The format the case was read from, verbatim from the Rust SourceFormat enum: one of "Matpower", "PowerModelsJson", "EgretJson", "Psse", "PowerWorld", "InMemory", "Normalized" (the last is the output of to_normalized).

First-class storage units; empty unless the source carries them (PowerModels, egret).

to_arrow(net::Network, table::Symbol; copy=true) -> NamedTuple | ArrowTable
to_arrow(path, table::Symbol; from=nothing, copy=true) -> NamedTuple | ArrowTable

Export one raw network table over the Arrow C Data Interface. table is :bus, :branch, :gen, :load, or :shunt; the columns are the parsed network fields with 1-based (external) bus ids, the same id space as to_dense. Takes a parsed Network (via its live handle) or a path to parse first. Needs powerio-capi built --features arrow; see arrow_available.

copy=true (default) returns a NamedTuple of owned Julia Vectors and releases the producer before returning: plain arrays, no lifetime caveat. copy=false returns a zero-copy ArrowTable of ArrowColumn views; each column roots the shared buffers, so columns may outlive the table, and close(t) frees the buffers eagerly. Both support result.<column> access, but only the copy=true NamedTuple is Tables.jl-shaped (flows into Arrow.write, DataFrame, etc.); collect a zero-copy column for an owned Vector. For the numeric tables alone, to_dense is a copy-free, unsafe_wrap-free fast path.

to_dense(net::Network) -> NamedTuple
to_dense(path; from=nothing) -> NamedTuple

Pull a case's numeric tables as dense typed arrays straight from the C ABI, skipping the JSON transport (the fast path for matrix assembly). Takes a parsed Network (via its live handle) or a path to parse first (which never builds the JSON view). Fields:

• n, m, ng — bus / branch / generator counts.
• base_mva — system base.
• bus_ids::Vector{Int64} — 1-based bus ids in dense order; row k of every per-bus table is bus bus_ids[k]. Invert it to map a 1-based endpoint id to a dense row.
• branch — NamedTuple of from, to (1-based bus ids), r, x, b, tap, shift (raw MATPOWER units, degrees, total charging, raw tap), and in_service::Vector{UInt8}.
• gen — NamedTuple of bus (1-based id, one row per machine), pg, pmax, pmin (MW), in_service.
• demand, shunt — NamedTuples of per-bus (pd, qd) and (gs, bs) in dense order.
• reference_bus::Int — dense 0-based index of the single reference bus, or -1.
• n_components::Int, is_radial::Bool — connectivity of the in-service topology.

For the rich, lossless element tables (costs, extras, storage, HVDC) use the accessors on a parse_file Network; for self-describing columnar export use to_arrow.

to_format(net::Network, to) -> (text, warnings)

Serialize a parsed network to format to without reparsing the input file. Returns the target text and any fidelity warnings.

to_json(net::Network) -> String

Serialize net to the C ABI's JSON transport, the same text from_json reads back. Uses the live handle when present, else the cached net.data.

to_matpower(net::Network) -> String

Serialize net to MATPOWER .m text, byte exact when the input was MATPOWER. For a file in one shot use convert_file(path, "matpower").

to_normalized(net::Network) -> Network

A computation-ready copy of net: per unit (powers ÷ base_mva), angles in radians, transformer tap 0 → 1, out-of-service and isolated elements dropped, source bus ids preserved, and bus types inferred (a bus with a surviving generator keeps REF if the source marked it so, else becomes PV; a generator-less bus becomes PQ). source_format of the result is "Normalized".

Needs net's live Rust handle (from parse_file). Errors if base_mva is not positive or no reference bus can be established.

to_powerdata(net; filtered=true, T=Float64) -> NamedTuple
to_powerdata(path; from=nothing, filtered=true, T=Float64) -> NamedTuple

Return a NamedTuple in ExaPowerIO's PowerData shape: version, baseMVA, bus, gen, branch, arc, and storage. Rows use the field names ExaModelsPower reads. With the default filtered=true, values are derived from to_normalized: bus_i preserves the source bus id, powers are per unit, branch angle fields are radians, and branch/generator bus references are indices into the bus vector.

to_powermodels(net::Network) -> Dict{String,Any}

Convert a parsed network to a PowerModels network data dictionary through the PowerIO writer. This is the post-parse network data shape PowerModels.jl consumes.

write_pypsa_csv_folder(net::Network, out_dir) -> (out_dir, warnings)

Write net as a PyPSA CSV folder under out_dir (created if absent) — the directory-shaped inverse of parse_file(out_dir; from="pypsa-csv"), where the other writers (to_format, convert_file) emit a single text document. Returns the output directory and any fidelity warnings the writer reports for fields the PyPSA static-network CSV schema can't carry. Needs net's live Rust handle (from parse_file).