Changes proposed in this Pull Request
Land transport is split from a single passenger-car aggregate into five motor
vehicle segments:
- passenger cars (
pc),
- powered two-wheelers (
ptw),
- buses/coaches (
bus),
- light commercial vehicles (
lcv), and
- heavy goods vehicles (
hgv)
so that fleet size, driving profile, propulsion-mix shares,
charging behaviour and EV characteristics can differ per segment.
- Demand build (
build_transport_demand.py) is now km-based per segment. Total
vehicle-km per country come from JRC-IDEES road (and, for bus/hgv, a share
of non-electrified rail moved onto road via load-factor ratios); Propulsion-based
correction of energy totals can be removed hence; the weekly demand shape derives
from per-segment BASt profiles.
- BASt profiles (
build_mobility_profiles.py) now also produce mot, bus,
lfw and lkw weekly profiles alongside kfz/pkw.
- Energy totals (
build_energy_totals.py) extract per-segment vehicle stock,
vehicle-km driven, load factors and an hgv efficiency from IDEES road and rail
activity sheets.
- Network build (
prepare_sector_network.py) loops over segments and builds a
segment-suffixed EV battery bus, load, BEV charger, DSM store and V2G link for
each. transport_electric_efficiency for bus/hgv is taken from
technology-data. low_voltage_charging selects which segments connect to the
LV distribution grid.
- Config: BEV/EV scalars and the propulsion-mix share dicts become
per-segment mappings; new options bev_dem_smoothing (was the previously
unconditional EV power smoothing) and low_voltage_charging. Pydantic models
in scripts/lib/validation/config/sector.py and config/schema.default.json
updated to accept value | {segment: value}.
- Plotting:
tech_colors for the per-segment EV battery, land transport demand, land transport EV, BEV charger and V2G carriers.
Status: this is a work in progress
Working end-to-end for the electric segments. Known gaps before this is
merge-ready are listed in the collapsed section below; the headline ones:
- No aggregation switch yet. The pipeline is hard-wired to five segments.
The intended pre-commit state is a config switch that (a) collapses all
segments into the kfz aggregate as today's default behaviour, (b) restricts
DSM to pkw only in that mode, while (c) keeping the new km-based demand
inputs. That switch is not implemented.
- ICE and fuel-cell segments are not disaggregated.
add_ice_cars and
add_fuel_cell_cars are still called with the pc share only (marked
# TEMP: pc) and currently sit inside the per-segment loop, so they misbehave
for a multi-segment run. Needs either a clean aggregate call outside the loop
or a proper per-segment treatment.
Checklist
Required:
If applicable:
Notes on the checklist: the config validation models are updated. No new rules were added (only new outputs on build_mobility_profiles and new inputs on build_transport_demand). The BASt source is unchanged: only additional columns are read. Release note and doc/ updates are pending.
Implementation notes — data flow, per-segment wiring, and open gaps (verbose, AI-assisted draft)
1. What changed, stage by stage
1a. Weekly profiles — build_mobility_profiles.py
BASt hourly counting-point data is aggregated to weekly profiles for the five
segments in addition to the existing kfz (all motor vehicles) and pkw:
| output |
BASt columns summed (both directions) |
meaning |
kfz |
KFZ_R1 + KFZ_R2 |
all motor vehicles (unchanged) |
pkw |
Pkw_* + PmA_* |
passenger cars incl. cars with trailer (PmA) → pc |
mot |
Mot_* |
motorcycles → ptw |
bus |
Bus_* |
buses/coaches → bus |
lfw |
Lfw_* |
light commercial vehicles → lcv |
lkw |
LoA_* + Lzg_* + Son_* |
lorry-with-trailer + tractor units + special → hgv |
The rule (rules/build_sector.smk) gains mot/bus/lfw/lkw outputs and
build_transport_demand gains the matching traffic_data_Mot/Bus/Lfw/Lkw
inputs. MOBILITY_PROFILES_DATASET["folder"] is now wrapped in Path(...) so
the / operator works.
1b. Energy / activity totals — build_energy_totals.py
New columns pulled from JRC-IDEES TrRoad_act, TrRoad_ene, TrRail_act,
TrRail_ene:
Number <segment>: vehicle stock for all five segments (was: passenger cars
only).
mio km-driven <segment>: road vehicle-km for all five segments.
mio km-driven Rail / Rail passenger / Rail freight.
load factor for Rail passenger, Rail freight, Heavy goods vehicles,
Motor coaches, buses and trolley buses.
heavy goods efficiency (kgoe/100km → ktoe/100km, like the existing
passenger car efficiency).
Unit handling: Number*, mio km-driven* and load factor* are excluded from
the ktoe→TWh scaling (str.fullmatch pattern list); efficiency columns are
divided by 1e6. Gap-filling now loops over car_cols / stats instead of the
single number cars / average fuel efficiency series. The renamed output
column is Number Passenger cars (old number cars is gone).
Gap / risk
- The Swiss patch only renames
passenger cars → Number Passenger cars; the
other four Number * / mio km-driven * columns are NaN for CH and rely
entirely on the per-column averaged gap-fill. Confirm that is acceptable.
1c. Demand time series — build_transport_demand.py
build_transport_demand was rewritten:
get_shape(traffic_fn) factored out: returns the normalised weekly shape.
- Per-segment total demand in
mio km:
pc, ptw, lcv: straight from mio km-driven <segment>.
bus: mio km-driven buses plus non_elec_rail · mio km-driven Rail passenger · (load factor Rail passenger / load factor bus).
hgv: mio km-driven HGV plus non_elec_rail · mio km-driven Rail freight · (load factor Rail freight / load factor HGV), where
non_elec_rail = 1 − electricity rail / total rail.
get_demand(profile, total, nyears, seg) → profile · total · 1e4 · nyears,
wrapped in a MultiIndex column level keyed by segment.
- Output is a wide frame with a two-level column index
(segment, node).
bev_availability_profile and bev_dsm_profile likewise produce
(segment, node) columns; availability reads bev_avail_max[seg] /
bev_avail_mean[seg], DSM reads bev_dsm_restriction_time[seg] /
bev_dsm_restriction_value[seg].
Behaviour change worth calling out
- The ICE heating/cooling demand correction (
dd_ICE, ice_correction) and
the whole temperature/transport_degree_factor block were removed from the
demand build. Demand is now purely km-based; temperature correction survives
only inside add_EVs via get_temp_efficency. FC and ICE demand no longer get
the build-stage correction at all.
airtemp_fn is still a parameter of build_transport_demand but is unused.
1d. Network — prepare_sector_network.py
add_EVs gains car_efficiency and transport_type parameters. Every added
component is suffixed with the segment: EV battery {t}, land transport EV {t}, BEV charger {t}, EV battery {t} store, V2G {t}. Carriers are
segment-specific too.
car_efficiency is passed in (no longer read from options inside add_EVs).
- The rolling-average power smoothing in
add_EVs is now gated on
options["bev_dem_smoothing"] (previously unconditional).
get_car_efficiencies() builds a DataFrame of 100km/MWh efficiencies:
pc/ptw/lcv from options["transport_electric_efficiency"][seg]; bus/hgv
from technology-data (costs.loc[car_keys[engine][seg], "efficiency"].mean(),
converted from kWh/km). engine_types is hard-coded to ["electric"] only.
add_land_transport: reads the three files with header=[0, 1], reindexes to
nodes on column level 1, loops for transport_type in transport_types and
calls add_EVs(...) per segment.
insert_electricity_distribution_grid: BEV chargers / V2G are moved to
" low voltage" only for carriers matching
BEV charger (pc|ptw|bus|lcv|hgv) filtered by options["low_voltage_charging"].
check_land_transport_shares now takes the shares DataFrame and warns per
column whose propulsion shares do not sum to 1.
1e. Config + validation
scripts/lib/validation/config/sector.py: _TRANSPORT_SEGMENTS and
_segment_share_default() helpers; the affected fields become
float | dict[str, float] / bool | dict[str, bool] /
dict[int, float] | dict[str, dict[int, float]] with per-segment
default_factory values.
bev_dsm (old scalar bool, "participate in DSM") keeps its name and meaning
but is now per-segment. bev_dem_smoothing is a new key that took over the
power-profile smoothing which used to be unconditional.
- Numeric defaults changed for existing single-value options, e.g.
bev_dsm_availability 0.5 → ~0.33–0.45, bev_charge_rate 0.011 → 0.0266 (pc),
transport_electric_efficiency 53.19 → 80.0 (pc). Needs a release note with the sources for the new numbers
config/schema.default.json mirrors the pydantic changes with anyOf
branches; both copies (sector and the scenario-override block) are updated.
2. Data-flow summary
×
ばつ weekly shape
×ばつ 1e4
×ばつ nyears ├─► transport_demand_s_{clusters}.csv columns = (segment, node), unit [100 km] ├─► avail_profile_s_{clusters}.csv columns = (segment, node) └─► dsm_profile_s_{clusters}.csv columns = (segment, node) ▼ prepare_sector_network.add_land_transport for segment in {pc, ptw, bus, lcv, hgv}: shares[engine, segment] from land_transport_<engine>_share[segment] electric > 0 ─► add_EVs(..., car_efficiency=get_car_efficiencies()[electric, segment], transport_type=segment) EV battery {seg} bus + load + BEV charger {seg} (+ EV battery {seg} store + V2G {seg} if bev_dsm[seg] / v2g[seg]) fuel_cell > 0 ─► add_fuel_cell_cars(... pc share ...) ◄── TEMP, not segmented ice > 0 ─► add_ice_cars(... pc share ...) ◄── TEMP, not segmented insert_electricity_distribution_grid: BEV charger {seg} / V2G {seg} → " low voltage" if seg in low_voltage_charging">
BASt hourly counts ──build_mobility_profiles──► kfz, pkw, mot, bus, lfw, lkw (weekly shape)
JRC-IDEES road+rail ─build_energy_totals──────► Number <seg>, mio km-driven <seg>,
mio km-driven Rail{,passenger,freight},
load factor {Rail p/f, HGV, bus},
passenger car / heavy goods efficiency
│
├─ build_transport_data ──► transport_data.csv (per-country stock, load factors,
│ average fuel efficiency)
▼
build_transport_demand
nodal split by population ─► nodal_transport_data
per-segment total mio km ─► + rail-passenger→bus, rail-freight→hgv (non-elec share only)
×ばつ weekly shape ×ばつ 1e4 ×ばつ nyears
├─► transport_demand_s_{clusters}.csv columns = (segment, node), unit [100 km]
├─► avail_profile_s_{clusters}.csv columns = (segment, node)
└─► dsm_profile_s_{clusters}.csv columns = (segment, node)
▼
prepare_sector_network.add_land_transport
for segment in {pc, ptw, bus, lcv, hgv}:
shares[engine, segment] from land_transport_<engine>_share[segment]
electric > 0 ─► add_EVs(..., car_efficiency=get_car_efficiencies()[electric, segment],
transport_type=segment)
EV battery {seg} bus + load + BEV charger {seg}
(+ EV battery {seg} store + V2G {seg} if bev_dsm[seg] / v2g[seg])
fuel_cell > 0 ─► add_fuel_cell_cars(... pc share ...) ◄── TEMP, not segmented
ice > 0 ─► add_ice_cars(... pc share ...) ◄── TEMP, not segmented
insert_electricity_distribution_grid:
BEV charger {seg} / V2G {seg} → " low voltage" if seg in low_voltage_charging
3. The aggregation switch (not yet implemented)
Target design for the pre-commit state:
- New config flag, e.g.
sector.land_transport_segmentation: false (default),
choosing between:
- aggregated: one segment, the
kfz weekly shape, one set of scalar
parameters, demand still taken from the km-based totals summed over segments
(overwriting the legacy energy-based path).
- segmented: the five-segment path in this PR.
- In aggregated mode, DSM is added for
pkw only (the historical assumption),
i.e. the DSM store / V2G are built for the passenger-car equivalent and skipped
for the rest.
- Both modes consume the same km-based inputs so the demand build does not fork.
Until that exists, config.default.yaml forces the five-segment path on every
run.
4. ICE / FC segmentation (open question)
car_keys in prepare_sector_network.py already lists technology-data cost
entries for fuel_cell and ice bus/hgv vehicles, so the efficiency lookup
generalises. What is missing:
get_car_efficiencies() limited to engine_types = ["electric"]; extend to
fuel_cell / ice and feed the per-segment efficiency into
add_fuel_cell_cars / add_ice_cars (both need a car_efficiency /
transport_type parameter, mirroring add_EVs).
- Decide the modelling scope: do
ptw / bus / lcv / hgv get their own
H2 / oil demand buses and carriers, or is FC/ICE kept aggregate while only EV
is segmented? The former multiplies carriers by 5; the latter needs the FC/ICE
calls pulled out of the per-segment loop and fed a summed p_set and a
fleet-weighted share.
5. Pre-review cleanup checklist
- Aggregation switch (
kfz fallback, pkw-only DSM) implemented and default.
- Scalar config values normalised to per-segment dicts at load time (or the
union / schema text dropped).
- FC / ICE calls moved out of the per-segment loop; decision on FC/ICE
segmentation recorded.
get_car_efficiencies() takes options / costs as parameters.
- Re-wire
traffic_data_KFZ rule input parameterfor the aggregation switch.
- Sources / citations for the new numeric defaults in the release note.
doc/ updates: configuration.md (new + changed options), sector.md
transport section, preparation.md / rules doc for the new profile outputs.
- Release note entry in
doc/release_notes.md (breaking config changes:
bev_dsm semantics, bev_dem_smoothing rename, per-segment defaults).
Uh oh!
There was an error while loading. Please reload this page.
Changes proposed in this Pull Request
Land transport is split from a single passenger-car aggregate into five motor
vehicle segments:
pc),ptw),bus),lcv), andhgv)so that fleet size, driving profile, propulsion-mix shares,
charging behaviour and EV characteristics can differ per segment.
build_transport_demand.py) is now km-based per segment. Totalvehicle-km per country come from JRC-IDEES road (and, for
bus/hgv, a shareof non-electrified rail moved onto road via load-factor ratios); Propulsion-based
correction of energy totals can be removed hence; the weekly demand shape derives
from per-segment BASt profiles.
build_mobility_profiles.py) now also producemot,bus,lfwandlkwweekly profiles alongsidekfz/pkw.build_energy_totals.py) extract per-segment vehicle stock,vehicle-km driven, load factors and an hgv efficiency from IDEES road and rail
activity sheets.
prepare_sector_network.py) loops over segments and builds asegment-suffixed EV battery bus, load, BEV charger, DSM store and V2G link for
each.
transport_electric_efficiencyforbus/hgvis taken fromtechnology-data.
low_voltage_chargingselects which segments connect to theLV distribution grid.
per-segment mappings; new options
bev_dem_smoothing(was the previouslyunconditional EV power smoothing) and
low_voltage_charging. Pydantic modelsin
scripts/lib/validation/config/sector.pyandconfig/schema.default.jsonupdated to accept
value | {segment: value}.tech_colorsfor the per-segmentEV battery,land transport demand,land transport EV,BEV chargerandV2Gcarriers.Status: this is a work in progress
Working end-to-end for the electric segments. Known gaps before this is
merge-ready are listed in the collapsed section below; the headline ones:
The intended pre-commit state is a config switch that (a) collapses all
segments into the
kfzaggregate as today's default behaviour, (b) restrictsDSM to
pkwonly in that mode, while (c) keeping the new km-based demandinputs. That switch is not implemented.
add_ice_carsandadd_fuel_cell_carsare still called with thepcshare only (marked# TEMP: pc) and currently sit inside the per-segment loop, so they misbehavefor a multi-segment run. Needs either a clean aggregate call outside the loop
or a proper per-segment treatment.
Checklist
Required:
doc/release_notes.rst.If applicable:
scripts/lib/validation.doc/*.rstfiles.Implementation notes — data flow, per-segment wiring, and open gaps (verbose, AI-assisted draft)
1. What changed, stage by stage
1a. Weekly profiles —
build_mobility_profiles.pyBASt hourly counting-point data is aggregated to weekly profiles for the five
segments in addition to the existing
kfz(all motor vehicles) andpkw:kfzKFZ_R1 + KFZ_R2pkwPkw_* + PmA_*PmA) →pcmotMot_*ptwbusBus_*buslfwLfw_*lcvlkwLoA_* + Lzg_* + Son_*hgvThe rule (
rules/build_sector.smk) gainsmot/bus/lfw/lkwoutputs andbuild_transport_demandgains the matchingtraffic_data_Mot/Bus/Lfw/Lkwinputs.
MOBILITY_PROFILES_DATASET["folder"]is now wrapped inPath(...)sothe
/operator works.1b. Energy / activity totals —
build_energy_totals.pyNew columns pulled from JRC-IDEES
TrRoad_act,TrRoad_ene,TrRail_act,TrRail_ene:Number <segment>: vehicle stock for all five segments (was:passenger carsonly).
mio km-driven <segment>: road vehicle-km for all five segments.mio km-driven Rail/Rail passenger/Rail freight.load factorforRail passenger,Rail freight,Heavy goods vehicles,Motor coaches, buses and trolley buses.heavy goods efficiency(kgoe/100km → ktoe/100km, like the existingpassenger car efficiency).Unit handling:
Number*,mio km-driven*andload factor*are excluded fromthe ktoe→TWh scaling (
str.fullmatchpattern list); efficiency columns aredivided by
1e6. Gap-filling now loops overcar_cols/statsinstead of thesingle
number cars/average fuel efficiencyseries. The renamed outputcolumn is
Number Passenger cars(oldnumber carsis gone).Gap / risk
passenger cars→Number Passenger cars; theother four
Number */mio km-driven *columns areNaNforCHand relyentirely on the per-column averaged gap-fill. Confirm that is acceptable.
1c. Demand time series —
build_transport_demand.pybuild_transport_demandwas rewritten:get_shape(traffic_fn)factored out: returns the normalised weekly shape.mio km:pc,ptw,lcv: straight frommio km-driven <segment>.bus:mio km-driven busesplusnon_elec_rail · mio km-driven Rail passenger · (load factor Rail passenger / load factor bus).hgv:mio km-driven HGVplusnon_elec_rail · mio km-driven Rail freight · (load factor Rail freight / load factor HGV), wherenon_elec_rail = 1 − electricity rail / total rail.get_demand(profile, total, nyears, seg)→profile · total · 1e4 · nyears,wrapped in a
MultiIndexcolumn level keyed by segment.(segment, node).bev_availability_profileandbev_dsm_profilelikewise produce(segment, node)columns; availability readsbev_avail_max[seg]/bev_avail_mean[seg], DSM readsbev_dsm_restriction_time[seg]/bev_dsm_restriction_value[seg].Behaviour change worth calling out
dd_ICE,ice_correction) andthe whole temperature/
transport_degree_factorblock were removed from thedemand build. Demand is now purely km-based; temperature correction survives
only inside
add_EVsviaget_temp_efficency. FC and ICE demand no longer getthe build-stage correction at all.
airtemp_fnis still a parameter ofbuild_transport_demandbut is unused.1d. Network —
prepare_sector_network.pyadd_EVsgainscar_efficiencyandtransport_typeparameters. Every addedcomponent is suffixed with the segment:
EV battery {t},land transport EV {t},BEV charger {t},EV battery {t}store,V2G {t}. Carriers aresegment-specific too.
car_efficiencyis passed in (no longer read fromoptionsinsideadd_EVs).add_EVsis now gated onoptions["bev_dem_smoothing"](previously unconditional).get_car_efficiencies()builds aDataFrameof100km/MWhefficiencies:pc/ptw/lcvfromoptions["transport_electric_efficiency"][seg];bus/hgvfrom technology-data (
costs.loc[car_keys[engine][seg], "efficiency"].mean(),converted from kWh/km).
engine_typesis hard-coded to["electric"]only.add_land_transport: reads the three files withheader=[0, 1], reindexes tonodeson column level 1, loopsfor transport_type in transport_typesandcalls
add_EVs(...)per segment.insert_electricity_distribution_grid: BEV chargers / V2G are moved to" low voltage"only for carriers matchingBEV charger (pc|ptw|bus|lcv|hgv)filtered byoptions["low_voltage_charging"].check_land_transport_sharesnow takes thesharesDataFrame and warns percolumn whose propulsion shares do not sum to 1.
1e. Config + validation
scripts/lib/validation/config/sector.py:_TRANSPORT_SEGMENTSand_segment_share_default()helpers; the affected fields becomefloat | dict[str, float]/bool | dict[str, bool]/dict[int, float] | dict[str, dict[int, float]]with per-segmentdefault_factoryvalues.bev_dsm(old scalarbool, "participate in DSM") keeps its name and meaningbut is now per-segment.
bev_dem_smoothingis a new key that took over thepower-profile smoothing which used to be unconditional.
bev_dsm_availability0.5 → ~0.33–0.45,bev_charge_rate0.011 → 0.0266 (pc),transport_electric_efficiency53.19 → 80.0 (pc). Needs a release note with the sources for the new numbersconfig/schema.default.jsonmirrors the pydantic changes withanyOfbranches; both copies (
sectorand the scenario-override block) are updated.2. Data-flow summary
3. The aggregation switch (not yet implemented)
Target design for the pre-commit state:
sector.land_transport_segmentation: false(default),choosing between:
kfzweekly shape, one set of scalarparameters, demand still taken from the km-based totals summed over segments
(overwriting the legacy energy-based path).
pkwonly (the historical assumption),i.e. the DSM store / V2G are built for the passenger-car equivalent and skipped
for the rest.
Until that exists,
config.default.yamlforces the five-segment path on everyrun.
4. ICE / FC segmentation (open question)
car_keysinprepare_sector_network.pyalready lists technology-data costentries for
fuel_cellandicebus/hgvvehicles, so the efficiency lookupgeneralises. What is missing:
get_car_efficiencies()limited toengine_types = ["electric"]; extend tofuel_cell/iceand feed the per-segment efficiency intoadd_fuel_cell_cars/add_ice_cars(both need acar_efficiency/transport_typeparameter, mirroringadd_EVs).ptw/bus/lcv/hgvget their ownH2 / oil demand buses and carriers, or is FC/ICE kept aggregate while only EV
is segmented? The former multiplies carriers by 5; the latter needs the FC/ICE
calls pulled out of the per-segment loop and fed a summed
p_setand afleet-weighted share.
5. Pre-review cleanup checklist
kfzfallback,pkw-only DSM) implemented and default.union / schema text dropped).
segmentation recorded.
get_car_efficiencies()takesoptions/costsas parameters.traffic_data_KFZrule input parameterfor the aggregation switch.doc/updates:configuration.md(new + changed options),sector.mdtransport section,
preparation.md/ rules doc for the new profile outputs.doc/release_notes.md(breaking config changes:bev_dsmsemantics,bev_dem_smoothingrename, per-segment defaults).