METS-R HPC module

METS-R HPC is the Python orchestration layer for interactive METS-R SIM experiments. It prepares run folders, launches one or more METS-R SIM containers, connects to each simulator over WebSocket, and provides helper clients for Kafka streams, CARLA co-simulation, METS-R Vis playback, and V2X network experiments through OMNeT++/Veins or Simu5G.

The repository is now organized around direct, scriptable workflows rather than one monolithic experiment runner. Most users start METS-R SIM with a JSON run configuration, create a METSRClient, and call query/control methods from a notebook or Python script.

What the module provides

clients/METSRClient.py: the main WebSocket client for METS-R SIM. It handles tick synchronization, query/control APIs, live METS-R Vis streaming, and offline trajectory-output discovery.
utils/util.py: run-configuration parsing, per-run folder preparation, Docker launch helpers, and offline visualization-server utilities.
clients/KafkaDataSender.py and clients/KafkaDataProcessor.py: producers and consumers for METS-R sensor, BSM, link travel-time, link-energy, and V2X metric streams.
clients/VeinsClient.py: a TCP JSON-lines client for the OMNeT++ bridge. It sends METS-R vehicle mobility and BSM payloads with sync_tick and receives delivery, latency, loss, and attack-event records.
utils/carla_util.py: helpers for CARLA vehicle spawning, coordinate conversion, route advancement, and co-simulation queue release.
utils/duckie_util.py: helper functions for Duckietown message conversion and synchronization.
models/AnomalyDetector.py: a lightweight example model for streaming data experiments.
tutorials/ and configs/: runnable examples and JSON run templates.
veins_bridge/omnetpp/: the included OMNeT++ bridge project, including abstract V2X profiles and the Simu5G Uu backend scaffold.

Quick start

Install the Python dependencies from the METS-R_HPC repository root:

pip install -r requirements.txt

Install Docker. On Docker Desktop for Windows or macOS, enable host networking under Settings > Resources > Network. METS-R SIM containers are launched with --net=host so the Python clients can connect to simulator services on localhost.

Run tutorials from the repository root so paths such as configs/..., data/..., and docker/... resolve correctly:

jupyter lab tutorials/basic_tutorial.ipynb

A minimal script follows the same pattern used in the notebooks:

from clients.METSRClient import METSRClient
from utils.util import read_run_config, prepare_sim_dirs, run_simulation_in_docker

config = read_run_config("configs/run_interactive_NYC.json")
prepare_sim_dirs(config)
run_simulation_in_docker(config)

client = METSRClient(
    host=config.metsr_host,
    port=config.ports[0],
    sim_folder=config.sim_dirs[0],
    timeout=300,
)

client.tick(10)
print(client.query_tick())
client.terminate()

Recommended entry points

File	Purpose
`tutorials/basic_tutorial.ipynb`	Starts a simulator, connects a client, advances ticks, queries state, controls vehicles and services, and streams to METS-R Vis.
`tutorials/advanced_commands.ipynb`	Longer command examples for co-simulation, dynamic infrastructure, ride-hailing control, Kafka, save/load, and cleanup.
`tutorials/security_examples.ipynb`	Local simulator cyber-range examples using direct `METSRClient` calls, Kafka where needed, and explicit before/after diagnostics.
`tutorials/security_sim5g_v2x_examples.ipynb`	Interactive METS-R plus Simu5G V2X attack workflow using `VeinsClient.sync_tick()`.
`tutorials/cosim_example.py`	CARLA/METS-R co-simulation and visualization script.

Run configurations

Run configurations are JSON files under configs/. The common fields include the Java path and options, METS-R SIM directory, host, random seeds, simulation step size, data files, output settings, and optional Kafka/V2X settings.

Important conventions:

num_simulations controls how many simulator instances are prepared.
prepare_sim_dirs(config) creates timestamped per-seed output folders and populates config.sim_dirs and config.ports.
run_simulation_in_docker(config) launches the simulator image from each prepared folder.
enable_trajectory_binary_write enables the current compact trajectory output format used by METS-R Vis.
json_output can still be enabled for legacy JSON trajectory files.
V2X configs such as configs/run_v2x_veins_Template.json add veins_host, veins_port, timeout settings, backend labels, and Kafka topics for V2X telemetry.

Interactive client APIs

METSRClient talks to one METS-R SIM instance over WebSocket. Unless noted otherwise, methods accept either scalar inputs or lists. When lists are passed, records are paired by index and the simulator response contains one DATA entry per requested record. Most control calls return the raw simulator response with top-level CODE and per-record STATUS fields.

Road IDs are SUMO/original road IDs. transform_coords=True asks METS-R SIM to convert between the simulator coordinate reference system and WGS84 longitude/latitude when the server-side endpoint supports it.

Ticks and lifecycle

Recent METS-R SIM builds include authoritative tick values in STEP, CTRL_reset, and CTRL_load responses. The client updates current_tick from those responses and can query the server tick when a step reply is slow or stale.

Method	Use
`query_tick()`	Return the current simulator tick reported by METS-R SIM.
`query_tick_status()`	Return server-side stepping status, including active-road stepping fields when enabled.
`tick(step_num=1, wait_forever=False, retry_interval=None, max_wait_seconds=None, poll_timeout=5, max_stalled_seconds=None)`	Advance the simulator and wait until the requested tick is reached. `wait_forever=True` still uses bounded polling so stalled simulators can be detected.
`reset()`	Reset the simulation to its initial state and resynchronize the client tick.
`save(filename)`	Save a full simulator snapshot to a zip archive.
`load(filename, reload_network=True)`	Restore a saved snapshot. Set `reload_network=False` when the network is unchanged and a faster state reload is sufficient.
`terminate()`	Ask the simulator process to end.
`close()`	Close only the client WebSocket connection.

State query APIs

Method	Use
`query_vehicle(id=None, private_veh=False, transform_coords=False)`	Return public/private vehicle ID lists, or full kinematic state for selected vehicles.
`query_on_road_vehicles(roadID=None)`	Return vehicles currently on selected road IDs, or grouped road records when no road is specified.
`query_active_roads()`	Return roads currently active in the simulator stepping set.
`query_taxi(id=None)`	Return electric taxi IDs or taxi state, including trip, passenger, battery, and remaining-distance fields.
`query_available_taxis(zoneID=None)`	Return taxis currently in the explicit available-dispatch pool, optionally filtered by zone.
`query_almost_finished_taxis(distance_threshold_miles=None, distance_threshold_meters=None, zoneID=None)`	Return taxis whose active trip is within a distance threshold of completion.
`query_bus(id=None)`	Return electric bus IDs or bus state, route, battery, stop, and passenger fields.
`query_road(id=None)`	Return road IDs or static and real-time road attributes.
`query_entering_vehicle_queue(roadID=None)`	Return vehicles waiting to enter co-simulation roads.
`query_cosim_entering_vehicle_queue()`	Return entering queues for all roads currently marked for co-simulation.
`query_centerline(id, lane_index=-1, transform_coords=False)`	Return road-level or lane-level centerline geometry.
`query_zone(id=None)`	Return zone IDs or demand, stock, location, and service counters.
`query_pending_requests(zoneID=None)`	Return pending taxi and bus requests across all zones or one zone.
`query_request(reqID)`	Return request status for one or more request IDs.
`query_pickup_taxi_info(reqID=None)`	Return taxi pickup assignments by request ID or all current pickup assignments.
`query_occupied_taxi_info(reqID=None)`	Return occupied taxi trip assignments by request ID or all current occupied assignments.
`query_signal(id=None)`	Return signal IDs or current phase state and next-update timing.
`query_signal_group(id=None)`	Map signal group or junction IDs to individual METS-R signal IDs.
`query_signal_between_roads(upstream_road, downstream_road)`	Return the signal controlling a road-to-road connection.
`query_chargingStation(id=None)`	Return charging station IDs or status, capacity, and prices.
`query_coSimVehicle()`	Return vehicles currently on co-simulation roads.
`query_route(orig_x, orig_y, dest_x, dest_y, transform_coords=False)`	Return the shortest route between two coordinate locations.
`query_k_routes(orig_x, orig_y, dest_x, dest_y, k, transform_coords=False)`	Return up to `k` coordinate-based route alternatives.
`query_route_between_roads(orig_road, dest_road)`	Return the shortest route between two road IDs.
`query_k_routes_between_roads(orig_road, dest_road, k)`	Return up to `k` road-to-road alternatives.
`query_road_weights(roadID=None)`	Return current routing weights for selected roads or all roads.
`query_bus_route(routeID=None)`	Return bus route names or stop-road sequences.
`query_route_bus(routeID=None)`	Return buses currently assigned to each route.
`query_routing_graph()`	Build a local `networkx.DiGraph` from road query results.

Co-simulation and vehicle control

Method	Use
`set_cosim_road(roadID)`	Mark one or more roads as externally controlled co-simulation roads.
`release_cosim_road(roadID)`	Return one or more co-simulation roads to normal METS-R control.
`enter_road_from_queue(vehID=None, roadID=None, private_veh=None, internal_vehicle_id=None, requests=None)`	Release queued vehicles onto co-simulation roads after inspecting `query_entering_vehicle_queue`.
`teleport_cosim_vehicle(vehID, x, y, bearing, speed=0, z=0.0, private_veh=False, transform_coords=False)`	Teleport a co-simulation vehicle to absolute coordinates with heading and speed.
`teleport_trace_replay_vehicle(vehID, roadID, laneID, dist=None, private_veh=False, x=None, y=None, transform_coords=False)`	Teleport a trace-replay vehicle by downstream lane distance, or by `x`/`y` coordinates that the simulator projects onto the target lane.
`enter_next_road(vehID, roadID="", private_veh=False)`	Force a co-simulation vehicle onto the next road, optionally overriding the planned route.
`reach_dest(vehID, private_veh=False)`	Mark a co-simulation vehicle as having reached its destination.
`control_vehicle(vehID, acc, private_veh=False)`	Override vehicle acceleration for the current tick.
`update_vehicle_sensor_type(vehID, sensorType, private_veh=False)`	Change the vehicle sensor type, for example `METSRClient.SENSOR_DSRC` or `METSRClient.SENSOR_CV2X`.
`update_vehicle_route(vehID, route, private_veh=False)`	Replace a vehicle route with an ordered list of road IDs.

Private trips, taxis, and transit

Method	Use
`generate_trip(vehID, origin=-1, destination=-1)`	Generate a private-vehicle trip between zones.
`generate_trip_between_roads(vehID, origin, destination)`	Generate a private-vehicle trip between road IDs.
`add_taxi_requests(zoneID, dest, num, max_waiting_time=None, maxWaitingTime=None)`	Create taxi request records and return request IDs for later dispatch or cancellation.
`add_taxi_requests_between_roads(orig, dest, num)`	Create taxi requests between road IDs.
`dispatch_taxi(vehID, reqID)`	Match available taxis to existing pending taxi requests.
`cancel_requests(reqID, zoneID=None)`	Cancel one or more taxi or bus requests. The client can infer origin zones from request records or `query_request` when possible.
`reposition_taxi(vehID, zoneID)`	Reposition idle or cruising taxis to destination zones.
`go_parking(vehID, zoneID=None, roadID=None)`	Send idle taxis to a target parking zone or road.
`add_bus_requests(zoneID, dest, routeName, num, max_waiting_time=None, maxWaitingTime=None)`	Create bus request records for later assignment.
`assign_request_to_bus(busID, reqID)`	Assign pending bus requests to active buses.
`add_bus_route(routeName, zone, road, paths=None)`	Add a bus route with stop zones, stop roads, and optional explicit paths between stops.
`add_bus_run(routeName, departTime)`	Schedule a bus departure on an existing route.
`insert_bus_stop(busID, routeName, zoneID, roadName, stopIndex)`	Insert a stop into an active bus route.
`remove_bus_stop(busID, routeName, stopIndex)`	Remove a stop from an active bus route.

Routing, charging, signals, and infrastructure

Method	Use
`update_road_weights(roadID, weight)`	Update routing weights such as travel time or energy cost.
`update_road_parking_capacity(roadID, parking_capacity=None, parkingCapacity=None, capacity=None)`	Update road parking capacity with any of the supported capacity argument names.
`update_charging_prices(stationID, stationType, price)`	Update charging prices by station and charger type.
`go_charging(vehID, veh_type, charger_type, cs_id=0)`	Send private EVs or public taxis to a selected or automatically chosen charging station.
`update_signal(signalID, targetPhase, phaseTime=None)`	Force a traffic signal to a target phase.
`update_signal_timing(signalID, greenTime, yellowTime, redTime)`	Update a signal’s fixed-time durations in ticks.
`set_signal_phase_plan(signalID, greenTime, yellowTime, redTime, startPhase, phaseOffset=None)`	Set a complete signal phase plan using seconds.
`set_signal_phase_plan_ticks(signalID, greenTicks, yellowTicks, redTicks, startPhase, tickOffset=None)`	Set a complete signal phase plan using simulation ticks.
`add_zone(x, y, capacity, zone_type, z=0.0, transform_coord=False)`	Add zones dynamically and attach them to nearby roads.
`remove_zone(zoneID)`	Remove zones when no active vehicles, requests, or routes still reference them.
`add_roads(centerline=None, upstream_road=None, downstream_road=None, orig_id=None, road_type=None, control_type=None, upstream_control_type=None, downstream_control_type=None, num_lanes=1, lane_width=None, transform_coord=False, roads=None, parking_capacity=None)`	Add generated roads or pass fully formed simulator road records.
`remove_road(roadID)`	Remove roads when doing so will not strand vehicles, requests, routes, or facilities.
`add_charging_station(x, y, num_l2, num_l3, num_bus, price_l2, price_l3, z=0.0, transform_coord=False)`	Add charging stations dynamically.
`remove_charging_station(stationID)`	Remove charging stations when no vehicle is queued, charging, or en route there.
`add_taxi(zoneID, num)`	Spawn electric taxis at selected zones.
`add_bus(routeName, num)`	Spawn electric buses on selected routes.

Visualization and trajectory helpers

METS-R SIM now writes compact binary trajectory chunks with a manifest.json by default when enable_trajectory_binary_write is enabled. The manifest describes the byte order, schemas, chunk list, road/zone/charging-station dictionaries, and sparse zone/charging-station frame groups. Older JSON trajectory files are still supported by the discovery helpers.

There are two visualization modes:

Live streaming: start_viz() opens a WebSocket stream for METS-R Vis. Open the METS-R Vis web app, click Stream, connect to the printed ws://... URL, and call render() whenever you want to push the current simulator state.
Offline playback: start_offline_viz() serves an existing trajectory output directory over HTTP so the METS-R Vis web app can replay it.

Method	Use
`start_viz(server_port=8765, host="127.0.0.1", ...)`	Start a live WebSocket stream for METS-R Vis. It can include public vehicles, private vehicles, links, zones, and charging stations.
`render(client_wait_timeout=5)`	Query the current METS-R tick and send one live frame to connected METS-R Vis clients.
`stop_viz_stream(join_timeout=2.0)`	Stop the live WebSocket stream.
`latest_trajectory_output_dir(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`	Find the newest trajectory output directory, preferring binary output.
`get_trajectory_manifest(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`	Read the manifest for the latest or selected binary trajectory output.
`get_trajectory_summary(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`	Return compact metadata about the latest trajectory output.
`start_offline_viz(trajectory_output_dir=None, server_port=8000, prefer_binary=True, wait_seconds=30)`	Serve a trajectory output directory for offline playback.
`stop_offline_viz()`	Stop the offline HTTP server.
`stop_viz()`	Stop both live and offline visualization servers if they are running.

CARLA co-simulation

tutorials/cosim_example.py demonstrates the current CARLA workflow. The script reads a CARLA run configuration, starts the auxiliary Docker services, launches METS-R SIM, connects a METSRClient, marks METS-R roads as co-simulation roads, and mirrors METS-R vehicles in CARLA.

Common options include:

python tutorials/cosim_example.py -r configs/run_cosim_CARLAT5.json -v

CARLA settings such as carla_dir, carla_host, carla_port, and carla_map live in the selected run configuration. The client-side controls listed above, especially set_cosim_road, query_entering_vehicle_queue, enter_road_from_queue, teleport_cosim_vehicle, enter_next_road, and reach_dest, are the low-level API calls behind the co-simulation loop.

Kafka streams

Kafka is optional and is used when an experiment needs explicit data-stream modeling or a cyber-physical data pipeline. Start the auxiliary stack from the HPC repository root:

cd docker
docker-compose up -d

The default V2X template uses localhost:29092 and topics such as link_tt, link_energy, bsm, v2x_tx_bsm, v2x_rx_bsm, v2x_link_metrics, and v2x_attack_events.

KafkaDataProcessor normalizes old and new payload schemas and exposes specialized handlers for BSM, transmitted BSM, received BSM, link metrics, attack events, link travel time, and link energy. KafkaDataSender is the matching lightweight producer for experiment scripts.

OMNeT++/Veins and Simu5G V2X

The V2X path is a separate TCP JSON-lines bridge between Python and OMNeT++. METS-R SIM remains the traffic simulator. VeinsClient sends vehicle mobility, BSM payloads, and optional attack events to the bridge once per METS-R tick. The bridge returns delivered BSMs, per-message link metrics, dropped messages, latency, attack events, and backend metadata.

Build and run the real Simu5G NR Uu backend from WSL:

export OMNETPP_HOME=~/src/omnetpp-6.1
source "$OMNETPP_HOME/setenv"

cd ~/src/METS-R_HPC/veins_bridge/omnetpp
bash ./check_sim5g_env.sh
bash ./build_sim5g.sh
bash ./run_sim5g_uu.sh

The Uu run script starts the bridge on TCP port 9099 by default after adding the generated Simu5G NED files and the local INET/Simu5G libraries to the runtime path.

For the V2X workflow documented here, use the sim5g_cellular_uu backend profile:

Config	Backend label	Meaning
`Sim5gCellularUu`	`sim5g_cellular_uu`	Real Simu5G/INET NR Uu backend. METS-R vehicle positions update UE mobility, BSM payloads are injected into Simu5G UE applications, and delivery is reported from the network receive path.

Open the Python notebook workflow after METS-R SIM and the bridge are listening:

jupyter lab tutorials/security_sim5g_v2x_examples.ipynb

The notebook connects METS-R vehicle states to the OMNeT++ bridge with VeinsClient.sync_tick() and lets the sim5g_cellular_uu backend report delivery, loss, latency, and attack-event records from the real 5G Uu path.

VeinsClient API summary:

Method	Use
`hello()`, `ping()`, `reset(**fields)`	Basic bridge handshake, health check, and reset calls.
`update_mobility(tick, vehicles)`	Send vehicle mobility records without sending BSMs.
`inject_bsm(tick, messages)`	Send BSM records to the bridge.
`inject_attacks(tick, attacks)`	Send attack-event records to the bridge.
`step_network(tick, duration_s=None)`	Advance only the network side for one tick.
`sync_tick(tick, vehicles, bsm_messages=None, attacks=None, duration_s=None)`	Preferred call for coupled experiments. It returns `received_bsms`, `link_metrics`, `attack_events`, backend labels, and the raw bridge response.

The stable bridge protocol accepts hello, ping, reset, and sync_tick. sync_tick receives vehicles, bsm_messages, and optional attacks. Message records should include stable matching fields such as message_id, sender_id, receiver_id, tx_time_s, radio_mode, payload_bytes, and BSM semantic fields such as x, y, speed_mps, and heading_deg.

Practical notes

Run commands from the METS-R_HPC repository root unless a bridge command explicitly changes into veins_bridge/omnetpp.
The Python clients expect simulator and bridge services on localhost by default. If the OMNeT++ bridge runs in WSL and Windows cannot reach 127.0.0.1, set the bridge host in the notebook or update the run config.
Restart notebook kernels after changing client source files. Existing Python objects keep the old class definitions in memory.
OMNeT++ Cmdenv progress lines count bridge polling and scheduled delivery events as well as user traffic, so high event counts do not necessarily mean high BSM volume.
Use the sim5g_cellular_uu profile for the documented real Simu5G NR Uu V2X workflow.