METS-R HPC module ================= `METS-R HPC `__ is the Python orchestration layer for interactive METS-R SIM experiments. It prepares run folders, launches one or more METS-R SIM containers, connects to each simulator over WebSocket, and provides helper clients for Kafka streams, CARLA co-simulation, METS-R Vis playback, and V2X network experiments through OMNeT++/Veins or Simu5G. The repository is now organized around direct, scriptable workflows rather than one monolithic experiment runner. Most users start METS-R SIM with a JSON run configuration, create a ``METSRClient``, and call query/control methods from a notebook or Python script. .. figure:: res/HPC_framework.png :alt: METS-R HPC framework METS-R HPC framework What the module provides ------------------------ - ``clients/METSRClient.py``: the main WebSocket client for METS-R SIM. It handles tick synchronization, query/control APIs, live METS-R Vis streaming, and offline trajectory-output discovery. - ``utils/util.py``: run-configuration parsing, per-run folder preparation, Docker launch helpers, and offline visualization-server utilities. - ``clients/KafkaDataSender.py`` and ``clients/KafkaDataProcessor.py``: producers and consumers for METS-R sensor, BSM, link travel-time, link-energy, and V2X metric streams. - ``clients/VeinsClient.py``: a TCP JSON-lines client for the OMNeT++ bridge. It sends METS-R vehicle mobility and BSM payloads with ``sync_tick`` and receives delivery, latency, loss, and attack-event records. - ``utils/carla_util.py``: helpers for CARLA vehicle spawning, coordinate conversion, route advancement, and co-simulation queue release. - ``utils/duckie_util.py``: helper functions for Duckietown message conversion and synchronization. - ``models/AnomalyDetector.py``: a lightweight example model for streaming data experiments. - ``tutorials/`` and ``configs/``: runnable examples and JSON run templates. - ``veins_bridge/omnetpp/``: the included OMNeT++ bridge project, including abstract V2X profiles and the Simu5G Uu backend scaffold. Quick start ----------- Install the Python dependencies from the ``METS-R_HPC`` repository root: .. code-block:: bash pip install -r requirements.txt Install Docker. On Docker Desktop for Windows or macOS, enable host networking under **Settings > Resources > Network**. METS-R SIM containers are launched with ``--net=host`` so the Python clients can connect to simulator services on ``localhost``. Run tutorials from the repository root so paths such as ``configs/...``, ``data/...``, and ``docker/...`` resolve correctly: .. code-block:: bash jupyter lab tutorials/basic_tutorial.ipynb A minimal script follows the same pattern used in the notebooks: .. code-block:: python from clients.METSRClient import METSRClient from utils.util import read_run_config, prepare_sim_dirs, run_simulation_in_docker config = read_run_config("configs/run_interactive_NYC.json") prepare_sim_dirs(config) run_simulation_in_docker(config) client = METSRClient( host=config.metsr_host, port=config.ports[0], sim_folder=config.sim_dirs[0], timeout=300, ) client.tick(10) print(client.query_tick()) client.terminate() Recommended entry points ------------------------ .. list-table:: :widths: 32 68 :header-rows: 1 * - File - Purpose * - ``tutorials/basic_tutorial.ipynb`` - Starts a simulator, connects a client, advances ticks, queries state, controls vehicles and services, and streams to METS-R Vis. * - ``tutorials/advanced_commands.ipynb`` - Longer command examples for co-simulation, dynamic infrastructure, ride-hailing control, Kafka, save/load, and cleanup. * - ``tutorials/security_examples.ipynb`` - Local simulator cyber-range examples using direct ``METSRClient`` calls, Kafka where needed, and explicit before/after diagnostics. * - ``tutorials/security_sim5g_v2x_examples.ipynb`` - Interactive METS-R plus Simu5G V2X attack workflow using ``VeinsClient.sync_tick()``. * - ``tutorials/cosim_example.py`` - CARLA/METS-R co-simulation and visualization script. Run configurations ------------------ Run configurations are JSON files under ``configs/``. The common fields include the Java path and options, METS-R SIM directory, host, random seeds, simulation step size, data files, output settings, and optional Kafka/V2X settings. Important conventions: - ``num_simulations`` controls how many simulator instances are prepared. - ``prepare_sim_dirs(config)`` creates timestamped per-seed output folders and populates ``config.sim_dirs`` and ``config.ports``. - ``run_simulation_in_docker(config)`` launches the simulator image from each prepared folder. - ``enable_trajectory_binary_write`` enables the current compact trajectory output format used by METS-R Vis. - ``json_output`` can still be enabled for legacy JSON trajectory files. - V2X configs such as ``configs/run_v2x_veins_Template.json`` add ``veins_host``, ``veins_port``, timeout settings, backend labels, and Kafka topics for V2X telemetry. .. _Interactive APIs: Interactive client APIs ----------------------- ``METSRClient`` talks to one METS-R SIM instance over WebSocket. Unless noted otherwise, methods accept either scalar inputs or lists. When lists are passed, records are paired by index and the simulator response contains one ``DATA`` entry per requested record. Most control calls return the raw simulator response with top-level ``CODE`` and per-record ``STATUS`` fields. Road IDs are SUMO/original road IDs. ``transform_coords=True`` asks METS-R SIM to convert between the simulator coordinate reference system and WGS84 longitude/latitude when the server-side endpoint supports it. Ticks and lifecycle ^^^^^^^^^^^^^^^^^^^ Recent METS-R SIM builds include authoritative tick values in ``STEP``, ``CTRL_reset``, and ``CTRL_load`` responses. The client updates ``current_tick`` from those responses and can query the server tick when a step reply is slow or stale. .. list-table:: :widths: 38 62 :header-rows: 1 * - Method - Use * - ``query_tick()`` - Return the current simulator tick reported by METS-R SIM. * - ``query_tick_status()`` - Return server-side stepping status, including active-road stepping fields when enabled. * - ``tick(step_num=1, wait_forever=False, retry_interval=None, max_wait_seconds=None, poll_timeout=5, max_stalled_seconds=None)`` - Advance the simulator and wait until the requested tick is reached. ``wait_forever=True`` still uses bounded polling so stalled simulators can be detected. * - ``reset()`` - Reset the simulation to its initial state and resynchronize the client tick. * - ``save(filename)`` - Save a full simulator snapshot to a zip archive. * - ``load(filename, reload_network=True)`` - Restore a saved snapshot. Set ``reload_network=False`` when the network is unchanged and a faster state reload is sufficient. * - ``terminate()`` - Ask the simulator process to end. * - ``close()`` - Close only the client WebSocket connection. State query APIs ^^^^^^^^^^^^^^^^ .. list-table:: :widths: 40 60 :header-rows: 1 * - Method - Use * - ``query_vehicle(id=None, private_veh=False, transform_coords=False)`` - Return public/private vehicle ID lists, or full kinematic state for selected vehicles. * - ``query_on_road_vehicles(roadID=None)`` - Return vehicles currently on selected road IDs, or grouped road records when no road is specified. * - ``query_active_roads()`` - Return roads currently active in the simulator stepping set. * - ``query_taxi(id=None)`` - Return electric taxi IDs or taxi state, including trip, passenger, battery, and remaining-distance fields. * - ``query_available_taxis(zoneID=None)`` - Return taxis currently in the explicit available-dispatch pool, optionally filtered by zone. * - ``query_almost_finished_taxis(distance_threshold_miles=None, distance_threshold_meters=None, zoneID=None)`` - Return taxis whose active trip is within a distance threshold of completion. * - ``query_bus(id=None)`` - Return electric bus IDs or bus state, route, battery, stop, and passenger fields. * - ``query_road(id=None)`` - Return road IDs or static and real-time road attributes. * - ``query_entering_vehicle_queue(roadID=None)`` - Return vehicles waiting to enter co-simulation roads. * - ``query_cosim_entering_vehicle_queue()`` - Return entering queues for all roads currently marked for co-simulation. * - ``query_centerline(id, lane_index=-1, transform_coords=False)`` - Return road-level or lane-level centerline geometry. * - ``query_zone(id=None)`` - Return zone IDs or demand, stock, location, and service counters. * - ``query_pending_requests(zoneID=None)`` - Return pending taxi and bus requests across all zones or one zone. * - ``query_request(reqID)`` - Return request status for one or more request IDs. * - ``query_pickup_taxi_info(reqID=None)`` - Return taxi pickup assignments by request ID or all current pickup assignments. * - ``query_occupied_taxi_info(reqID=None)`` - Return occupied taxi trip assignments by request ID or all current occupied assignments. * - ``query_signal(id=None)`` - Return signal IDs or current phase state and next-update timing. * - ``query_signal_group(id=None)`` - Map signal group or junction IDs to individual METS-R signal IDs. * - ``query_signal_between_roads(upstream_road, downstream_road)`` - Return the signal controlling a road-to-road connection. * - ``query_chargingStation(id=None)`` - Return charging station IDs or status, capacity, and prices. * - ``query_coSimVehicle()`` - Return vehicles currently on co-simulation roads. * - ``query_route(orig_x, orig_y, dest_x, dest_y, transform_coords=False)`` - Return the shortest route between two coordinate locations. * - ``query_k_routes(orig_x, orig_y, dest_x, dest_y, k, transform_coords=False)`` - Return up to ``k`` coordinate-based route alternatives. * - ``query_route_between_roads(orig_road, dest_road)`` - Return the shortest route between two road IDs. * - ``query_k_routes_between_roads(orig_road, dest_road, k)`` - Return up to ``k`` road-to-road alternatives. * - ``query_road_weights(roadID=None)`` - Return current routing weights for selected roads or all roads. * - ``query_bus_route(routeID=None)`` - Return bus route names or stop-road sequences. * - ``query_route_bus(routeID=None)`` - Return buses currently assigned to each route. * - ``query_routing_graph()`` - Build a local ``networkx.DiGraph`` from road query results. Co-simulation and vehicle control ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 40 60 :header-rows: 1 * - Method - Use * - ``set_cosim_road(roadID)`` - Mark one or more roads as externally controlled co-simulation roads. * - ``release_cosim_road(roadID)`` - Return one or more co-simulation roads to normal METS-R control. * - ``enter_road_from_queue(vehID=None, roadID=None, private_veh=None, internal_vehicle_id=None, requests=None)`` - Release queued vehicles onto co-simulation roads after inspecting ``query_entering_vehicle_queue``. * - ``teleport_cosim_vehicle(vehID, x, y, bearing, speed=0, z=0.0, private_veh=False, transform_coords=False)`` - Teleport a co-simulation vehicle to absolute coordinates with heading and speed. * - ``teleport_trace_replay_vehicle(vehID, roadID, laneID, dist=None, private_veh=False, x=None, y=None, transform_coords=False)`` - Teleport a trace-replay vehicle by downstream lane distance, or by ``x``/``y`` coordinates that the simulator projects onto the target lane. * - ``enter_next_road(vehID, roadID="", private_veh=False)`` - Force a co-simulation vehicle onto the next road, optionally overriding the planned route. * - ``reach_dest(vehID, private_veh=False)`` - Mark a co-simulation vehicle as having reached its destination. * - ``control_vehicle(vehID, acc, private_veh=False)`` - Override vehicle acceleration for the current tick. * - ``update_vehicle_sensor_type(vehID, sensorType, private_veh=False)`` - Change the vehicle sensor type, for example ``METSRClient.SENSOR_DSRC`` or ``METSRClient.SENSOR_CV2X``. * - ``update_vehicle_route(vehID, route, private_veh=False)`` - Replace a vehicle route with an ordered list of road IDs. Private trips, taxis, and transit ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 40 60 :header-rows: 1 * - Method - Use * - ``generate_trip(vehID, origin=-1, destination=-1)`` - Generate a private-vehicle trip between zones. * - ``generate_trip_between_roads(vehID, origin, destination)`` - Generate a private-vehicle trip between road IDs. * - ``add_taxi_requests(zoneID, dest, num, max_waiting_time=None, maxWaitingTime=None)`` - Create taxi request records and return request IDs for later dispatch or cancellation. * - ``add_taxi_requests_between_roads(orig, dest, num)`` - Create taxi requests between road IDs. * - ``dispatch_taxi(vehID, reqID)`` - Match available taxis to existing pending taxi requests. * - ``cancel_requests(reqID, zoneID=None)`` - Cancel one or more taxi or bus requests. The client can infer origin zones from request records or ``query_request`` when possible. * - ``reposition_taxi(vehID, zoneID)`` - Reposition idle or cruising taxis to destination zones. * - ``go_parking(vehID, zoneID=None, roadID=None)`` - Send idle taxis to a target parking zone or road. * - ``add_bus_requests(zoneID, dest, routeName, num, max_waiting_time=None, maxWaitingTime=None)`` - Create bus request records for later assignment. * - ``assign_request_to_bus(busID, reqID)`` - Assign pending bus requests to active buses. * - ``add_bus_route(routeName, zone, road, paths=None)`` - Add a bus route with stop zones, stop roads, and optional explicit paths between stops. * - ``add_bus_run(routeName, departTime)`` - Schedule a bus departure on an existing route. * - ``insert_bus_stop(busID, routeName, zoneID, roadName, stopIndex)`` - Insert a stop into an active bus route. * - ``remove_bus_stop(busID, routeName, stopIndex)`` - Remove a stop from an active bus route. Routing, charging, signals, and infrastructure ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 40 60 :header-rows: 1 * - Method - Use * - ``update_road_weights(roadID, weight)`` - Update routing weights such as travel time or energy cost. * - ``update_road_parking_capacity(roadID, parking_capacity=None, parkingCapacity=None, capacity=None)`` - Update road parking capacity with any of the supported capacity argument names. * - ``update_charging_prices(stationID, stationType, price)`` - Update charging prices by station and charger type. * - ``go_charging(vehID, veh_type, charger_type, cs_id=0)`` - Send private EVs or public taxis to a selected or automatically chosen charging station. * - ``update_signal(signalID, targetPhase, phaseTime=None)`` - Force a traffic signal to a target phase. * - ``update_signal_timing(signalID, greenTime, yellowTime, redTime)`` - Update a signal's fixed-time durations in ticks. * - ``set_signal_phase_plan(signalID, greenTime, yellowTime, redTime, startPhase, phaseOffset=None)`` - Set a complete signal phase plan using seconds. * - ``set_signal_phase_plan_ticks(signalID, greenTicks, yellowTicks, redTicks, startPhase, tickOffset=None)`` - Set a complete signal phase plan using simulation ticks. * - ``add_zone(x, y, capacity, zone_type, z=0.0, transform_coord=False)`` - Add zones dynamically and attach them to nearby roads. * - ``remove_zone(zoneID)`` - Remove zones when no active vehicles, requests, or routes still reference them. * - ``add_roads(centerline=None, upstream_road=None, downstream_road=None, orig_id=None, road_type=None, control_type=None, upstream_control_type=None, downstream_control_type=None, num_lanes=1, lane_width=None, transform_coord=False, roads=None, parking_capacity=None)`` - Add generated roads or pass fully formed simulator road records. * - ``remove_road(roadID)`` - Remove roads when doing so will not strand vehicles, requests, routes, or facilities. * - ``add_charging_station(x, y, num_l2, num_l3, num_bus, price_l2, price_l3, z=0.0, transform_coord=False)`` - Add charging stations dynamically. * - ``remove_charging_station(stationID)`` - Remove charging stations when no vehicle is queued, charging, or en route there. * - ``add_taxi(zoneID, num)`` - Spawn electric taxis at selected zones. * - ``add_bus(routeName, num)`` - Spawn electric buses on selected routes. Visualization and trajectory helpers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ METS-R SIM now writes compact binary trajectory chunks with a ``manifest.json`` by default when ``enable_trajectory_binary_write`` is enabled. The manifest describes the byte order, schemas, chunk list, road/zone/charging-station dictionaries, and sparse zone/charging-station frame groups. Older JSON trajectory files are still supported by the discovery helpers. There are two visualization modes: - Live streaming: ``start_viz()`` opens a WebSocket stream for METS-R Vis. Open the METS-R Vis web app, click **Stream**, connect to the printed ``ws://...`` URL, and call ``render()`` whenever you want to push the current simulator state. - Offline playback: ``start_offline_viz()`` serves an existing trajectory output directory over HTTP so the METS-R Vis web app can replay it. .. list-table:: :widths: 42 58 :header-rows: 1 * - Method - Use * - ``start_viz(server_port=8765, host="127.0.0.1", ...)`` - Start a live WebSocket stream for METS-R Vis. It can include public vehicles, private vehicles, links, zones, and charging stations. * - ``render(client_wait_timeout=5)`` - Query the current METS-R tick and send one live frame to connected METS-R Vis clients. * - ``stop_viz_stream(join_timeout=2.0)`` - Stop the live WebSocket stream. * - ``latest_trajectory_output_dir(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`` - Find the newest trajectory output directory, preferring binary output. * - ``get_trajectory_manifest(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`` - Read the manifest for the latest or selected binary trajectory output. * - ``get_trajectory_summary(trajectory_output_dir=None, prefer_binary=True, wait_seconds=0)`` - Return compact metadata about the latest trajectory output. * - ``start_offline_viz(trajectory_output_dir=None, server_port=8000, prefer_binary=True, wait_seconds=30)`` - Serve a trajectory output directory for offline playback. * - ``stop_offline_viz()`` - Stop the offline HTTP server. * - ``stop_viz()`` - Stop both live and offline visualization servers if they are running. CARLA co-simulation ------------------- ``tutorials/cosim_example.py`` demonstrates the current CARLA workflow. The script reads a CARLA run configuration, starts the auxiliary Docker services, launches METS-R SIM, connects a ``METSRClient``, marks METS-R roads as co-simulation roads, and mirrors METS-R vehicles in CARLA. Common options include: .. code-block:: bash python tutorials/cosim_example.py -r configs/run_cosim_CARLAT5.json -v CARLA settings such as ``carla_dir``, ``carla_host``, ``carla_port``, and ``carla_map`` live in the selected run configuration. The client-side controls listed above, especially ``set_cosim_road``, ``query_entering_vehicle_queue``, ``enter_road_from_queue``, ``teleport_cosim_vehicle``, ``enter_next_road``, and ``reach_dest``, are the low-level API calls behind the co-simulation loop. Kafka streams ------------- Kafka is optional and is used when an experiment needs explicit data-stream modeling or a cyber-physical data pipeline. Start the auxiliary stack from the HPC repository root: .. code-block:: bash cd docker docker-compose up -d The default V2X template uses ``localhost:29092`` and topics such as ``link_tt``, ``link_energy``, ``bsm``, ``v2x_tx_bsm``, ``v2x_rx_bsm``, ``v2x_link_metrics``, and ``v2x_attack_events``. ``KafkaDataProcessor`` normalizes old and new payload schemas and exposes specialized handlers for BSM, transmitted BSM, received BSM, link metrics, attack events, link travel time, and link energy. ``KafkaDataSender`` is the matching lightweight producer for experiment scripts. OMNeT++/Veins and Simu5G V2X ---------------------------- The V2X path is a separate TCP JSON-lines bridge between Python and OMNeT++. METS-R SIM remains the traffic simulator. ``VeinsClient`` sends vehicle mobility, BSM payloads, and optional attack events to the bridge once per METS-R tick. The bridge returns delivered BSMs, per-message link metrics, dropped messages, latency, attack events, and backend metadata. Build and run the real Simu5G NR Uu backend from WSL: .. code-block:: bash export OMNETPP_HOME=~/src/omnetpp-6.1 source "$OMNETPP_HOME/setenv" cd ~/src/METS-R_HPC/veins_bridge/omnetpp bash ./check_sim5g_env.sh bash ./build_sim5g.sh bash ./run_sim5g_uu.sh The Uu run script starts the bridge on TCP port ``9099`` by default after adding the generated Simu5G NED files and the local INET/Simu5G libraries to the runtime path. For the V2X workflow documented here, use the ``sim5g_cellular_uu`` backend profile: .. list-table:: :widths: 28 28 44 :header-rows: 1 * - Config - Backend label - Meaning * - ``Sim5gCellularUu`` - ``sim5g_cellular_uu`` - Real Simu5G/INET NR Uu backend. METS-R vehicle positions update UE mobility, BSM payloads are injected into Simu5G UE applications, and delivery is reported from the network receive path. Open the Python notebook workflow after METS-R SIM and the bridge are listening: .. code-block:: bash jupyter lab tutorials/security_sim5g_v2x_examples.ipynb The notebook connects METS-R vehicle states to the OMNeT++ bridge with ``VeinsClient.sync_tick()`` and lets the ``sim5g_cellular_uu`` backend report delivery, loss, latency, and attack-event records from the real 5G Uu path. ``VeinsClient`` API summary: .. list-table:: :widths: 34 66 :header-rows: 1 * - Method - Use * - ``hello()``, ``ping()``, ``reset(**fields)`` - Basic bridge handshake, health check, and reset calls. * - ``update_mobility(tick, vehicles)`` - Send vehicle mobility records without sending BSMs. * - ``inject_bsm(tick, messages)`` - Send BSM records to the bridge. * - ``inject_attacks(tick, attacks)`` - Send attack-event records to the bridge. * - ``step_network(tick, duration_s=None)`` - Advance only the network side for one tick. * - ``sync_tick(tick, vehicles, bsm_messages=None, attacks=None, duration_s=None)`` - Preferred call for coupled experiments. It returns ``received_bsms``, ``link_metrics``, ``attack_events``, backend labels, and the raw bridge response. The stable bridge protocol accepts ``hello``, ``ping``, ``reset``, and ``sync_tick``. ``sync_tick`` receives ``vehicles``, ``bsm_messages``, and optional ``attacks``. Message records should include stable matching fields such as ``message_id``, ``sender_id``, ``receiver_id``, ``tx_time_s``, ``radio_mode``, ``payload_bytes``, and BSM semantic fields such as ``x``, ``y``, ``speed_mps``, and ``heading_deg``. Practical notes --------------- - Run commands from the ``METS-R_HPC`` repository root unless a bridge command explicitly changes into ``veins_bridge/omnetpp``. - The Python clients expect simulator and bridge services on ``localhost`` by default. If the OMNeT++ bridge runs in WSL and Windows cannot reach `127.0.0.1`, set the bridge host in the notebook or update the run config. - Restart notebook kernels after changing client source files. Existing Python objects keep the old class definitions in memory. - OMNeT++ ``Cmdenv`` progress lines count bridge polling and scheduled delivery events as well as user traffic, so high event counts do not necessarily mean high BSM volume. - Use the ``sim5g_cellular_uu`` profile for the documented real Simu5G NR Uu V2X workflow.