Network comparison demoĀ¶
Note - this script can also be opened in interactive Python if you wanted to
play around. On the GitHub it is in docs/notebooks
. To run this on your
local machine, you will need to install the optional dependencies for doc:
pip install swmmanywhere[doc]
IntroductionĀ¶
This script demonstrates how to use SWMManywhere when you have a real network. It shows how to tell SWMManywhere where the necessary data is, and how, when this is provided, a suite of metrics are calculated to compare the real network with the synthesised network.
Since this is a notebook, we will define config
as a dictionary rather than a yaml file, but the same principles apply.
Initial setupĀ¶
Here we will run one of the Bellinge sub-networks which is provided in the test data. We will keep everything in a temporary directory.
InĀ [1]:
Copied!
# Imports
from __future__ import annotations
import tempfile
from pathlib import Path
from swmmanywhere.defs import copy_test_data
from swmmanywhere.logging import set_verbose
from swmmanywhere.swmmanywhere import swmmanywhere
from swmmanywhere.utilities import plot_map
# Create temporary directory
temp_dir = tempfile.TemporaryDirectory(dir=".")
base_dir = Path(temp_dir.name)
# Make a folder for real data
real_dir = base_dir / "real"
real_dir.mkdir(exist_ok=True)
# Copy test data into the real data folder
copy_test_data(real_dir)
# Imports
from __future__ import annotations
import tempfile
from pathlib import Path
from swmmanywhere.defs import copy_test_data
from swmmanywhere.logging import set_verbose
from swmmanywhere.swmmanywhere import swmmanywhere
from swmmanywhere.utilities import plot_map
# Create temporary directory
temp_dir = tempfile.TemporaryDirectory(dir=".")
base_dir = Path(temp_dir.name)
# Make a folder for real data
real_dir = base_dir / "real"
real_dir.mkdir(exist_ok=True)
# Copy test data into the real data folder
copy_test_data(real_dir)
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/networkx/readwrite/json_graph/node_link.py:287: FutureWarning: The default value will be changed to `edges="edges" in NetworkX 3.6. To make this warning go away, explicitly set the edges kwarg, e.g.: nx.node_link_graph(data, edges="links") to preserve current behavior, or nx.node_link_graph(data, edges="edges") for forward compatibility. warnings.warn(
Create config fileĀ¶
Below, we update our config file to use new coordinates, and provide the real data. As set out in the schema, a variety of data entries can be provided to describe the real network, all CRS of shapefiles must be in the UTM CRS for the project:
graph(essential) - a graph file in JSON format created fromnx.node_link_datasubcatchments(essential) - a GeoJSON file with subcatchment outlines, with headings forid, andimpervious_area(only needed if calculating the metric,bias_flood_depth).inp- an inp file to run the model with, matching the appropriate graph and subcatchments.results- a results file to compare the model with.
At least one of results or inp must be provided. If inp is provided then the
model will be run and the results compared to the real data, otherwise results will
be loaded directly.
Note that the need to separately provide a graph and subcatchments will be
removed following fixing of this.
InĀ [2]:
Copied!
# Define config
config = {
"base_dir": base_dir,
"project": "bellinge_small",
"bbox": [10.309, 55.333, 10.317, 55.339],
"run_settings": {"duration": 3600},
"real": {
"graph": real_dir / "bellinge_small_graph.json",
"inp": real_dir / "bellinge_small.inp",
"subcatchments": real_dir / "bellinge_small_subcatchments.geojson",
},
"parameter_overrides": {
"topology_derivation": {
"allowable_networks": ["drive"],
"omit_edges": ["bridge"],
}
},
}
# Define config
config = {
"base_dir": base_dir,
"project": "bellinge_small",
"bbox": [10.309, 55.333, 10.317, 55.339],
"run_settings": {"duration": 3600},
"real": {
"graph": real_dir / "bellinge_small_graph.json",
"inp": real_dir / "bellinge_small.inp",
"subcatchments": real_dir / "bellinge_small_subcatchments.geojson",
},
"parameter_overrides": {
"topology_derivation": {
"allowable_networks": ["drive"],
"omit_edges": ["bridge"],
}
},
}
Run SWMManywhereĀ¶
We make the swmmanywhere call as normal, but can observe that there is an additional
model run (scroll towards the bottom) where the real model inp is being run and the
metrics are shown to have been calculated in the log.
InĀ [3]:
Copied!
## Run SWMManywhere
set_verbose(True)
outputs = swmmanywhere(config)
## Run SWMManywhere
set_verbose(True)
outputs = swmmanywhere(config)
2025/05/22 10:49:50 | Creating project structure.
2025/05/22 10:49:50 | Project structure created at tmpyi9zszy5
2025/05/22 10:49:50 | Project name: bellinge_small
2025/05/22 10:49:50 | Bounding box: [10.309, 55.333, 10.317, 55.339],
number: 1
2025/05/22 10:49:50 | Model number: 1
2025/05/22 10:49:50 | Loading and setting parameters.
2025/05/22 10:49:50 | Setting topology_derivation allowable_networks to ['drive']
2025/05/22 10:49:50 | Setting topology_derivation omit_edges to ['bridge']
2025/05/22 10:49:50 | Allowable networks have been changed, removing old street graph.
2025/05/22 10:49:50 | Running downloads.
2025/05/22 10:49:50 | downloading elevation to tmpyi9zszy5/bellinge_small/bbox_1/download/elevation.tif
2025/05/22 10:49:55 | downloading buildings to tmpyi9zszy5/bellinge_small/bbox_1/download/building.geoparquet
2025/05/22 10:50:06 | downloading network to tmpyi9zszy5/bellinge_small/bbox_1/download/street.parquet
2025/05/22 10:50:08 | downloading river network to tmpyi9zszy5/bellinge_small/bbox_1/download/river.parquet
2025/05/22 10:50:10 | No water network found within the bounding box.
2025/05/22 10:50:10 | Iterating graphs.
2025/05/22 10:50:10 | Iterating graph functions.
2025/05/22 10:50:10 | graphfcn: assign_id completed.
2025/05/22 10:50:10 | graphfcn: fix_geometries completed.
2025/05/22 10:50:10 | graphfcn: remove_non_pipe_allowable_links completed.
2025/05/22 10:50:10 | graphfcn: calculate_streetcover completed.
2025/05/22 10:50:10 | graphfcn: remove_parallel_edges completed.
2025/05/22 10:50:10 | graphfcn: to_undirected completed.
2025/05/22 10:50:10 | graphfcn: split_long_edges completed.
2025/05/22 10:50:10 | graphfcn: merge_street_nodes completed.
2025/05/22 10:50:10 | graphfcn: assign_id completed.
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/networkx/readwrite/json_graph/node_link.py:287: FutureWarning: The default value will be changed to `edges="edges" in NetworkX 3.6. To make this warning go away, explicitly set the edges kwarg, e.g.: nx.node_link_graph(data, edges="links") to preserve current behavior, or nx.node_link_graph(data, edges="edges") for forward compatibility. warnings.warn(
Downloading WhiteboxTools from https://www.whiteboxgeo.com/WBT_Linux/WhiteboxTools_linux_amd64.zip
Extracted WhiteboxTools to tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn/WBT
Using existing WhiteboxTools executable: tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn/WBT/whitebox_tools
Starting WhiteboxTools session with source directory: /home/runner/work/SWMManywhere/SWMManywhere/tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn
Running WhiteboxTools version: 2.4.0
Command: tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn/WBT/whitebox_tools --run=BreachDepressions --wd=tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn --compress_rasters=false --max_procs=1 -i=dem.tif --fillpits -o=dem_corr.tif
WhiteboxTools output:
******************************** * Welcome to BreachDepressions * * Powered by WhiteboxTools * * www.whiteboxgeo.com * ******************************** Reading data... Progress: 0% Progress: 1% Progress: 2% Progress: 3% Progress: 4% Progress: 5% Progress: 6% Progress: 7% Progress: 8% Progress: 9% Progress: 10% Progress: 11% Progress: 12% Progress: 13% Progress: 14% Progress: 15% Progress: 16% Progress: 17% Progress: 18% Progress: 19% Progress: 20% Progress: 21% Progress: 22% Progress: 23% Progress: 24% Progress: 25% Progress: 26% Progress: 27% Progress: 28% Progress: 29% Progress: 30% Progress: 31% Progress: 32% Progress: 33% Progress: 34% Progress: 35% Progress: 36% Progress: 37% Progress: 38% Progress: 39% Progress: 40% Progress: 41% Progress: 42% Progress: 43% Progress: 44% Progress: 45% Progress: 46% Progress: 47% Progress: 48% Progress: 49% Progress: 50% Progress: 51% Progress: 52% Progress: 53% Progress: 54% Progress: 55% Progress: 56% Progress: 57% Progress: 58% Progress: 59% Progress: 60% Progress: 61% Progress: 62% Progress: 63% Progress: 64% Progress: 65% Progress: 66% Progress: 67% Progress: 68% Progress: 69% Progress: 70% Progress: 71% Progress: 72% Progress: 73% Progress: 74% Progress: 75% Progress: 76% Progress: 77% Progress: 78% Progress: 79% Progress: 80% Progress: 81% Progress: 82% Progress: 83% Progress: 84% Progress: 85% Progress: 86% Progress: 87% Progress: 88% Progress: 89% Progress: 90% Progress: 91% Progress: 92% Progress: 93% Progress: 94% Progress: 95% Progress: 96% Progress: 97% Progress: 98% Progress: 99% Progress: 100% Progress: 101% Progress: 102% Progress: 103% Progress: 104% Progress: 105% Progress: 106% Progress: 107% Progress: 108% Progress: 109% Progress: 110% Progress: 111% Progress: 112% Progress: 113% Progress: 114% Saving data... Output file written Elapsed Time (excluding I/O): 0.0s
Running WhiteboxTools version: 2.4.0
Command: tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn/WBT/whitebox_tools --run=D8Pointer --wd=tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn --compress_rasters=false --max_procs=1 -i=dem_corr.tif -o=fdir.tif
WhiteboxTools output:
**************************** * Welcome to D8Pointer * * Powered by WhiteboxTools * * www.whiteboxgeo.com * **************************** Reading data... Progress: 0% Progress: 3% Progress: 6% Progress: 10% Progress: 13% Progress: 17% Progress: 20% Progress: 24% Progress: 27% Progress: 31% Progress: 34% Progress: 37% Progress: 41% Progress: 44% Progress: 48% Progress: 51% Progress: 55% Progress: 58% Progress: 62% Progress: 65% Progress: 68% Progress: 72% Progress: 75% Progress: 79% Progress: 82% Progress: 86% Progress: 89% Progress: 93% Progress: 96% Progress: 100% Saving data... Output file written Elapsed Time (excluding I/O): 0.0s
Completed WhiteboxTools session with source directory: /home/runner/work/SWMManywhere/SWMManywhere/tmpyi9zszy5/bellinge_small/bbox_1/download/tmp7ontnwdn
2025/05/22 10:50:14 | graphfcn: clip_to_catchments completed.
Extracted WhiteboxTools to tmpigmo_wbe/tmpwbq4geiy/WBT
Using existing WhiteboxTools executable: tmpigmo_wbe/tmpwbq4geiy/WBT/whitebox_tools
Starting WhiteboxTools session with source directory: /home/runner/work/SWMManywhere/SWMManywhere/tmpigmo_wbe/tmpwbq4geiy
Running WhiteboxTools version: 2.4.0
Command: tmpigmo_wbe/tmpwbq4geiy/WBT/whitebox_tools --run=BreachDepressions --wd=tmpigmo_wbe/tmpwbq4geiy --compress_rasters=false --max_procs=1 -i=dem.tif --fillpits -o=dem_corr.tif
WhiteboxTools output:
******************************** * Welcome to BreachDepressions * * Powered by WhiteboxTools * * www.whiteboxgeo.com * ******************************** Reading data... Progress: 0% Progress: 1% Progress: 2% Progress: 3% Progress: 4% Progress: 5% Progress: 6% Progress: 7% Progress: 8% Progress: 9% Progress: 10% Progress: 11% Progress: 12% Progress: 13% Progress: 14% Progress: 15% Progress: 16% Progress: 17% Progress: 18% Progress: 19% Progress: 20% Progress: 21% Progress: 22% Progress: 23% Progress: 24% Progress: 25% Progress: 26% Progress: 27% Progress: 28% Progress: 29% Progress: 30% Progress: 31% Progress: 32% Progress: 33% Progress: 34% Progress: 35% Progress: 36% Progress: 37% Progress: 38% Progress: 39% Progress: 40% Progress: 41% Progress: 42% Progress: 43% Progress: 44% Progress: 45% Progress: 46% Progress: 47% Progress: 48% Progress: 49% Progress: 50% Progress: 51% Progress: 52% Progress: 53% Progress: 54% Progress: 55% Progress: 56% Progress: 57% Progress: 58% Progress: 59% Progress: 60% Progress: 61% Progress: 62% Progress: 63% Progress: 64% Progress: 65% Progress: 66% Progress: 67% Progress: 68% Progress: 69% Progress: 70% Progress: 71% Progress: 72% Progress: 73% Progress: 74% Progress: 75% Progress: 76% Progress: 77% Progress: 78% Progress: 79% Progress: 80% Progress: 81% Progress: 82% Progress: 83% Progress: 84% Progress: 85% Progress: 86% Progress: 87% Progress: 88% Progress: 89% Progress: 90% Progress: 91% Progress: 92% Progress: 93% Progress: 94% Progress: 95% Progress: 96% Progress: 97% Progress: 98% Progress: 99% Progress: 100% Progress: 101% Progress: 102% Progress: 103% Progress: 104% Progress: 105% Progress: 106% Progress: 107% Progress: 108% Progress: 109% Progress: 110% Progress: 111% Progress: 112% Progress: 113% Progress: 114% Saving data... Output file written Elapsed Time (excluding I/O): 0.0s
Running WhiteboxTools version: 2.4.0
Command: tmpigmo_wbe/tmpwbq4geiy/WBT/whitebox_tools --run=D8Pointer --wd=tmpigmo_wbe/tmpwbq4geiy --compress_rasters=false --max_procs=1 -i=dem_corr.tif -o=fdir.tif
WhiteboxTools output:
**************************** * Welcome to D8Pointer * * Powered by WhiteboxTools * * www.whiteboxgeo.com * **************************** Reading data... Progress: 0% Progress: 3% Progress: 6% Progress: 10% Progress: 13% Progress: 17% Progress: 20% Progress: 24% Progress: 27% Progress: 31% Progress: 34% Progress: 37% Progress: 41% Progress: 44% Progress: 48% Progress: 51% Progress: 55% Progress: 58% Progress: 62% Progress: 65% Progress: 68% Progress: 72% Progress: 75% Progress: 79% Progress: 82% Progress: 86% Progress: 89% Progress: 93% Progress: 96% Progress: 100% Saving data... Output file written Elapsed Time (excluding I/O): 0.0s
Completed WhiteboxTools session with source directory: /home/runner/work/SWMManywhere/SWMManywhere/tmpigmo_wbe/tmpwbq4geiy
2025/05/22 10:50:15 | graphfcn: calculate_contributing_area completed.
2025/05/22 10:50:15 | graphfcn: set_elevation completed.
2025/05/22 10:50:15 | graphfcn: double_directed completed.
2025/05/22 10:50:15 | graphfcn: fix_geometries completed.
2025/05/22 10:50:15 | graphfcn: set_surface_slope completed.
2025/05/22 10:50:15 | graphfcn: set_chahinian_slope completed.
2025/05/22 10:50:15 | graphfcn: set_chahinian_angle completed.
2025/05/22 10:50:15 | graphfcn: calculate_weights completed.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
110, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
83, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
116, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
117, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
152, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
166, using this node as outfall.
2025/05/22 10:50:15 | No outfalls found for subgraph containing
200, using this node as outfall.
2025/05/22 10:50:15 | graphfcn: identify_outfalls completed.
2025/05/22 10:50:15 | Total graph weight 59.229756405972964.
2025/05/22 10:50:15 | graphfcn: derive_topology completed.
0%| | 0/148 [00:00<?, ?it/s]
23%|āāā | 34/148 [00:00<00:00, 330.53it/s]
49%|āāāāā | 73/148 [00:00<00:00, 360.34it/s]
74%|āāāāāāāā | 110/148 [00:00<00:00, 333.64it/s]
97%|āāāāāāāāāā| 144/148 [00:00<00:00, 283.90it/s]
100%|āāāāāāāāāā| 148/148 [00:00<00:00, 298.54it/s]
2025/05/22 10:50:16 | graphfcn: pipe_by_pipe completed.
2025/05/22 10:50:16 | graphfcn: fix_geometries completed.
2025/05/22 10:50:16 | graphfcn: assign_id completed.
2025/05/22 10:50:16 | Saving final graph and writing inp file.
Skipping field osmid: unsupported OGR type: 1
Skipping field lanes: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
2025/05/22 10:50:16 | Running the synthetic model.
2025/05/22 10:50:16 | tmpyi9zszy5/bellinge_small/bbox_1/model_1/model_1.inp initialised in pyswmm
2025/05/22 10:50:16 | Starting simulation for: tmpyi9zszy5/bellinge_small/bbox_1/model_1/model_1.inp
0%| | 0/3600 [00:00<?, ?it/s]
42%|āāāāā | 1520.0/3600 [00:00<00:00, 15187.16it/s]
84%|āāāāāāāāā | 3040.0/3600 [00:00<00:00, 14600.02it/s]
2025/05/22 10:50:16 | Model run complete.
3601.0it [00:00, 14208.82it/s]
2025/05/22 10:50:16 | Writing synthetic results.
2025/05/22 10:50:16 | Running the real model.
2025/05/22 10:50:16 | tmpyi9zszy5/real/bellinge_small.inp initialised in pyswmm
2025/05/22 10:50:16 | Starting simulation for: tmpyi9zszy5/real/bellinge_small.inp
0%| | 0/3600 [00:00<?, ?it/s]
2025/05/22 10:50:16 | Model run complete.
3603.0it [00:00, 153240.08it/s]
2025/05/22 10:50:16 | Iterating metrics.
2025/05/22 10:50:17 | outfall_nse_flow completed
2025/05/22 10:50:17 | outfall_kge_flow completed
2025/05/22 10:50:17 | outfall_relerror_flow completed
2025/05/22 10:50:17 | outfall_relerror_length completed
2025/05/22 10:50:17 | outfall_relerror_npipes completed
2025/05/22 10:50:17 | outfall_relerror_nmanholes completed
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/networkx/readwrite/json_graph/node_link.py:287: FutureWarning: The default value will be changed to `edges="edges" in NetworkX 3.6. To make this warning go away, explicitly set the edges kwarg, e.g.: nx.node_link_graph(data, edges="links") to preserve current behavior, or nx.node_link_graph(data, edges="edges") for forward compatibility. warnings.warn(
2025/05/22 10:50:17 | outfall_relerror_diameter completed
2025/05/22 10:50:17 | outfall_nse_flooding completed
2025/05/22 10:50:17 | outfall_kge_flooding completed
2025/05/22 10:50:17 | outfall_relerror_flooding completed
2025/05/22 10:50:17 | grid_nse_flooding completed
2025/05/22 10:50:17 | grid_kge_flooding completed
2025/05/22 10:50:17 | grid_relerror_flooding completed
2025/05/22 10:50:17 | subcatchment_nse_flooding completed
2025/05/22 10:50:17 | subcatchment_kge_flooding completed
2025/05/22 10:50:17 | subcatchment_relerror_flooding completed
2025/05/22 10:50:17 | bias_flood_depth completed
2025/05/22 10:50:17 | kstest_edge_betweenness completed
2025/05/22 10:50:17 | kstest_betweenness completed
2025/05/22 10:50:17 | outfall_kstest_diameters completed
2025/05/22 10:50:17 | nc_deltacon0 completed
2025/05/22 10:50:17 | nc_laplacian_dist completed
2025/05/22 10:50:17 | nc_vertex_edge_distance completed
2025/05/22 10:50:17 | Metrics complete
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/swmmanywhere/metric_utilities.py:1077: RuntimeWarning: divide by zero encountered in scalar divide return (syn_tot - real_tot) / real_tot
Plot resultsĀ¶
We can plot the real network data and simulation (click on links for flow and nodes for flooding).
InĀ [4]:
Copied!
## View real data
plot_map(real_dir)
## View real data
plot_map(real_dir)
Out[4]:
Make this Notebook Trusted to load map: File -> Trust Notebook
... and we can plot the synthesised network
InĀ [5]:
Copied!
## View output
model_file = outputs[0]
plot_map(model_file.parent)
## View output
model_file = outputs[0]
plot_map(model_file.parent)
Skipping field osmid: unsupported OGR type: 1
Skipping field lanes: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
Out[5]:
Make this Notebook Trusted to load map: File -> Trust Notebook
.. but of course we can see that the two networks do not perfectly line up. So we
can't exactly plot our timeseries side-by-side. To quantify this properly we need to
draw on SWMManywhere's ability to compare two networks that don't line up, which is
done using the metrics output, automatically calculated when real data is
provided.
InĀ [6]:
Copied!
# View metrics
metrics = outputs[1]
print(metrics)
# View metrics
metrics = outputs[1]
print(metrics)
{'outfall_nse_flow': np.float64(0.2238031965014633), 'outfall_kge_flow': np.float64(0.06435783886618196), 'outfall_relerror_flow': np.float64(-0.5207437819237698), 'outfall_relerror_length': np.float64(0.05450306902826656), 'outfall_relerror_npipes': np.float64(0.42857142857142855), 'outfall_relerror_nmanholes': np.float64(0.4), 'outfall_relerror_diameter': np.float64(-0.17826086956521758), 'outfall_nse_flooding': inf, 'outfall_kge_flooding': inf, 'outfall_relerror_flooding': inf, 'grid_nse_flooding': nan, 'grid_kge_flooding': nan, 'grid_relerror_flooding': nan, 'subcatchment_nse_flooding': nan, 'subcatchment_kge_flooding': nan, 'subcatchment_relerror_flooding': nan, 'bias_flood_depth': np.float64(inf), 'kstest_edge_betweenness': np.float64(0.9133858267716536), 'kstest_betweenness': np.float64(0.572972972972973), 'outfall_kstest_diameters': np.float64(0.65), 'nc_deltacon0': np.float64(0.0009323275547564582), 'nc_laplacian_dist': np.float64(3.7416573867739413), 'nc_vertex_edge_distance': 0.940766550522648}
For more information on using metrics see our metrics guide. To understand how to make use of such information, see our paper for example.