Custom data demo¶
Note - this script can also be opened in interactive Python if you wanted to
play around. On the GitHub it is in docs/notebooks.
To run this on your local machine, you will need to install the optional dependencies
for doc:
pip install swmmanywhere[doc]
Introduction¶
This script demonstrates how to use swmmanywhere when you have custom data.
Since this is a notebook, we will define config
as a dictionary rather than a yaml file, but the same principles apply.
Initial setup¶
We will use the same example as the extended demo, but with a custom elevation dataset. Let's start by rerunning it.
In [1]:
Copied!
# Imports
from __future__ import annotations
import tempfile
from pathlib import Path
import folium
import geopandas as gpd
from swmmanywhere.logging import set_verbose
from swmmanywhere.swmmanywhere import swmmanywhere
from swmmanywhere.utilities import plot_map
# Create temporary directory
temp_dir = tempfile.TemporaryDirectory(dir=".")
base_dir = Path(temp_dir.name)
# Define minimum viable config
bbox = (1.52740, 42.50524, 1.54273, 42.51259)
config = {
"base_dir": base_dir,
"project": "my_first_swmm",
"bbox": bbox,
"run_settings": {"duration": 3600},
"parameter_overrides": {
"topology_derivation": {
"allowable_networks": ["drive"],
"omit_edges": ["bridge"],
},
"outfall_derivation": {
"outfall_length": 5,
"river_buffer_distance": 30,
},
},
}
set_verbose(True) # Set verbosity
# Run SWMManywhere
outputs = swmmanywhere(config)
model_dir = outputs[0].parent
# Imports
from __future__ import annotations
import tempfile
from pathlib import Path
import folium
import geopandas as gpd
from swmmanywhere.logging import set_verbose
from swmmanywhere.swmmanywhere import swmmanywhere
from swmmanywhere.utilities import plot_map
# Create temporary directory
temp_dir = tempfile.TemporaryDirectory(dir=".")
base_dir = Path(temp_dir.name)
# Define minimum viable config
bbox = (1.52740, 42.50524, 1.54273, 42.51259)
config = {
"base_dir": base_dir,
"project": "my_first_swmm",
"bbox": bbox,
"run_settings": {"duration": 3600},
"parameter_overrides": {
"topology_derivation": {
"allowable_networks": ["drive"],
"omit_edges": ["bridge"],
},
"outfall_derivation": {
"outfall_length": 5,
"river_buffer_distance": 30,
},
},
}
set_verbose(True) # Set verbosity
# Run SWMManywhere
outputs = swmmanywhere(config)
model_dir = outputs[0].parent
/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
2025/10/10 14:35:28 | Creating project structure.
2025/10/10 14:35:28 | Project structure created at tmpq9xfrgih
2025/10/10 14:35:28 | Project name: my_first_swmm
2025/10/10 14:35:28 | Bounding box: (1.5274, 42.50524, 1.54273, 42.51259),
number: 1
2025/10/10 14:35:28 | Model number: 1
2025/10/10 14:35:28 | Loading and setting parameters.
2025/10/10 14:35:28 | Setting topology_derivation allowable_networks to ['drive']
2025/10/10 14:35:28 | Setting topology_derivation omit_edges to ['bridge']
2025/10/10 14:35:28 | Setting outfall_derivation outfall_length to 5
2025/10/10 14:35:28 | Setting outfall_derivation river_buffer_distance to 30
2025/10/10 14:35:28 | Allowable networks have been changed, removing old street graph.
2025/10/10 14:35:28 | Running downloads.
2025/10/10 14:35:28 | downloading elevation to tmpq9xfrgih/my_first_swmm/bbox_1/download/elevation.tif
2025/10/10 14:35:35 | downloading buildings to tmpq9xfrgih/my_first_swmm/bbox_1/download/building.geoparquet
2025/10/10 14:35:48 | downloading network to tmpq9xfrgih/my_first_swmm/bbox_1/download/street.parquet
2025/10/10 14:36:51 | downloading river network to tmpq9xfrgih/my_first_swmm/bbox_1/download/river.parquet
2025/10/10 14:36:52 | Iterating graphs.
2025/10/10 14:36:52 | Iterating graph functions.
2025/10/10 14:36:52 | graphfcn: assign_id completed.
2025/10/10 14:36:52 | graphfcn: fix_geometries completed.
2025/10/10 14:36:52 | graphfcn: remove_non_pipe_allowable_links completed.
/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/networkx/readwrite/json_graph/node_link.py:290: FutureWarning: The default value will be changed to `edges="edges" in NetworkX 3.6. To make this warning go away, explicitly set the edges kwarg, e.g.: nx.node_link_graph(data, edges="links") to preserve current behavior, or nx.node_link_graph(data, edges="edges") for forward compatibility. warnings.warn(
2025/10/10 14:36:52 | graphfcn: calculate_streetcover completed.
2025/10/10 14:36:52 | graphfcn: remove_parallel_edges completed.
2025/10/10 14:36:53 | graphfcn: to_undirected completed.
2025/10/10 14:36:53 | graphfcn: split_long_edges completed.
2025/10/10 14:36:53 | graphfcn: merge_street_nodes completed.
2025/10/10 14:36:53 | graphfcn: assign_id completed.
2025/10/10 14:36:58 | graphfcn: clip_to_catchments completed.
2025/10/10 14:36:59 | graphfcn: calculate_contributing_area completed.
2025/10/10 14:36:59 | graphfcn: set_elevation completed.
2025/10/10 14:36:59 | graphfcn: double_directed completed.
2025/10/10 14:37:00 | graphfcn: fix_geometries completed.
2025/10/10 14:37:00 | graphfcn: set_surface_slope completed.
2025/10/10 14:37:00 | graphfcn: set_chahinian_slope completed.
2025/10/10 14:37:00 | graphfcn: set_chahinian_angle completed.
2025/10/10 14:37:00 | graphfcn: calculate_weights completed.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
858, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
823, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
507, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
674, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
754, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
779, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
1255, using this node as outfall.
2025/10/10 14:37:00 | No outfalls found for subgraph containing
1535, using this node as outfall.
2025/10/10 14:37:00 | graphfcn: identify_outfalls completed.
2025/10/10 14:37:01 | Total graph weight 264.28645874589057.
2025/10/10 14:37:01 | graphfcn: derive_topology completed.
0%| | 0/395 [00:00<?, ?it/s]
11%|█ | 42/395 [00:00<00:00, 415.47it/s]
22%|██▏ | 88/395 [00:00<00:00, 433.94it/s]
33%|███▎ | 132/395 [00:00<00:00, 407.61it/s]
44%|████▍ | 173/395 [00:00<00:00, 344.30it/s]
53%|█████▎ | 209/395 [00:00<00:00, 323.32it/s]
62%|██████▏ | 243/395 [00:00<00:00, 291.37it/s]
69%|██████▉ | 273/395 [00:00<00:00, 283.80it/s]
77%|███████▋ | 304/395 [00:00<00:00, 287.43it/s]
85%|████████▍ | 334/395 [00:01<00:00, 286.69it/s]
92%|█████████▏| 363/395 [00:01<00:00, 256.55it/s]
99%|█████████▊| 390/395 [00:01<00:00, 259.13it/s]
100%|██████████| 395/395 [00:01<00:00, 297.39it/s]
2025/10/10 14:37:02 | graphfcn: pipe_by_pipe completed.
2025/10/10 14:37:02 | graphfcn: fix_geometries completed.
2025/10/10 14:37:02 | graphfcn: assign_id completed.
2025/10/10 14:37:02 | Saving final graph and writing inp file.
Skipping field osmid: unsupported OGR type: 1
Skipping field highway: unsupported OGR type: 5
Skipping field lanes: unsupported OGR type: 5
Skipping field maxspeed: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
2025/10/10 14:37:03 | Running the synthetic model.
2025/10/10 14:37:03 | tmpq9xfrgih/my_first_swmm/bbox_1/model_1/model_1.inp initialised in pyswmm
2025/10/10 14:37:03 | Starting simulation for: tmpq9xfrgih/my_first_swmm/bbox_1/model_1/model_1.inp
0%| | 0/3600 [00:00<?, ?it/s]
13%|█▎ | 483.0/3600 [00:00<00:00, 4815.13it/s]
27%|██▋ | 965.0/3600 [00:00<00:01, 2388.51it/s]
35%|███▌ | 1268.0/3600 [00:00<00:01, 2160.15it/s]
44%|████▍ | 1580.0/3600 [00:00<00:00, 2406.82it/s]
56%|█████▌ | 2021.0/3600 [00:00<00:00, 2948.13it/s]
75%|███████▍ | 2695.0/3600 [00:00<00:00, 4001.00it/s]
2025/10/10 14:37:04 | Model run complete.
3605.0it [00:00, 3748.40it/s]
2025/10/10 14:37:04 | Writing synthetic results.
2025/10/10 14:37:04 | No real network provided, returning SWMM .inp file.
Plotting output¶
Now we can plot the output. To highlight the differences in the supplied data that we are about to demonstrate, we also plot the subbasins.
In [2]:
Copied!
m = plot_map(model_dir)
subbasins = gpd.read_file(model_dir / "subbasins.geoparquet")
folium.GeoJson(subbasins, fill_opacity=0, color="blue", weight=2).add_to(m)
m
m = plot_map(model_dir)
subbasins = gpd.read_file(model_dir / "subbasins.geoparquet")
folium.GeoJson(subbasins, fill_opacity=0, color="blue", weight=2).add_to(m)
m
Skipping field osmid: unsupported OGR type: 1
Skipping field highway: unsupported OGR type: 5
Skipping field lanes: unsupported OGR type: 5
Skipping field maxspeed: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
Out[2]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Supply custom elevation data¶
To keep things simple, we will just download some elevation data for the same area and perturb it, though in practice you are likely to use some higher resolution or more accurate data.
You don't need to worry about your files lining up perfectly (though of course if they do not overlap at all then you will run into problems).
In [3]:
Copied!
# Import NASADEM downloader and reprojection tools
from swmmanywhere.geospatial_utilities import ( # noqa: E402
get_utm_epsg,
reproject_raster,
)
from swmmanywhere.prepare_data import download_elevation # noqa: E402
# Download and reproject the correct elevation to UTM
download_elevation(base_dir / "elevation.tif", bbox)
reproject_raster(
get_utm_epsg(bbox[0], bbox[1]),
base_dir / "elevation.tif",
base_dir / "elevation_utm.tif",
)
# Flip it
import numpy as np # noqa: E402
import rasterio # noqa: E402
with rasterio.open(base_dir / "elevation_utm.tif") as src:
data = np.fliplr(src.read(1))
with rasterio.open(base_dir / "fake_elevation.tif", "w", **src.profile) as dst:
dst.write(data, 1)
# Import NASADEM downloader and reprojection tools
from swmmanywhere.geospatial_utilities import ( # noqa: E402
get_utm_epsg,
reproject_raster,
)
from swmmanywhere.prepare_data import download_elevation # noqa: E402
# Download and reproject the correct elevation to UTM
download_elevation(base_dir / "elevation.tif", bbox)
reproject_raster(
get_utm_epsg(bbox[0], bbox[1]),
base_dir / "elevation.tif",
base_dir / "elevation_utm.tif",
)
# Flip it
import numpy as np # noqa: E402
import rasterio # noqa: E402
with rasterio.open(base_dir / "elevation_utm.tif") as src:
data = np.fliplr(src.read(1))
with rasterio.open(base_dir / "fake_elevation.tif", "w", **src.profile) as dst:
dst.write(data, 1)
Update config and run again¶
Now we update the elevation entry in the address_overrides part of the
config to point to the new elevation data, then rerun swmmanywhere.
In [4]:
Copied!
# Update config
config["address_overrides"] = {
"elevation": str(base_dir / "fake_elevation.tif"),
}
# Run again
outputs = swmmanywhere(config)
model_dir = outputs[0].parent
# Update config
config["address_overrides"] = {
"elevation": str(base_dir / "fake_elevation.tif"),
}
# Run again
outputs = swmmanywhere(config)
model_dir = outputs[0].parent
2025/10/10 14:38:13 | Creating project structure.
2025/10/10 14:38:13 | Project structure created at tmpq9xfrgih
2025/10/10 14:38:13 | Project name: my_first_swmm
2025/10/10 14:38:13 | Bounding box: (1.5274, 42.50524, 1.54273, 42.51259),
number: 1
2025/10/10 14:38:13 | Model number: 2
2025/10/10 14:38:13 | Loading and setting parameters.
2025/10/10 14:38:13 | Setting topology_derivation allowable_networks to ['drive']
2025/10/10 14:38:13 | Setting topology_derivation omit_edges to ['bridge']
2025/10/10 14:38:13 | Setting outfall_derivation outfall_length to 5
2025/10/10 14:38:13 | Setting outfall_derivation river_buffer_distance to 30
2025/10/10 14:38:13 | Allowable networks have been changed, removing old street graph.
2025/10/10 14:38:13 | Running downloads.
2025/10/10 14:38:13 | downloading network to tmpq9xfrgih/my_first_swmm/bbox_1/download/street.parquet
2025/10/10 14:38:16 | Iterating graphs.
2025/10/10 14:38:16 | Iterating graph functions.
2025/10/10 14:38:16 | graphfcn: assign_id completed.
2025/10/10 14:38:16 | graphfcn: fix_geometries completed.
2025/10/10 14:38:16 | graphfcn: remove_non_pipe_allowable_links completed.
/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/networkx/readwrite/json_graph/node_link.py:290: FutureWarning: The default value will be changed to `edges="edges" in NetworkX 3.6. To make this warning go away, explicitly set the edges kwarg, e.g.: nx.node_link_graph(data, edges="links") to preserve current behavior, or nx.node_link_graph(data, edges="edges") for forward compatibility. warnings.warn(
2025/10/10 14:38:16 | graphfcn: calculate_streetcover completed.
2025/10/10 14:38:16 | graphfcn: remove_parallel_edges completed.
2025/10/10 14:38:16 | graphfcn: to_undirected completed.
2025/10/10 14:38:17 | graphfcn: split_long_edges completed.
2025/10/10 14:38:17 | graphfcn: merge_street_nodes completed.
2025/10/10 14:38:17 | graphfcn: assign_id completed.
2025/10/10 14:38:18 | graphfcn: clip_to_catchments completed.
2025/10/10 14:38:20 | graphfcn: calculate_contributing_area completed.
2025/10/10 14:38:20 | graphfcn: set_elevation completed.
2025/10/10 14:38:20 | graphfcn: double_directed completed.
2025/10/10 14:38:20 | graphfcn: fix_geometries completed.
2025/10/10 14:38:20 | graphfcn: set_surface_slope completed.
2025/10/10 14:38:20 | graphfcn: set_chahinian_slope completed.
2025/10/10 14:38:20 | graphfcn: set_chahinian_angle completed.
2025/10/10 14:38:21 | graphfcn: calculate_weights completed.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
474, using this node as outfall.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
653, using this node as outfall.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
713, using this node as outfall.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
745, using this node as outfall.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
780, using this node as outfall.
2025/10/10 14:38:21 | No outfalls found for subgraph containing
1536, using this node as outfall.
2025/10/10 14:38:21 | graphfcn: identify_outfalls completed.
2025/10/10 14:38:21 | Total graph weight 277.4753575488574.
2025/10/10 14:38:21 | graphfcn: derive_topology completed.
0%| | 0/386 [00:00<?, ?it/s]
9%|▉ | 36/386 [00:00<00:00, 356.55it/s]
19%|█▊ | 72/386 [00:00<00:01, 297.76it/s]
27%|██▋ | 103/386 [00:00<00:01, 276.75it/s]
34%|███▍ | 132/386 [00:00<00:00, 281.12it/s]
42%|████▏ | 161/386 [00:00<00:01, 217.84it/s]
48%|████▊ | 185/386 [00:00<00:01, 194.16it/s]
53%|█████▎ | 206/386 [00:00<00:00, 185.06it/s]
60%|█████▉ | 230/386 [00:01<00:00, 196.92it/s]
65%|██████▌ | 251/386 [00:01<00:00, 198.12it/s]
72%|███████▏ | 276/386 [00:01<00:00, 210.65it/s]
79%|███████▉ | 304/386 [00:01<00:00, 227.93it/s]
86%|████████▌ | 331/386 [00:01<00:00, 238.36it/s]
92%|█████████▏| 356/386 [00:01<00:00, 227.92it/s]
98%|█████████▊| 380/386 [00:01<00:00, 223.79it/s]
100%|██████████| 386/386 [00:01<00:00, 225.39it/s]
2025/10/10 14:38:23 | graphfcn: pipe_by_pipe completed.
2025/10/10 14:38:23 | graphfcn: fix_geometries completed.
2025/10/10 14:38:23 | graphfcn: assign_id completed.
2025/10/10 14:38:23 | Saving final graph and writing inp file.
Skipping field osmid: unsupported OGR type: 1
Skipping field highway: unsupported OGR type: 5
Skipping field lanes: unsupported OGR type: 5
Skipping field maxspeed: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
2025/10/10 14:38:24 | Running the synthetic model.
2025/10/10 14:38:24 | tmpq9xfrgih/my_first_swmm/bbox_1/model_2/model_2.inp initialised in pyswmm
2025/10/10 14:38:24 | Starting simulation for: tmpq9xfrgih/my_first_swmm/bbox_1/model_2/model_2.inp
0%| | 0/3600 [00:00<?, ?it/s]
14%|█▍ | 501.0/3600 [00:00<00:00, 4992.22it/s]
28%|██▊ | 1001.0/3600 [00:00<00:01, 2497.80it/s]
37%|███▋ | 1317.0/3600 [00:00<00:00, 2399.73it/s]
46%|████▋ | 1668.0/3600 [00:00<00:00, 2701.30it/s]
64%|██████▍ | 2297.0/3600 [00:00<00:00, 3704.22it/s]
86%|████████▌ | 3096.0/3600 [00:00<00:00, 4929.06it/s]
2025/10/10 14:38:25 | Model run complete.
3603.0it [00:00, 4074.28it/s]
2025/10/10 14:38:25 | Writing synthetic results.
2025/10/10 14:38:25 | No real network provided, returning SWMM .inp file.
Plotting output¶
This time we will include both the original (blue) and the new (red) subbasins to highlight the impact of flipping the elevation data.
In [5]:
Copied!
m = plot_map(model_dir)
subbasins_new = gpd.read_file(model_dir / "subbasins.geoparquet")
folium.GeoJson(subbasins_new, fill_opacity=0, color="red", weight=2).add_to(m)
folium.GeoJson(subbasins, fill_opacity=0, color="blue", weight=2).add_to(m)
m
m = plot_map(model_dir)
subbasins_new = gpd.read_file(model_dir / "subbasins.geoparquet")
folium.GeoJson(subbasins_new, fill_opacity=0, color="red", weight=2).add_to(m)
folium.GeoJson(subbasins, fill_opacity=0, color="blue", weight=2).add_to(m)
m
Skipping field osmid: unsupported OGR type: 1
Skipping field highway: unsupported OGR type: 5
Skipping field lanes: unsupported OGR type: 5
Skipping field maxspeed: unsupported OGR type: 5
Skipping field name: unsupported OGR type: 5
Skipping field reversed: unsupported OGR type: 1
Out[5]:
Make this Notebook Trusted to load map: File -> Trust Notebook
In Summary¶
This tutorial has demonstrated how we can supply our own data in place of the existing files that are downloaded by SWMManywhere. This method works for any filepath, which are the files used by the current workflow.
You can also use address_overrides to you add your own custom data that
might be used by a custom graphfcn
, for example. If you are adding new data to the SWMManywhere workflow in this way,
you will have to access it from the FilePaths object with the get_path method.