WSIMOD model demonstration - Oxford (.py)¶

Note - this script can also be opened in interactive Python if you wanted to play around. On the GitHub it is in docs/demo/scripts

1. Introduction

2. Data

3. Nodes

   3.1 Freshwater Treatment Works

   3.2 Land

   3.3 Demand

   3.4 Reservoir

   3.5 Distribution

   3.6 Wastewater Treatment Works

   3.7 Sewers

   3.8 Groundwater

   3.9 Node list

4. Arcs

   4.1 Arc parameters

   4.2 Create the arcs

5. Mapping

6. Orchestration

   6.1 Orchestrating an individual timestep

   6.2 Ending a timestep

7. Model object

   7.1 Validation

We will cover a demo WSIMOD case study¶

The glamorous town of Oxford will be our demo case study. Below, we will create these nodes and arcs, orchestrate them into a model, and run simulations.

Although GIS is pretty, the schematic below is a more accurate representation of what will be created. WSIMOD treats everything as a node or an arc.

Imports and forcing data¶

Import packages

In [1]:

import os

import pandas as pd

from wsimod.arcs.arcs import Arc
from wsimod.core import constants
from wsimod.nodes.catchment import Catchment
from wsimod.nodes.demand import ResidentialDemand
from wsimod.nodes.land import Land
from wsimod.nodes.nodes import Node
from wsimod.nodes.sewer import Sewer
from wsimod.nodes.storage import Groundwater, Reservoir
from wsimod.nodes.waste import Waste
from wsimod.nodes.wtw import FWTW, WWTW
from wsimod.orchestration.model import Model

os.environ["USE_PYGEOS"] = "0"
import geopandas as gpd
from matplotlib import pyplot as plt
from shapely.geometry import LineString

import os import pandas as pd from wsimod.arcs.arcs import Arc from wsimod.core import constants from wsimod.nodes.catchment import Catchment from wsimod.nodes.demand import ResidentialDemand from wsimod.nodes.land import Land from wsimod.nodes.nodes import Node from wsimod.nodes.sewer import Sewer from wsimod.nodes.storage import Groundwater, Reservoir from wsimod.nodes.waste import Waste from wsimod.nodes.wtw import FWTW, WWTW from wsimod.orchestration.model import Model os.environ["USE_PYGEOS"] = "0" import geopandas as gpd from matplotlib import pyplot as plt from shapely.geometry import LineString

Load input data

In [2]:

# Select the root path for the data folder. Use the appropriate value for your case.
data_folder = os.path.join(os.path.abspath(""), "docs", "demo", "data")

input_fid = os.path.join(data_folder, "processed", "timeseries_data.csv")
input_data = pd.read_csv(input_fid)
input_data.loc[input_data.variable == "flow", "value"] *= constants.M3_S_TO_M3_DT
input_data.loc[input_data.variable == "precipitation", "value"] *= constants.MM_TO_M
input_data.date = pd.to_datetime(input_data.date)
data_input_dict = input_data.set_index(["variable", "date"]).value.to_dict()
data_input_dict = (
    input_data.groupby("site")
    .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict())
    .to_dict()
)
print(input_data.sample(10))

# Select the root path for the data folder. Use the appropriate value for your case. data_folder = os.path.join(os.path.abspath(""), "docs", "demo", "data") input_fid = os.path.join(data_folder, "processed", "timeseries_data.csv") input_data = pd.read_csv(input_fid) input_data.loc[input_data.variable == "flow", "value"] *= constants.M3_S_TO_M3_DT input_data.loc[input_data.variable == "precipitation", "value"] *= constants.MM_TO_M input_data.date = pd.to_datetime(input_data.date) data_input_dict = input_data.set_index(["variable", "date"]).value.to_dict() data_input_dict = ( input_data.groupby("site") .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict()) .to_dict() ) print(input_data.sample(10))

              site       date     variable      value
69379       thames 2011-10-06        boron   0.000061
78395  oxford_land 2012-07-12          et0   0.002000
48165     evenlode 2009-06-28    potassium   0.003786
5425        thames 2012-01-24  temperature   6.800000
49550          ray 2009-04-18    potassium   0.009433
4777        thames 2010-04-16  temperature  11.212500
52413       thames 2013-02-23    potassium   0.002529
18726     cherwell 2012-08-08     chloride   0.042571
49088     evenlode 2012-01-07    potassium   0.004171
40848     cherwell 2009-05-22       sodium   0.031900

C:\Users\bdobson\AppData\Local\Temp\ipykernel_14356\946595242.py:12: DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.
  .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict())

Input data is stored in dicts

In [3]:

print(data_input_dict["cherwell"][("boron", pd.to_datetime("2010-11-20"))])

print(data_input_dict["cherwell"][("boron", pd.to_datetime("2010-11-20"))])

6.985714285714286e-05

We select dates that are available in the input data

In [4]:

dates = input_data.date.unique()
dates = dates[dates.argsort()]
dates = [pd.Timestamp(x) for x in dates]
print(dates[0:10])

dates = input_data.date.unique() dates = dates[dates.argsort()] dates = [pd.Timestamp(x) for x in dates] print(dates[0:10])

[Timestamp('2009-03-03 00:00:00'), Timestamp('2009-03-04 00:00:00'), Timestamp('2009-03-05 00:00:00'), Timestamp('2009-03-06 00:00:00'), Timestamp('2009-03-07 00:00:00'), Timestamp('2009-03-08 00:00:00'), Timestamp('2009-03-09 00:00:00'), Timestamp('2009-03-10 00:00:00'), Timestamp('2009-03-11 00:00:00'), Timestamp('2009-03-12 00:00:00')]

We can specify the pollutants. In this example we choose based on what pollutants we have input data for.

In [5]:

constants.POLLUTANTS = input_data.variable.unique().tolist()
constants.POLLUTANTS.remove("flow")
constants.POLLUTANTS.remove("precipitation")
constants.POLLUTANTS.remove("et0")
constants.NON_ADDITIVE_POLLUTANTS = ["temperature"]
constants.ADDITIVE_POLLUTANTS = list(
    set(constants.POLLUTANTS).difference(constants.NON_ADDITIVE_POLLUTANTS)
)
constants.FLOAT_ACCURACY = 1e-11
print(constants.POLLUTANTS)

constants.POLLUTANTS = input_data.variable.unique().tolist() constants.POLLUTANTS.remove("flow") constants.POLLUTANTS.remove("precipitation") constants.POLLUTANTS.remove("et0") constants.NON_ADDITIVE_POLLUTANTS = ["temperature"] constants.ADDITIVE_POLLUTANTS = list( set(constants.POLLUTANTS).difference(constants.NON_ADDITIVE_POLLUTANTS) ) constants.FLOAT_ACCURACY = 1e-11 print(constants.POLLUTANTS)

['temperature', 'silicon', 'fluoride', 'chloride', 'nitrate', 'sulphate', 'nitrogen', 'sodium', 'potassium', 'calcium', 'magnesium', 'boron']

Create nodes¶

For waste nodes, no parameters are needed, they are just the model outlet

In [6]:

thames_above_abingdon = Waste(name="thames_above_abingdon")

thames_above_abingdon = Waste(name="thames_above_abingdon")

For junctions and abstraction locations, we can simply use the default nodes

In [7]:

farmoor_abstraction = Node(name="farmoor_abstraction")
evenlode_thames = Node(name="evenlode_thames")
cherwell_ray = Node(name="cherwell_ray")
cherwell_thames = Node(name="cherwell_thames")
thames_mixer = Node(name="thames_mixer")

farmoor_abstraction = Node(name="farmoor_abstraction") evenlode_thames = Node(name="evenlode_thames") cherwell_ray = Node(name="cherwell_ray") cherwell_thames = Node(name="cherwell_thames") thames_mixer = Node(name="thames_mixer")

For catchment nodes, we only need to specify the input data (as a dictionary format).

In [8]:

evenlode = Catchment(name="evenlode", data_input_dict=data_input_dict["evenlode"])
thames = Catchment(name="thames", data_input_dict=data_input_dict["thames"])
ray = Catchment(name="ray", data_input_dict=data_input_dict["ray"])
cherwell = Catchment(name="cherwell", data_input_dict=data_input_dict["cherwell"])

evenlode = Catchment(name="evenlode", data_input_dict=data_input_dict["evenlode"]) thames = Catchment(name="thames", data_input_dict=data_input_dict["thames"]) ray = Catchment(name="ray", data_input_dict=data_input_dict["ray"]) cherwell = Catchment(name="cherwell", data_input_dict=data_input_dict["cherwell"])

We can see that, even though we provided mimimal information (name and input data) each node comes with many predefined functions.

In [9]:

print(dir(evenlode))

print(dir(evenlode))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'apply_overrides', 'blend_vqip', 'check_basic', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'data_input_dict', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'end_timestep_', 'extract_vqip', 'extract_vqip_c', 'get_avail', 'get_connected', 'get_data_input', 'get_direction_arcs', 'get_flow', 'in_arcs', 'in_arcs_type', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'node_mass_balance', 'out_arcs', 'out_arcs_type', 'pull_check', 'pull_check_abstraction', 'pull_check_basic', 'pull_check_deny', 'pull_check_handler', 'pull_distributed', 'pull_set', 'pull_set_abstraction', 'pull_set_deny', 'pull_set_handler', 'push_check', 'push_check_accept', 'push_check_basic', 'push_check_deny', 'push_check_handler', 'push_distributed', 'push_set', 'push_set_deny', 'push_set_handler', 'query_handler', 'reinit', 'route', 'sum_vqip', 't', 'total_in', 'total_out', 'total_to_concentration', 'unrouted_water', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c']

Freshwater treatment works¶

Each type of node uses different parameters (see API reference). Below we create a freshwater treatment works (FWTW)

In [10]:

oxford_fwtw = FWTW(
    service_reservoir_storage_capacity=1e5,
    service_reservoir_storage_area=2e4,
    treatment_throughput_capacity=4.5e4,
    name="oxford_fwtw",
)

oxford_fwtw = FWTW( service_reservoir_storage_capacity=1e5, service_reservoir_storage_area=2e4, treatment_throughput_capacity=4.5e4, name="oxford_fwtw", )

Each node type has different types of functionality available

In [11]:

print(dir(oxford_fwtw))

print(dir(oxford_fwtw))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_liquor_multiplier', '_percent_solids', 'apply_overrides', 'blend_vqip', 'calculate_volume', 'check_basic', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'current_input', 'data_input_dict', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'end_timestep_', 'extract_vqip', 'extract_vqip_c', 'get_connected', 'get_data_input', 'get_direction_arcs', 'get_excess_throughput', 'in_arcs', 'in_arcs_type', 'liquor', 'liquor_multiplier', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'node_mass_balance', 'out_arcs', 'out_arcs_type', 'percent_solids', 'previous_pulled', 'process_parameters', 'pull_check', 'pull_check_basic', 'pull_check_deny', 'pull_check_fwtw', 'pull_check_handler', 'pull_distributed', 'pull_set', 'pull_set_deny', 'pull_set_fwtw', 'pull_set_handler', 'push_check', 'push_check_accept', 'push_check_basic', 'push_check_deny', 'push_check_handler', 'push_distributed', 'push_set', 'push_set_deny', 'push_set_handler', 'query_handler', 'reinit', 'service_reservoir_initial_storage', 'service_reservoir_storage_area', 'service_reservoir_storage_capacity', 'service_reservoir_storage_elevation', 'service_reservoir_tank', 'solids', 'sum_vqip', 't', 'total_deficit', 'total_in', 'total_out', 'total_pulled', 'total_to_concentration', 'treat_current_input', 'treat_water', 'treated', 'treatment_throughput_capacity', 'unpushed_sludge', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c']

The FWTW node has a tank representing the service reservoirs, we can see that it has been initialised empty.

In [12]:

print(oxford_fwtw.service_reservoir_tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

If we try to pull water from the FWTW, it responds that there is no water to pull.

In [13]:

print(oxford_fwtw.pull_check({"volume": 10}))

print(oxford_fwtw.pull_check({"volume": 10}))

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

If we add in some water, we see the pull check responds that water is available.

In [14]:

oxford_fwtw.service_reservoir_tank.storage["volume"] += 25
print(oxford_fwtw.pull_check({"volume": 10}))

oxford_fwtw.service_reservoir_tank.storage["volume"] += 25 print(oxford_fwtw.pull_check({"volume": 10}))

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 10.0}

When we set a pull request, we see that we successfully receive the water and the tank is updated.

In [15]:

reply = oxford_fwtw.pull_set({"volume": 10})
print(reply)
print(oxford_fwtw.service_reservoir_tank.storage)

reply = oxford_fwtw.pull_set({"volume": 10}) print(reply) print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 10.0}
{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

Land¶

We will now create a land node, it is a bit involved so you might want to skip ahead to demand, or check out the land node tutorial

Data inputs are a single dictionary

In [16]:

land_inputs = data_input_dict["oxford_land"]

print(land_inputs[("precipitation", pd.to_datetime("2010-11-20"))])
print(land_inputs[("et0", pd.to_datetime("2010-11-20"))])
print(land_inputs[("temperature", pd.to_datetime("2009-10-15"))])

land_inputs = data_input_dict["oxford_land"] print(land_inputs[("precipitation", pd.to_datetime("2010-11-20"))]) print(land_inputs[("et0", pd.to_datetime("2010-11-20"))]) print(land_inputs[("temperature", pd.to_datetime("2009-10-15"))])

0.0003
0.002
13.107142857142858

Assign some default pollutant deposition values (kg/m2/d)

In [17]:

pollutant_deposition = {
    "boron": 100e-10,
    "calcium": 70e-7,
    "chloride": 60e-10,
    "fluoride": 0.2e-7,
    "magnesium": 6e-7,
    "nitrate": 2e-9,
    "nitrogen": 4e-7,
    "potassium": 7e-7,
    "silicon": 7e-9,
    "sodium": 30e-9,
    "sulphate": 70e-7,
}

pollutant_deposition = { "boron": 100e-10, "calcium": 70e-7, "chloride": 60e-10, "fluoride": 0.2e-7, "magnesium": 6e-7, "nitrate": 2e-9, "nitrogen": 4e-7, "potassium": 7e-7, "silicon": 7e-9, "sodium": 30e-9, "sulphate": 70e-7, }

Create two surfaces as a list of dicts

In [18]:

surface = [
    {
        "type_": "PerviousSurface",
        "area": 2e7,
        "pollutant_load": pollutant_deposition,
        "surface": "rural",
        "field_capacity": 0.3,
        "depth": 0.5,
        "initial_storage": 2e7 * 0.4 * 0.5,
    },
    {
        "type_": "ImperviousSurface",
        "area": 1e7,
        "pollutant_load": pollutant_deposition,
        "surface": "urban",
        "initial_storage": 5e6,
    },
]

surface = [ { "type_": "PerviousSurface", "area": 2e7, "pollutant_load": pollutant_deposition, "surface": "rural", "field_capacity": 0.3, "depth": 0.5, "initial_storage": 2e7 * 0.4 * 0.5, }, { "type_": "ImperviousSurface", "area": 1e7, "pollutant_load": pollutant_deposition, "surface": "urban", "initial_storage": 5e6, }, ]

Create the land node from these surfaces and the input data

In [19]:

oxford_land = Land(surfaces=surface, name="oxford_land", data_input_dict=land_inputs)

oxford_land = Land(surfaces=surface, name="oxford_land", data_input_dict=land_inputs)

We can see the land node has various tanks that have been initialised empty

In [20]:

print(oxford_land.surface_runoff.storage)
print(oxford_land.subsurface_runoff.storage)
print(oxford_land.percolation.storage)

print(oxford_land.surface_runoff.storage) print(oxford_land.subsurface_runoff.storage) print(oxford_land.percolation.storage)

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

We can see the surfaces have also been initialised, although they are not empty because we provided 'initial_storage' parameters.

In [21]:

rural_surface = oxford_land.get_surface("rural")
urban_surface = oxford_land.get_surface("urban")
print("{0}-{1}".format("rural", rural_surface.storage))
print("{0}-{1}".format("urban", urban_surface.storage))

rural_surface = oxford_land.get_surface("rural") urban_surface = oxford_land.get_surface("urban") print("{0}-{1}".format("rural", rural_surface.storage)) print("{0}-{1}".format("urban", urban_surface.storage))

rural-{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 4000000.0}
urban-{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 5000000.0}

We can run a timestep of the land node with the 'run' command

In [22]:

oxford_land.t = pd.to_datetime("2012-12-22")

oxford_land.run()

oxford_land.t = pd.to_datetime("2012-12-22") oxford_land.run()

We can see that the land and surface tanks have been updated

In [23]:

print(oxford_land.surface_runoff.storage)
print(oxford_land.subsurface_runoff.storage)
print(oxford_land.percolation.storage)

print("{0}-{1}".format("rural", rural_surface.storage))
print("{0}-{1}".format("urban", urban_surface.storage))

print(oxford_land.surface_runoff.storage) print(oxford_land.subsurface_runoff.storage) print(oxford_land.percolation.storage) print("{0}-{1}".format("rural", rural_surface.storage)) print("{0}-{1}".format("urban", urban_surface.storage))

{'temperature': 3.521534008631931, 'silicon': 0.00020062235262918442, 'fluoride': 0.0005732067217976697, 'chloride': 0.0001719620165393009, 'nitrate': 5.732067217976698e-05, 'sulphate': 0.20062235262918443, 'nitrogen': 0.011464134435953396, 'sodium': 0.0008598100826965045, 'potassium': 0.020062235262918445, 'calcium': 0.20062235262918443, 'magnesium': 0.017196201653930095, 'boron': 0.00028660336089883487, 'volume': 12400.000000000002}
{'temperature': 0.9722493642718524, 'silicon': 0.0017554455855053634, 'fluoride': 0.0050155588157296096, 'chloride': 0.0015046676447188828, 'nitrate': 0.000501555881572961, 'sulphate': 1.7554455855053634, 'nitrogen': 0.1003111763145922, 'sodium': 0.007523338223594414, 'potassium': 0.17554455855053633, 'calcium': 1.7554455855053634, 'magnesium': 0.1504667644718883, 'boron': 0.0025077794078648048, 'volume': 58900.0}
{'temperature': 0.3349282031818326, 'silicon': 0.005868203814403644, 'fluoride': 0.016766296612581843, 'chloride': 0.005029888983774552, 'nitrate': 0.001676629661258184, 'sulphate': 5.868203814403645, 'nitrogen': 0.3353259322516368, 'sodium': 0.02514944491887276, 'potassium': 0.5868203814403645, 'calcium': 5.868203814403645, 'magnesium': 0.5029888983774553, 'boron': 0.008383148306290921, 'volume': 176700.00000000003}
rural-{'temperature': 6.203473168254872, 'silicon': 0.1321757282474618, 'fluoride': 0.3776449378498909, 'chloride': 0.11329348135496725, 'nitrate': 0.037764493784989084, 'sulphate': 132.17572824746182, 'nitrogen': 7.552898756997817, 'sodium': 0.5664674067748363, 'potassium': 13.21757282474618, 'calcium': 132.17572824746182, 'magnesium': 11.329348135496726, 'boron': 0.18882246892494545, 'volume': 3980000.0}
urban-{'temperature': 0.13343969249141666, 'silicon': 0.06999999999999999, 'fluoride': 0.2, 'chloride': 0.06, 'nitrate': 0.02, 'sulphate': 70.0, 'nitrogen': 4.0, 'sodium': 0.3, 'potassium': 7.0, 'calcium': 70.0, 'magnesium': 6.0, 'boron': 0.1, 'volume': 5104000.0}

Residential demand¶

The residential demand node requires population, per capita demand and a pollutant_load dictionary that defines how much (weight in kg) pollution is generated per person per day.

In [24]:

oxford = ResidentialDemand(
    name="oxford",
    population=2e5,
    per_capita=0.15,
    pollutant_load={
        "boron": 500 * constants.UG_L_TO_KG_M3 * 0.15,
        "calcium": 150 * constants.MG_L_TO_KG_M3 * 0.15,
        "chloride": 180 * constants.MG_L_TO_KG_M3 * 0.15,
        "fluoride": 0.4 * constants.MG_L_TO_KG_M3 * 0.15,
        "magnesium": 30 * constants.MG_L_TO_KG_M3 * 0.15,
        "nitrate": 60 * constants.MG_L_TO_KG_M3 * 0.15,
        "nitrogen": 50 * constants.MG_L_TO_KG_M3 * 0.15,
        "potassium": 30 * constants.MG_L_TO_KG_M3 * 0.15,
        "silicon": 20 * constants.MG_L_TO_KG_M3 * 0.15,
        "sodium": 200 * constants.MG_L_TO_KG_M3 * 0.15,
        "sulphate": 250 * constants.MG_L_TO_KG_M3 * 0.15,
        "temperature": 14,
    },
    data_input_dict=land_inputs,
)
# pollutant_load calculated based on expected effluent at WWTW

oxford = ResidentialDemand( name="oxford", population=2e5, per_capita=0.15, pollutant_load={ "boron": 500 * constants.UG_L_TO_KG_M3 * 0.15, "calcium": 150 * constants.MG_L_TO_KG_M3 * 0.15, "chloride": 180 * constants.MG_L_TO_KG_M3 * 0.15, "fluoride": 0.4 * constants.MG_L_TO_KG_M3 * 0.15, "magnesium": 30 * constants.MG_L_TO_KG_M3 * 0.15, "nitrate": 60 * constants.MG_L_TO_KG_M3 * 0.15, "nitrogen": 50 * constants.MG_L_TO_KG_M3 * 0.15, "potassium": 30 * constants.MG_L_TO_KG_M3 * 0.15, "silicon": 20 * constants.MG_L_TO_KG_M3 * 0.15, "sodium": 200 * constants.MG_L_TO_KG_M3 * 0.15, "sulphate": 250 * constants.MG_L_TO_KG_M3 * 0.15, "temperature": 14, }, data_input_dict=land_inputs, ) # pollutant_load calculated based on expected effluent at WWTW

Reservoir¶

A reservoir node is used to make abstractions from rivers and supply FWTWs

In [25]:

farmoor = Reservoir(
    name="farmoor", capacity=1e7, initial_storage=1e7, area=1.5e6, datum=62
)

farmoor = Reservoir( name="farmoor", capacity=1e7, initial_storage=1e7, area=1.5e6, datum=62 )

Distribution¶

We use a generic Node as a junction to represent the distribution network between the FWTW and households

In [26]:

distribution = Node(name="oxford_distribution")

distribution = Node(name="oxford_distribution")

Wastewater treatment works¶

Wastewater treatment works (WWTW) are nodes that can store sewage water temporarily in storm tanks, and reduce the pollution amounts in water before releasing them onwards to rivers.

In [27]:

oxford_wwtw = WWTW(
    stormwater_storage_capacity=2e4,
    stormwater_storage_area=2e4,
    treatment_throughput_capacity=5e4,
    name="oxford_wwtw",
)

oxford_wwtw = WWTW( stormwater_storage_capacity=2e4, stormwater_storage_area=2e4, treatment_throughput_capacity=5e4, name="oxford_wwtw", )

Sewers¶

Sewer nodes enable water to transition between households and WWTWs, and between impervious surfaces and rivers or WWTWs. They use a timearea diagram to represent travel time, which assigns a specified percentage of water to take a specified duration to pass through the sewer node.

In [28]:

combined_sewer = Sewer(
    capacity=4e6, pipe_timearea={0: 0.8, 1: 0.15, 2: 0.05}, name="combined_sewer"
)

combined_sewer = Sewer( capacity=4e6, pipe_timearea={0: 0.8, 1: 0.15, 2: 0.05}, name="combined_sewer" )

Groundwater¶

Groundwater nodes implement a simple residence time to determine baseflow

In [29]:

gw = Groundwater(capacity=3.2e9, area=3.2e8, name="gw", residence_time=20)

gw = Groundwater(capacity=3.2e9, area=3.2e8, name="gw", residence_time=20)

Create a nodelist¶

To keep all the nodes in one place, we put them into a list

In [30]:

nodelist = [
    thames_above_abingdon,
    evenlode,
    thames,
    ray,
    cherwell,
    oxford,
    distribution,
    farmoor,
    oxford_fwtw,
    oxford_wwtw,
    combined_sewer,
    oxford_land,
    gw,
    farmoor_abstraction,
    evenlode_thames,
    cherwell_ray,
    cherwell_thames,
    thames_mixer,
]

print(nodelist)

nodelist = [ thames_above_abingdon, evenlode, thames, ray, cherwell, oxford, distribution, farmoor, oxford_fwtw, oxford_wwtw, combined_sewer, oxford_land, gw, farmoor_abstraction, evenlode_thames, cherwell_ray, cherwell_thames, thames_mixer, ] print(nodelist)

[<wsimod.nodes.waste.Waste object at 0x00000201EB8540D0>, <wsimod.nodes.catchment.Catchment object at 0x00000201EBB32610>, <wsimod.nodes.catchment.Catchment object at 0x00000201EB9270D0>, <wsimod.nodes.catchment.Catchment object at 0x00000201EB35B890>, <wsimod.nodes.catchment.Catchment object at 0x00000201EC2577D0>, <wsimod.nodes.demand.ResidentialDemand object at 0x00000201EC280B50>, <wsimod.nodes.nodes.Node object at 0x00000201EC2681D0>, <wsimod.nodes.storage.Reservoir object at 0x00000201EC00A9D0>, <wsimod.nodes.wtw.FWTW object at 0x00000201EC276110>, <wsimod.nodes.wtw.WWTW object at 0x00000201EBFC3410>, <wsimod.nodes.sewer.Sewer object at 0x00000201EBFC1810>, <wsimod.nodes.land.Land object at 0x00000201EC20AFD0>, <wsimod.nodes.storage.Groundwater object at 0x00000201EBFAFE50>, <wsimod.nodes.nodes.Node object at 0x00000201EC275110>, <wsimod.nodes.nodes.Node object at 0x00000201EC274E10>, <wsimod.nodes.nodes.Node object at 0x00000201EC26B250>, <wsimod.nodes.nodes.Node object at 0x00000201EC26A290>, <wsimod.nodes.nodes.Node object at 0x00000201EC2694D0>]

Arcs¶

Arcs link nodes. An example arc is the link between a FWTW and the distribution node

In [31]:

# Standard simple arcs
fwtw_to_distribution = Arc(
    in_port=oxford_fwtw, out_port=distribution, name="fwtw_to_distribution"
)
print(fwtw_to_distribution)

# Standard simple arcs fwtw_to_distribution = Arc( in_port=oxford_fwtw, out_port=distribution, name="fwtw_to_distribution" ) print(fwtw_to_distribution)

<wsimod.arcs.arcs.Arc object at 0x00000201EBFC3550>

As with nodes, even though we only gave it a few parameters, the arc comes with a lot built in

In [32]:

print(dir(fwtw_to_distribution))

print(dir(fwtw_to_distribution))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'apply_overrides', 'arc_mass_balance', 'blend_vqip', 'capacity', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'extract_vqip', 'extract_vqip_c', 'flow_in', 'flow_out', 'get_excess', 'in_port', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'out_port', 'preference', 'reinit', 'send_pull_check', 'send_pull_request', 'send_push_check', 'send_push_request', 'sum_vqip', 'total_to_concentration', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c', 'vqip_in', 'vqip_out']

We can see that the arc links the two nodes

In [33]:

print(fwtw_to_distribution.in_port)

print(fwtw_to_distribution.in_port)

<wsimod.nodes.wtw.FWTW object at 0x00000201EC276110>

In [34]:

print(fwtw_to_distribution.out_port)

print(fwtw_to_distribution.out_port)

<wsimod.nodes.nodes.Node object at 0x00000201EC2681D0>

And that it has updated the nodes that it is connecting.

In [35]:

print(oxford_fwtw.out_arcs)

print(oxford_fwtw.out_arcs)

{'fwtw_to_distribution': <wsimod.arcs.arcs.Arc object at 0x00000201EBFC3550>}

In [36]:

print(distribution.in_arcs)

print(distribution.in_arcs)

{'fwtw_to_distribution': <wsimod.arcs.arcs.Arc object at 0x00000201EBFC3550>}

We use arcs to send checks and requests..

In [37]:

print(fwtw_to_distribution.send_pull_check({"volume": 20}))

print(fwtw_to_distribution.send_pull_check({"volume": 20}))

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

In [38]:

reply = fwtw_to_distribution.send_pull_request({"volume": 20})
print(reply)

reply = fwtw_to_distribution.send_pull_request({"volume": 20}) print(reply)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

They convey this information to the nodes that they connect to, which update their state variables

In [39]:

print(oxford_fwtw.service_reservoir_tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 0.0}

In turn, the arcs update their own state variables.

In [40]:

print(fwtw_to_distribution.flow_in)
print(fwtw_to_distribution.flow_out)

print(fwtw_to_distribution.flow_in) print(fwtw_to_distribution.flow_out)

15.0
15.0

Arc parameters¶

Besides the in/out ports and names, arcs can have a parameter for their capacity, to limit the flow that may pass through it each timestep. A typical example would be on river abstractions to a reservoir

In [41]:

abstraction_to_farmoor = Arc(
    in_port=farmoor_abstraction,
    out_port=farmoor,
    name="abstraction_to_farmoor",
    capacity=5e4,
)

abstraction_to_farmoor = Arc( in_port=farmoor_abstraction, out_port=farmoor, name="abstraction_to_farmoor", capacity=5e4, )

A bit more sophisticated is the 'preference' parameter. We use preference to express where we would prefer the model to send water. In this example, the sewer can send water to both the treatment plant and directly into the river. Of course we would always to prefer to send water to the plant, so we give it a very high preference. Discharging into the river should only be done if there is no capacity left at the WWTW, so we give the arc a very low preference.

In [42]:

sewer_to_wwtw = Arc(
    in_port=combined_sewer, out_port=oxford_wwtw, preference=1e10, name="sewer_to_wwtw"
)
sewer_overflow = Arc(
    in_port=combined_sewer,
    out_port=thames_mixer,
    preference=1e-10,
    name="sewer_overflow",
)

sewer_to_wwtw = Arc( in_port=combined_sewer, out_port=oxford_wwtw, preference=1e10, name="sewer_to_wwtw" ) sewer_overflow = Arc( in_port=combined_sewer, out_port=thames_mixer, preference=1e-10, name="sewer_overflow", )

Create arcs¶

Arcs are a bit less interesting than nodes because they generally don't capture complicated physical behaviours. So we just create all of them below.

In [43]:

evenlode_to_thames = Arc(
    in_port=evenlode, out_port=evenlode_thames, name="evenlode_to_thames"
)

thames_to_thames = Arc(
    in_port=thames, out_port=evenlode_thames, name="thames_to_thames"
)

ray_to_cherwell = Arc(in_port=ray, out_port=cherwell_ray, name="ray_to_cherwell")

cherwell_to_cherwell = Arc(
    in_port=cherwell, out_port=cherwell_ray, name="cherwell_to_cherwell"
)

thames_to_farmoor = Arc(
    in_port=evenlode_thames, out_port=farmoor_abstraction, name="thames_to_farmoor"
)

farmoor_to_mixer = Arc(
    in_port=farmoor_abstraction, out_port=thames_mixer, name="farmoor_to_mixer"
)

cherwell_to_mixer = Arc(
    in_port=cherwell_ray, out_port=thames_mixer, name="cherwell_to_mixer"
)

wwtw_to_mixer = Arc(in_port=oxford_wwtw, out_port=thames_mixer, name="wwtw_to_mixer")

mixer_to_waste = Arc(
    in_port=thames_mixer, out_port=thames_above_abingdon, name="mixer_to_waste"
)

distribution_to_demand = Arc(
    in_port=distribution, out_port=oxford, name="distribution_to_demand"
)

reservoir_to_fwtw = Arc(in_port=farmoor, out_port=oxford_fwtw, name="reservoir_to_fwtw")

fwtw_to_sewer = Arc(in_port=oxford_fwtw, out_port=combined_sewer, name="fwtw_to_sewer")

demand_to_sewer = Arc(in_port=oxford, out_port=combined_sewer, name="demand_to_sewer")

land_to_sewer = Arc(in_port=oxford_land, out_port=combined_sewer, name="land_to_sewer")

land_to_gw = Arc(in_port=oxford_land, out_port=gw, name="land_to_gw")

garden_to_gw = Arc(in_port=oxford, out_port=gw, name="garden_to_gw")

gw_to_mixer = Arc(in_port=gw, out_port=thames_mixer, name="gw_to_mixer")

evenlode_to_thames = Arc( in_port=evenlode, out_port=evenlode_thames, name="evenlode_to_thames" ) thames_to_thames = Arc( in_port=thames, out_port=evenlode_thames, name="thames_to_thames" ) ray_to_cherwell = Arc(in_port=ray, out_port=cherwell_ray, name="ray_to_cherwell") cherwell_to_cherwell = Arc( in_port=cherwell, out_port=cherwell_ray, name="cherwell_to_cherwell" ) thames_to_farmoor = Arc( in_port=evenlode_thames, out_port=farmoor_abstraction, name="thames_to_farmoor" ) farmoor_to_mixer = Arc( in_port=farmoor_abstraction, out_port=thames_mixer, name="farmoor_to_mixer" ) cherwell_to_mixer = Arc( in_port=cherwell_ray, out_port=thames_mixer, name="cherwell_to_mixer" ) wwtw_to_mixer = Arc(in_port=oxford_wwtw, out_port=thames_mixer, name="wwtw_to_mixer") mixer_to_waste = Arc( in_port=thames_mixer, out_port=thames_above_abingdon, name="mixer_to_waste" ) distribution_to_demand = Arc( in_port=distribution, out_port=oxford, name="distribution_to_demand" ) reservoir_to_fwtw = Arc(in_port=farmoor, out_port=oxford_fwtw, name="reservoir_to_fwtw") fwtw_to_sewer = Arc(in_port=oxford_fwtw, out_port=combined_sewer, name="fwtw_to_sewer") demand_to_sewer = Arc(in_port=oxford, out_port=combined_sewer, name="demand_to_sewer") land_to_sewer = Arc(in_port=oxford_land, out_port=combined_sewer, name="land_to_sewer") land_to_gw = Arc(in_port=oxford_land, out_port=gw, name="land_to_gw") garden_to_gw = Arc(in_port=oxford, out_port=gw, name="garden_to_gw") gw_to_mixer = Arc(in_port=gw, out_port=thames_mixer, name="gw_to_mixer")

Again, we keep all the arcs in a tidy list together.

In [44]:

arclist = [
    evenlode_to_thames,
    thames_to_thames,
    ray_to_cherwell,
    cherwell_to_cherwell,
    thames_to_farmoor,
    farmoor_to_mixer,
    cherwell_to_mixer,
    wwtw_to_mixer,
    sewer_overflow,
    mixer_to_waste,
    abstraction_to_farmoor,
    distribution_to_demand,
    demand_to_sewer,
    land_to_sewer,
    sewer_to_wwtw,
    fwtw_to_sewer,
    fwtw_to_distribution,
    reservoir_to_fwtw,
    land_to_gw,
    garden_to_gw,
    gw_to_mixer,
]

arclist = [ evenlode_to_thames, thames_to_thames, ray_to_cherwell, cherwell_to_cherwell, thames_to_farmoor, farmoor_to_mixer, cherwell_to_mixer, wwtw_to_mixer, sewer_overflow, mixer_to_waste, abstraction_to_farmoor, distribution_to_demand, demand_to_sewer, land_to_sewer, sewer_to_wwtw, fwtw_to_sewer, fwtw_to_distribution, reservoir_to_fwtw, land_to_gw, garden_to_gw, gw_to_mixer, ]

Mapping¶

Remember, WSIMOD is an integrated model. Because it covers so many different things, it is very easy to make mistakes. Thus it is always good practice to plot your data!

Below we load the node location data and create arcs from the information in the arclist.

In [45]:

location_fn = os.path.join(data_folder, "raw", "points_locations.geojson")
nodes_gdf = gpd.read_file(location_fn).set_index("name")
arcs_gdf = []

for arc in arclist:
    arcs_gdf.append(
        {
            "name": arc.name,
            "geometry": LineString(
                [
                    nodes_gdf.loc[arc.in_port.name, "geometry"],
                    nodes_gdf.loc[arc.out_port.name, "geometry"],
                ]
            ),
        }
    )

arcs_gdf = gpd.GeoDataFrame(arcs_gdf, crs=nodes_gdf.crs)

location_fn = os.path.join(data_folder, "raw", "points_locations.geojson") nodes_gdf = gpd.read_file(location_fn).set_index("name") arcs_gdf = [] for arc in arclist: arcs_gdf.append( { "name": arc.name, "geometry": LineString( [ nodes_gdf.loc[arc.in_port.name, "geometry"], nodes_gdf.loc[arc.out_port.name, "geometry"], ] ), } ) arcs_gdf = gpd.GeoDataFrame(arcs_gdf, crs=nodes_gdf.crs)

Because we converted the information as GeoDataFrames, we can simply plot them below

In [46]:

f, ax = plt.subplots()
arcs_gdf.plot(ax=ax)
nodes_gdf.plot(color="r", ax=ax, zorder=10)

f, ax = plt.subplots() arcs_gdf.plot(ax=ax) nodes_gdf.plot(color="r", ax=ax, zorder=10)

Out[46]:

<Axes: >

Orchestration¶

Orchestration is making the simulation happen by calling functions in the nodes. These functions simulate physical behaviour within the node, and cause pulls/pushes to happen which in turn triggers physical behaviour in other nodes.

Orchestrating an individual timestep¶

We will start below by manually orchestrating a single timestep.

We start by setting the date, so that every node knows what forcing data to read for this timestep.

In [47]:

date = dates[0]

for node in nodelist:
    node.t = date

print(date)
print(oxford_fwtw.t)

date = dates[0] for node in nodelist: node.t = date print(date) print(oxford_fwtw.t)

2009-03-03 00:00:00
2009-03-03 00:00:00

We can see the service reservoirs are empty but the supply reservoir is not!

In [48]:

print(oxford_fwtw.service_reservoir_tank.storage)
print(farmoor.tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage) print(farmoor.tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 0.0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 10000000.0}

If we call the FWTW's treat_water function it will pull water from the supply reservoir and update its service reservoirs

In [49]:

oxford_fwtw.treat_water()

print(oxford_fwtw.service_reservoir_tank.storage)
print(farmoor.tank.storage)

oxford_fwtw.treat_water() print(oxford_fwtw.service_reservoir_tank.storage) print(farmoor.tank.storage)

{'temperature': 0.0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 43641.0}
{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 9955000.0}

This information is tracked in the arcs that enter the FWTW

In [50]:

print(oxford_fwtw.in_arcs)

print(oxford_fwtw.in_arcs)

{'reservoir_to_fwtw': <wsimod.arcs.arcs.Arc object at 0x00000201EBF041D0>}

In [51]:

print(reservoir_to_fwtw.flow_in)
print(reservoir_to_fwtw.flow_out)

print(reservoir_to_fwtw.flow_in) print(reservoir_to_fwtw.flow_out)

45000.0
45000.0

Although none of that water has yet entered the distribution network (only some small flow from the earlier demonstration)

In [52]:

print(fwtw_to_distribution.flow_in)

print(fwtw_to_distribution.flow_in)

15.0

That is because no water consumption demand had yet been generated.

If we call the demand node's create_demand function we see that the distribution arc becomes utilised.

In [53]:

oxford.create_demand()
print(fwtw_to_distribution.flow_in)

oxford.create_demand() print(fwtw_to_distribution.flow_in)

30015.0

We also see that this gets pushed onwards into the sewer system

In [54]:

print(demand_to_sewer.flow_in)

print(demand_to_sewer.flow_in)

30000.0

Many nodes have functions intended to be called during orchestration. These functions are described in the documentation. For example, we see in the Land node API reference that the 'run' function is intended to be called from orchestration.

In [55]:

oxford_land.run()

oxford_land.run()

Below we call the functions for other nodes

In [56]:

# Discharge GW
gw.distribute()

# Discharge sewers (pushed to other sewers or WWTW)
combined_sewer.make_discharge()

# Run WWTW model
oxford_wwtw.calculate_discharge()

# Make abstractions
farmoor.make_abstractions()

# Discharge WW
oxford_wwtw.make_discharge()

# Route catchments
evenlode.route()
thames.route()
ray.route()
cherwell.route()

# Discharge GW gw.distribute() # Discharge sewers (pushed to other sewers or WWTW) combined_sewer.make_discharge() # Run WWTW model oxford_wwtw.calculate_discharge() # Make abstractions farmoor.make_abstractions() # Discharge WW oxford_wwtw.make_discharge() # Route catchments evenlode.route() thames.route() ray.route() cherwell.route()

Ending a timestep¶

Because mistakes happen, it is essential to carry out mass balance testing. Each node has a mass balance function that can be called. We see a mass balance violation resulting from the demonstration with the FWTW earlier.

In [57]:

for node in nodelist:
    in_, ds_, out_ = node.node_mass_balance()

for node in nodelist: in_, ds_, out_ = node.node_mass_balance()

mass balance error for volume of 15.0 in oxford_distribution
mass balance error for volume of -15.0 in oxford_fwtw

We should also call the end_timestep function in nodes and arcs. This is important for mass balance testing and capturing the behaviour of some dynamic processes in nodes.

In [58]:

for node in nodelist:
    node.end_timestep()

for arc in arclist:
    arc.end_timestep()

for node in nodelist: node.end_timestep() for arc in arclist: arc.end_timestep()

Model object¶

Of course it would be a massive pain to manually orchestrate every timestep. So instead we store node and arc information in a model object that will do the orchestration for us.

Because we have already created the nodes/arcs above, we simply need to add the instantiated lists above.

In [59]:

my_model = Model()
my_model.add_instantiated_nodes(nodelist)
my_model.add_instantiated_arcs(arclist)
my_model.dates = dates

my_model = Model() my_model.add_instantiated_nodes(nodelist) my_model.add_instantiated_arcs(arclist) my_model.dates = dates

The model object lets us reinitialise the nodes/arcs, and run all of the orchestration with a 'run' function.

In [60]:

my_model.reinit()

flows, _, _, _ = my_model.run()

my_model.reinit() flows, _, _, _ = my_model.run()

  0%|                                                                                                                                                                                                                      | 0/1456 [00:00<?, ?it/s]

  3%|██████▎                                                                                                                                                                                                     | 45/1456 [00:00<00:03, 447.00it/s]

  6%|████████████▌                                                                                                                                                                                               | 90/1456 [00:00<00:03, 414.78it/s]

  9%|██████████████████▊                                                                                                                                                                                        | 135/1456 [00:00<00:03, 428.78it/s]

 12%|████████████████████████▉                                                                                                                                                                                  | 179/1456 [00:00<00:03, 420.75it/s]

 15%|███████████████████████████████                                                                                                                                                                            | 223/1456 [00:00<00:02, 426.45it/s]

 18%|█████████████████████████████████████                                                                                                                                                                      | 266/1456 [00:00<00:02, 426.80it/s]

 21%|███████████████████████████████████████████                                                                                                                                                                | 309/1456 [00:00<00:02, 426.81it/s]

 24%|█████████████████████████████████████████████████▏                                                                                                                                                         | 353/1456 [00:00<00:02, 428.71it/s]

 27%|███████████████████████████████████████████████████████▎                                                                                                                                                   | 397/1456 [00:00<00:02, 430.00it/s]

 30%|█████████████████████████████████████████████████████████████▌                                                                                                                                             | 442/1456 [00:01<00:02, 433.90it/s]

 33%|███████████████████████████████████████████████████████████████████▊                                                                                                                                       | 486/1456 [00:01<00:02, 432.27it/s]

 36%|█████████████████████████████████████████████████████████████████████████▉                                                                                                                                 | 530/1456 [00:01<00:02, 431.54it/s]

 39%|████████████████████████████████████████████████████████████████████████████████                                                                                                                           | 574/1456 [00:01<00:02, 433.20it/s]

 42%|██████████████████████████████████████████████████████████████████████████████████████▏                                                                                                                    | 618/1456 [00:01<00:01, 431.80it/s]

 45%|████████████████████████████████████████████████████████████████████████████████████████████▎                                                                                                              | 662/1456 [00:01<00:01, 434.10it/s]

 48%|██████████████████████████████████████████████████████████████████████████████████████████████████▍                                                                                                        | 706/1456 [00:01<00:01, 435.00it/s]

 52%|████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                                                                  | 750/1456 [00:01<00:01, 435.63it/s]

 55%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████▊                                                                                            | 795/1456 [00:01<00:01, 437.65it/s]

 58%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                                                                                      | 840/1456 [00:01<00:01, 437.87it/s]

 61%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                                                                               | 884/1456 [00:02<00:01, 436.35it/s]

 64%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍                                                                         | 928/1456 [00:02<00:01, 436.97it/s]

 67%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋                                                                   | 973/1456 [00:02<00:01, 438.78it/s]

 70%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                                                             | 1017/1456 [00:02<00:01, 437.90it/s]

 73%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                                                      | 1061/1456 [00:02<00:00, 437.66it/s]

 76%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎                                                | 1105/1456 [00:02<00:00, 437.49it/s]

 79%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍                                          | 1149/1456 [00:02<00:00, 434.79it/s]

 82%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                    | 1193/1456 [00:02<00:00, 433.52it/s]

 85%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                              | 1237/1456 [00:02<00:00, 433.30it/s]

 88%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋                        | 1281/1456 [00:02<00:00, 435.00it/s]

 91%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊                  | 1325/1456 [00:03<00:00, 434.24it/s]

 94%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉            | 1369/1456 [00:03<00:00, 435.57it/s]

 97%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████      | 1413/1456 [00:03<00:00, 432.18it/s]

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1456/1456 [00:03<00:00, 433.20it/s]

The model outputs flows as a dictionary which can be converted to a dataframe

In [61]:

flows = pd.DataFrame(flows)

print(flows.sample(10))

flows = pd.DataFrame(flows) print(flows.sample(10))

                          arc          flow       time  temperature  \
25112       reservoir_to_fwtw  3.093421e+04 2012-06-10    10.759018   
11285          sewer_overflow  2.448684e+05 2010-08-22    17.287603   
25028       reservoir_to_fwtw  3.093421e+04 2012-06-06    10.720785   
1473     cherwell_to_cherwell  1.347840e+05 2009-05-12    14.314286   
21685           land_to_sewer  4.300000e+04 2011-12-30     6.696667   
15657         demand_to_sewer  3.000000e+04 2011-03-18    13.317143   
30515         ray_to_cherwell  2.298240e+05 2013-02-23     4.114286   
25967  distribution_to_demand  3.000000e+04 2012-07-21    11.208837   
14412       cherwell_to_mixer  1.706400e+06 2011-01-18     6.531392   
9935          ray_to_cherwell  2.151360e+04 2010-06-19    17.471429   

            silicon    fluoride      chloride       nitrate       sulphate  \
25112    169.871678    4.210554   1121.164343    173.093150    1651.523664   
11285    589.586882   12.246788   4877.585325   1537.259480    6906.730862   
25028    169.254398    4.213290   1124.629190    173.207992    1655.912948   
1473     457.379877   26.956800   6758.454857    825.829087    8664.685714   
21685      0.070000    0.200000      0.060000      0.020000      70.000000   
15657    600.000000   12.000000   5400.000000   1800.000000    7500.000000   
30515   1483.198733   35.130240   7823.537280   1520.054898   15858.840960   
25967      1.795152    0.042601     10.875994      1.709112      16.157880   
14412  16475.805216  341.280000  58135.233600  12130.501490  104067.374400   
9935     138.314622    3.257774   1474.941682    182.265495    2212.397157   

           nitrogen        sodium    potassium        calcium    magnesium  \
25112    187.907787    733.535019   144.797149    3076.430237   148.347166   
11285   1308.729667   5123.814419   794.607997    5582.810155   808.598123   
25028    188.107437    736.508938   145.098907    3077.238049   148.544941   
1473     883.797943   5121.792000   870.319543   15598.359771  1253.491200   
21685      4.000000      0.300000     7.000000      70.000000     6.000000   
15657   1500.000000   6000.000000   900.000000    4500.000000   900.000000   
30515   1583.815680   5079.110400  1119.571200   27175.046400  1162.252800   
25967      1.848418      7.094495     1.424015      31.179492     1.485822   
14412  15153.091200  33774.192000  8168.256000  163194.048000  8809.776000   
9935     195.988896   1164.500434   249.865097    2349.592457   119.861486   

           boron  
25112   1.508747  
11285  12.862486  
25028   1.509840  
1473   11.591424  
21685   0.100000  
15657  15.000000  
30515  13.257562  
25967   0.015263  
14412  97.575840  
9935    2.603146  

Validation plots¶

Of course we shouldn't trust a model without proof, and this is doubly true for an integrated model whose errors may easily propagate.

Thankfully we have rich in-river water quality sampling in Oxford that we can use for validation

We load and format this data for dates and pollutants that overlap with what we have simulated

In [62]:

mixer_val_df = pd.read_csv(os.path.join(data_folder, "raw", "mixer_wims_val.csv"))
mixer_val_df.date = pd.to_datetime(mixer_val_df.date)
mixer_val_df = mixer_val_df.loc[mixer_val_df.date.isin(dates)]
val_pollutants = mixer_val_df.variable.unique()
mixer_val_df = mixer_val_df.pivot(index="date", columns="variable", values="value")

mixer_val_df = pd.read_csv(os.path.join(data_folder, "raw", "mixer_wims_val.csv")) mixer_val_df.date = pd.to_datetime(mixer_val_df.date) mixer_val_df = mixer_val_df.loc[mixer_val_df.date.isin(dates)] val_pollutants = mixer_val_df.variable.unique() mixer_val_df = mixer_val_df.pivot(index="date", columns="variable", values="value")

We get rid of the first day of flows because our tanks were initialised empty and will not be informative

In [63]:

flows = flows.loc[flows.time != dates[0]]

flows = flows.loc[flows.time != dates[0]]

We convert our flows, which are simulated in kg/d into a concentration value (kg/m3/d).

In [64]:

flows_plot = flows.copy()

# Convert to KG/M3
for pol in set(val_pollutants):
    if pol != "temperature":
        flows_plot[pol] /= flows_plot.flow

flows_plot = flows.copy() # Convert to KG/M3 for pol in set(val_pollutants): if pol != "temperature": flows_plot[pol] /= flows_plot.flow

We pick the model outlet as a validation location and make a pretty plot - fantastic!

In [65]:

plot_arc = "mixer_to_waste"
f, axs = plt.subplots(val_pollutants.size, 1)
for pol, ax in zip(val_pollutants, axs):
    ax.plot(
        flows_plot.loc[flows_plot.arc == plot_arc, [pol, "time"]].set_index("time"),
        color="b",
        label="simulation",
    )
    ax.plot(mixer_val_df[pol], ls="", marker="o", color="r", label="spot sample")
    ax.set_ylabel(pol)
plt.legend()

plot_arc = "mixer_to_waste" f, axs = plt.subplots(val_pollutants.size, 1) for pol, ax in zip(val_pollutants, axs): ax.plot( flows_plot.loc[flows_plot.arc == plot_arc, [pol, "time"]].set_index("time"), color="b", label="simulation", ) ax.plot(mixer_val_df[pol], ls="", marker="o", color="r", label="spot sample") ax.set_ylabel(pol) plt.legend()

Out[65]:

<matplotlib.legend.Legend at 0x201ebfc33d0>