WSIMOD model demonstration - Oxford (.py)¶

Note - this script can also be opened in interactive Python if you wanted to play around. On the GitHub it is in docs/demo/scripts

1. Introduction

2. Data

3. Nodes

   3.1 Freshwater Treatment Works

   3.2 Land

   3.3 Demand

   3.4 Reservoir

   3.5 Distribution

   3.6 Wastewater Treatment Works

   3.7 Sewers

   3.8 Groundwater

   3.9 Node list

4. Arcs

   4.1 Arc parameters

   4.2 Create the arcs

5. Mapping

6. Orchestration

   6.1 Orchestrating an individual timestep

   6.2 Ending a timestep

7. Model object

   7.1 Validation

We will cover a demo WSIMOD case study¶

The glamorous town of Oxford will be our demo case study. Below, we will create these nodes and arcs, orchestrate them into a model, and run simulations.

Although GIS is pretty, the schematic below is a more accurate representation of what will be created. WSIMOD treats everything as a node or an arc.

Imports and forcing data¶

Import packages

In [1]:

import os

import pandas as pd

from wsimod.arcs.arcs import Arc
from wsimod.core import constants
from wsimod.nodes.catchment import Catchment
from wsimod.nodes.demand import ResidentialDemand
from wsimod.nodes.land import Land
from wsimod.nodes.nodes import Node
from wsimod.nodes.sewer import Sewer
from wsimod.nodes.storage import Groundwater, Reservoir
from wsimod.nodes.waste import Waste
from wsimod.nodes.wtw import FWTW, WWTW
from wsimod.orchestration.model import Model

os.environ["USE_PYGEOS"] = "0"
import geopandas as gpd
from matplotlib import pyplot as plt
from shapely.geometry import LineString

import os import pandas as pd from wsimod.arcs.arcs import Arc from wsimod.core import constants from wsimod.nodes.catchment import Catchment from wsimod.nodes.demand import ResidentialDemand from wsimod.nodes.land import Land from wsimod.nodes.nodes import Node from wsimod.nodes.sewer import Sewer from wsimod.nodes.storage import Groundwater, Reservoir from wsimod.nodes.waste import Waste from wsimod.nodes.wtw import FWTW, WWTW from wsimod.orchestration.model import Model os.environ["USE_PYGEOS"] = "0" import geopandas as gpd from matplotlib import pyplot as plt from shapely.geometry import LineString

Load input data

In [2]:

# Select the root path for the data folder. Use the appropriate value for your case.
data_folder = os.path.join(os.path.abspath(""), "docs", "demo", "data")

input_fid = os.path.join(data_folder, "processed", "timeseries_data.csv")
input_data = pd.read_csv(input_fid)
input_data.loc[input_data.variable == "flow", "value"] *= constants.M3_S_TO_M3_DT
input_data.loc[input_data.variable == "precipitation", "value"] *= constants.MM_TO_M
input_data.date = pd.to_datetime(input_data.date)
data_input_dict = input_data.set_index(["variable", "date"]).value.to_dict()
data_input_dict = (
    input_data.groupby("site")
    .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict())
    .to_dict()
)
print(input_data.sample(10))

# Select the root path for the data folder. Use the appropriate value for your case. data_folder = os.path.join(os.path.abspath(""), "docs", "demo", "data") input_fid = os.path.join(data_folder, "processed", "timeseries_data.csv") input_data = pd.read_csv(input_fid) input_data.loc[input_data.variable == "flow", "value"] *= constants.M3_S_TO_M3_DT input_data.loc[input_data.variable == "precipitation", "value"] *= constants.MM_TO_M input_data.date = pd.to_datetime(input_data.date) data_input_dict = input_data.set_index(["variable", "date"]).value.to_dict() data_input_dict = ( input_data.groupby("site") .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict()) .to_dict() ) print(input_data.sample(10))

              site       date   variable     value
40226       thames 2011-09-03   nitrogen  0.006291
27881       thames 2009-10-06    nitrate  0.005799
65696     evenlode 2009-08-26      boron  0.000061
77511  oxford_land 2010-02-09        et0  0.002000
29458     cherwell 2010-02-04   sulphate  0.057974
38208          ray 2010-02-18   nitrogen  0.007623
49888          ray 2010-03-22  potassium  0.006871
62446          ray 2012-09-17  magnesium  0.004829
37491     evenlode 2012-02-27   nitrogen  0.006310
56626          ray 2012-09-21    calcium  0.119300

/tmp/ipykernel_2132/946595242.py:12: DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.
  .apply(lambda x: x.set_index(["variable", "date"]).value.to_dict())

Input data is stored in dicts

In [3]:

print(data_input_dict["cherwell"][("boron", pd.to_datetime("2010-11-20"))])

print(data_input_dict["cherwell"][("boron", pd.to_datetime("2010-11-20"))])

6.985714285714286e-05

We select dates that are available in the input data

In [4]:

dates = input_data.date.unique()
dates = dates[dates.argsort()]
dates = [pd.Timestamp(x) for x in dates]
print(dates[0:10])

dates = input_data.date.unique() dates = dates[dates.argsort()] dates = [pd.Timestamp(x) for x in dates] print(dates[0:10])

[Timestamp('2009-03-03 00:00:00'), Timestamp('2009-03-04 00:00:00'), Timestamp('2009-03-05 00:00:00'), Timestamp('2009-03-06 00:00:00'), Timestamp('2009-03-07 00:00:00'), Timestamp('2009-03-08 00:00:00'), Timestamp('2009-03-09 00:00:00'), Timestamp('2009-03-10 00:00:00'), Timestamp('2009-03-11 00:00:00'), Timestamp('2009-03-12 00:00:00')]

We can specify the pollutants. In this example we choose based on what pollutants we have input data for.

In [5]:

constants.POLLUTANTS = input_data.variable.unique().tolist()
constants.POLLUTANTS.remove("flow")
constants.POLLUTANTS.remove("precipitation")
constants.POLLUTANTS.remove("et0")
constants.NON_ADDITIVE_POLLUTANTS = ["temperature"]
constants.ADDITIVE_POLLUTANTS = list(
    set(constants.POLLUTANTS).difference(constants.NON_ADDITIVE_POLLUTANTS)
)
constants.FLOAT_ACCURACY = 1e-11
print(constants.POLLUTANTS)

constants.POLLUTANTS = input_data.variable.unique().tolist() constants.POLLUTANTS.remove("flow") constants.POLLUTANTS.remove("precipitation") constants.POLLUTANTS.remove("et0") constants.NON_ADDITIVE_POLLUTANTS = ["temperature"] constants.ADDITIVE_POLLUTANTS = list( set(constants.POLLUTANTS).difference(constants.NON_ADDITIVE_POLLUTANTS) ) constants.FLOAT_ACCURACY = 1e-11 print(constants.POLLUTANTS)

['temperature', 'silicon', 'fluoride', 'chloride', 'nitrate', 'sulphate', 'nitrogen', 'sodium', 'potassium', 'calcium', 'magnesium', 'boron']

Create nodes¶

For waste nodes, no parameters are needed, they are just the model outlet

In [6]:

thames_above_abingdon = Waste(name="thames_above_abingdon")

thames_above_abingdon = Waste(name="thames_above_abingdon")

For junctions and abstraction locations, we can simply use the default nodes

In [7]:

farmoor_abstraction = Node(name="farmoor_abstraction")
evenlode_thames = Node(name="evenlode_thames")
cherwell_ray = Node(name="cherwell_ray")
cherwell_thames = Node(name="cherwell_thames")
thames_mixer = Node(name="thames_mixer")

farmoor_abstraction = Node(name="farmoor_abstraction") evenlode_thames = Node(name="evenlode_thames") cherwell_ray = Node(name="cherwell_ray") cherwell_thames = Node(name="cherwell_thames") thames_mixer = Node(name="thames_mixer")

For catchment nodes, we only need to specify the input data (as a dictionary format).

In [8]:

evenlode = Catchment(name="evenlode", data_input_dict=data_input_dict["evenlode"])
thames = Catchment(name="thames", data_input_dict=data_input_dict["thames"])
ray = Catchment(name="ray", data_input_dict=data_input_dict["ray"])
cherwell = Catchment(name="cherwell", data_input_dict=data_input_dict["cherwell"])

evenlode = Catchment(name="evenlode", data_input_dict=data_input_dict["evenlode"]) thames = Catchment(name="thames", data_input_dict=data_input_dict["thames"]) ray = Catchment(name="ray", data_input_dict=data_input_dict["ray"]) cherwell = Catchment(name="cherwell", data_input_dict=data_input_dict["cherwell"])

We can see that, even though we provided mimimal information (name and input data) each node comes with many predefined functions.

In [9]:

print(dir(evenlode))

print(dir(evenlode))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'apply_overrides', 'blend_vqip', 'check_basic', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'data_input_dict', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'end_timestep_', 'extract_vqip', 'extract_vqip_c', 'get_avail', 'get_connected', 'get_data_input', 'get_direction_arcs', 'get_flow', 'in_arcs', 'in_arcs_type', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'node_mass_balance', 'out_arcs', 'out_arcs_type', 'pull_check', 'pull_check_abstraction', 'pull_check_basic', 'pull_check_deny', 'pull_check_handler', 'pull_distributed', 'pull_set', 'pull_set_abstraction', 'pull_set_deny', 'pull_set_handler', 'push_check', 'push_check_accept', 'push_check_basic', 'push_check_deny', 'push_check_handler', 'push_distributed', 'push_set', 'push_set_deny', 'push_set_handler', 'query_handler', 'reinit', 'route', 'sum_vqip', 't', 'total_in', 'total_out', 'total_to_concentration', 'unrouted_water', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c']

Freshwater treatment works¶

Each type of node uses different parameters (see API reference). Below we create a freshwater treatment works (FWTW)

In [10]:

oxford_fwtw = FWTW(
    service_reservoir_storage_capacity=1e5,
    service_reservoir_storage_area=2e4,
    treatment_throughput_capacity=4.5e4,
    name="oxford_fwtw",
)

oxford_fwtw = FWTW( service_reservoir_storage_capacity=1e5, service_reservoir_storage_area=2e4, treatment_throughput_capacity=4.5e4, name="oxford_fwtw", )

Each node type has different types of functionality available

In [11]:

print(dir(oxford_fwtw))

print(dir(oxford_fwtw))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_liquor_multiplier', '_percent_solids', 'apply_overrides', 'blend_vqip', 'calculate_volume', 'check_basic', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'current_input', 'data_input_dict', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'end_timestep_', 'extract_vqip', 'extract_vqip_c', 'get_connected', 'get_data_input', 'get_direction_arcs', 'get_excess_throughput', 'in_arcs', 'in_arcs_type', 'liquor', 'liquor_multiplier', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'node_mass_balance', 'out_arcs', 'out_arcs_type', 'percent_solids', 'previous_pulled', 'process_parameters', 'pull_check', 'pull_check_basic', 'pull_check_deny', 'pull_check_fwtw', 'pull_check_handler', 'pull_distributed', 'pull_set', 'pull_set_deny', 'pull_set_fwtw', 'pull_set_handler', 'push_check', 'push_check_accept', 'push_check_basic', 'push_check_deny', 'push_check_handler', 'push_distributed', 'push_set', 'push_set_deny', 'push_set_handler', 'query_handler', 'reinit', 'service_reservoir_initial_storage', 'service_reservoir_storage_area', 'service_reservoir_storage_capacity', 'service_reservoir_storage_elevation', 'service_reservoir_tank', 'solids', 'sum_vqip', 't', 'total_deficit', 'total_in', 'total_out', 'total_pulled', 'total_to_concentration', 'treat_current_input', 'treat_water', 'treated', 'treatment_throughput_capacity', 'unpushed_sludge', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c']

The FWTW node has a tank representing the service reservoirs, we can see that it has been initialised empty.

In [12]:

print(oxford_fwtw.service_reservoir_tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

If we try to pull water from the FWTW, it responds that there is no water to pull.

In [13]:

print(oxford_fwtw.pull_check({"volume": 10}))

print(oxford_fwtw.pull_check({"volume": 10}))

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

If we add in some water, we see the pull check responds that water is available.

In [14]:

oxford_fwtw.service_reservoir_tank.storage["volume"] += 25
print(oxford_fwtw.pull_check({"volume": 10}))

oxford_fwtw.service_reservoir_tank.storage["volume"] += 25 print(oxford_fwtw.pull_check({"volume": 10}))

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 10.0}

When we set a pull request, we see that we successfully receive the water and the tank is updated.

In [15]:

reply = oxford_fwtw.pull_set({"volume": 10})
print(reply)
print(oxford_fwtw.service_reservoir_tank.storage)

reply = oxford_fwtw.pull_set({"volume": 10}) print(reply) print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 10.0}
{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

Land¶

We will now create a land node, it is a bit involved so you might want to skip ahead to demand, or check out the land node tutorial

Data inputs are a single dictionary

In [16]:

land_inputs = data_input_dict["oxford_land"]

print(land_inputs[("precipitation", pd.to_datetime("2010-11-20"))])
print(land_inputs[("et0", pd.to_datetime("2010-11-20"))])
print(land_inputs[("temperature", pd.to_datetime("2009-10-15"))])

land_inputs = data_input_dict["oxford_land"] print(land_inputs[("precipitation", pd.to_datetime("2010-11-20"))]) print(land_inputs[("et0", pd.to_datetime("2010-11-20"))]) print(land_inputs[("temperature", pd.to_datetime("2009-10-15"))])

0.0003
0.002
13.107142857142858

Assign some default pollutant deposition values (kg/m2/d)

In [17]:

pollutant_deposition = {
    "boron": 100e-10,
    "calcium": 70e-7,
    "chloride": 60e-10,
    "fluoride": 0.2e-7,
    "magnesium": 6e-7,
    "nitrate": 2e-9,
    "nitrogen": 4e-7,
    "potassium": 7e-7,
    "silicon": 7e-9,
    "sodium": 30e-9,
    "sulphate": 70e-7,
}

pollutant_deposition = { "boron": 100e-10, "calcium": 70e-7, "chloride": 60e-10, "fluoride": 0.2e-7, "magnesium": 6e-7, "nitrate": 2e-9, "nitrogen": 4e-7, "potassium": 7e-7, "silicon": 7e-9, "sodium": 30e-9, "sulphate": 70e-7, }

Create two surfaces as a list of dicts

In [18]:

surface = [
    {
        "type_": "PerviousSurface",
        "area": 2e7,
        "pollutant_load": pollutant_deposition,
        "surface": "rural",
        "field_capacity": 0.3,
        "depth": 0.5,
        "initial_storage": 2e7 * 0.4 * 0.5,
    },
    {
        "type_": "ImperviousSurface",
        "area": 1e7,
        "pollutant_load": pollutant_deposition,
        "surface": "urban",
        "initial_storage": 5e6,
    },
]

surface = [ { "type_": "PerviousSurface", "area": 2e7, "pollutant_load": pollutant_deposition, "surface": "rural", "field_capacity": 0.3, "depth": 0.5, "initial_storage": 2e7 * 0.4 * 0.5, }, { "type_": "ImperviousSurface", "area": 1e7, "pollutant_load": pollutant_deposition, "surface": "urban", "initial_storage": 5e6, }, ]

Create the land node from these surfaces and the input data

In [19]:

oxford_land = Land(surfaces=surface, name="oxford_land", data_input_dict=land_inputs)

oxford_land = Land(surfaces=surface, name="oxford_land", data_input_dict=land_inputs)

We can see the land node has various tanks that have been initialised empty

In [20]:

print(oxford_land.surface_runoff.storage)
print(oxford_land.subsurface_runoff.storage)
print(oxford_land.percolation.storage)

print(oxford_land.surface_runoff.storage) print(oxford_land.subsurface_runoff.storage) print(oxford_land.percolation.storage)

{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 0}

We can see the surfaces have also been initialised, although they are not empty because we provided 'initial_storage' parameters.

In [21]:

rural_surface = oxford_land.get_surface("rural")
urban_surface = oxford_land.get_surface("urban")
print("{0}-{1}".format("rural", rural_surface.storage))
print("{0}-{1}".format("urban", urban_surface.storage))

rural_surface = oxford_land.get_surface("rural") urban_surface = oxford_land.get_surface("urban") print("{0}-{1}".format("rural", rural_surface.storage)) print("{0}-{1}".format("urban", urban_surface.storage))

rural-{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 4000000.0}
urban-{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 5000000.0}

We can run a timestep of the land node with the 'run' command

In [22]:

oxford_land.t = pd.to_datetime("2012-12-22")

oxford_land.run()

oxford_land.t = pd.to_datetime("2012-12-22") oxford_land.run()

We can see that the land and surface tanks have been updated

In [23]:

print(oxford_land.surface_runoff.storage)
print(oxford_land.subsurface_runoff.storage)
print(oxford_land.percolation.storage)

print("{0}-{1}".format("rural", rural_surface.storage))
print("{0}-{1}".format("urban", urban_surface.storage))

print(oxford_land.surface_runoff.storage) print(oxford_land.subsurface_runoff.storage) print(oxford_land.percolation.storage) print("{0}-{1}".format("rural", rural_surface.storage)) print("{0}-{1}".format("urban", urban_surface.storage))

{'temperature': 3.521534008631931, 'silicon': 0.00020062235262918442, 'fluoride': 0.0005732067217976697, 'chloride': 0.0001719620165393009, 'nitrate': 5.732067217976698e-05, 'sulphate': 0.20062235262918443, 'nitrogen': 0.011464134435953396, 'sodium': 0.0008598100826965045, 'potassium': 0.020062235262918445, 'calcium': 0.20062235262918443, 'magnesium': 0.017196201653930095, 'boron': 0.00028660336089883487, 'volume': 12400.000000000002}
{'temperature': 0.9722493642718524, 'silicon': 0.0017554455855053634, 'fluoride': 0.0050155588157296096, 'chloride': 0.0015046676447188828, 'nitrate': 0.000501555881572961, 'sulphate': 1.7554455855053634, 'nitrogen': 0.1003111763145922, 'sodium': 0.007523338223594414, 'potassium': 0.17554455855053633, 'calcium': 1.7554455855053634, 'magnesium': 0.1504667644718883, 'boron': 0.0025077794078648048, 'volume': 58900.0}
{'temperature': 0.3349282031818326, 'silicon': 0.005868203814403644, 'fluoride': 0.016766296612581843, 'chloride': 0.005029888983774552, 'nitrate': 0.001676629661258184, 'sulphate': 5.868203814403645, 'nitrogen': 0.3353259322516368, 'sodium': 0.02514944491887276, 'potassium': 0.5868203814403645, 'calcium': 5.868203814403645, 'magnesium': 0.5029888983774553, 'boron': 0.008383148306290921, 'volume': 176700.00000000003}
rural-{'temperature': 6.203473168254872, 'silicon': 0.1321757282474618, 'fluoride': 0.3776449378498909, 'chloride': 0.11329348135496725, 'nitrate': 0.037764493784989084, 'sulphate': 132.17572824746182, 'nitrogen': 7.552898756997817, 'sodium': 0.5664674067748363, 'potassium': 13.21757282474618, 'calcium': 132.17572824746182, 'magnesium': 11.329348135496726, 'boron': 0.18882246892494545, 'volume': 3980000.0}
urban-{'temperature': 0.13343969249141666, 'silicon': 0.06999999999999999, 'fluoride': 0.2, 'chloride': 0.06, 'nitrate': 0.02, 'sulphate': 70.0, 'nitrogen': 4.0, 'sodium': 0.3, 'potassium': 7.0, 'calcium': 70.0, 'magnesium': 6.0, 'boron': 0.1, 'volume': 5104000.0}

Residential demand¶

The residential demand node requires population, per capita demand and a pollutant_load dictionary that defines how much (weight in kg) pollution is generated per person per day.

In [24]:

oxford = ResidentialDemand(
    name="oxford",
    population=2e5,
    per_capita=0.15,
    pollutant_load={
        "boron": 500 * constants.UG_L_TO_KG_M3 * 0.15,
        "calcium": 150 * constants.MG_L_TO_KG_M3 * 0.15,
        "chloride": 180 * constants.MG_L_TO_KG_M3 * 0.15,
        "fluoride": 0.4 * constants.MG_L_TO_KG_M3 * 0.15,
        "magnesium": 30 * constants.MG_L_TO_KG_M3 * 0.15,
        "nitrate": 60 * constants.MG_L_TO_KG_M3 * 0.15,
        "nitrogen": 50 * constants.MG_L_TO_KG_M3 * 0.15,
        "potassium": 30 * constants.MG_L_TO_KG_M3 * 0.15,
        "silicon": 20 * constants.MG_L_TO_KG_M3 * 0.15,
        "sodium": 200 * constants.MG_L_TO_KG_M3 * 0.15,
        "sulphate": 250 * constants.MG_L_TO_KG_M3 * 0.15,
        "temperature": 14,
    },
    data_input_dict=land_inputs,
)
# pollutant_load calculated based on expected effluent at WWTW

oxford = ResidentialDemand( name="oxford", population=2e5, per_capita=0.15, pollutant_load={ "boron": 500 * constants.UG_L_TO_KG_M3 * 0.15, "calcium": 150 * constants.MG_L_TO_KG_M3 * 0.15, "chloride": 180 * constants.MG_L_TO_KG_M3 * 0.15, "fluoride": 0.4 * constants.MG_L_TO_KG_M3 * 0.15, "magnesium": 30 * constants.MG_L_TO_KG_M3 * 0.15, "nitrate": 60 * constants.MG_L_TO_KG_M3 * 0.15, "nitrogen": 50 * constants.MG_L_TO_KG_M3 * 0.15, "potassium": 30 * constants.MG_L_TO_KG_M3 * 0.15, "silicon": 20 * constants.MG_L_TO_KG_M3 * 0.15, "sodium": 200 * constants.MG_L_TO_KG_M3 * 0.15, "sulphate": 250 * constants.MG_L_TO_KG_M3 * 0.15, "temperature": 14, }, data_input_dict=land_inputs, ) # pollutant_load calculated based on expected effluent at WWTW

Reservoir¶

A reservoir node is used to make abstractions from rivers and supply FWTWs

In [25]:

farmoor = Reservoir(
    name="farmoor", capacity=1e7, initial_storage=1e7, area=1.5e6, datum=62
)

farmoor = Reservoir( name="farmoor", capacity=1e7, initial_storage=1e7, area=1.5e6, datum=62 )

Distribution¶

We use a generic Node as a junction to represent the distribution network between the FWTW and households

In [26]:

distribution = Node(name="oxford_distribution")

distribution = Node(name="oxford_distribution")

Wastewater treatment works¶

Wastewater treatment works (WWTW) are nodes that can store sewage water temporarily in storm tanks, and reduce the pollution amounts in water before releasing them onwards to rivers.

In [27]:

oxford_wwtw = WWTW(
    stormwater_storage_capacity=2e4,
    stormwater_storage_area=2e4,
    treatment_throughput_capacity=5e4,
    name="oxford_wwtw",
)

oxford_wwtw = WWTW( stormwater_storage_capacity=2e4, stormwater_storage_area=2e4, treatment_throughput_capacity=5e4, name="oxford_wwtw", )

Sewers¶

Sewer nodes enable water to transition between households and WWTWs, and between impervious surfaces and rivers or WWTWs. They use a timearea diagram to represent travel time, which assigns a specified percentage of water to take a specified duration to pass through the sewer node.

In [28]:

combined_sewer = Sewer(
    capacity=4e6, pipe_timearea={0: 0.8, 1: 0.15, 2: 0.05}, name="combined_sewer"
)

combined_sewer = Sewer( capacity=4e6, pipe_timearea={0: 0.8, 1: 0.15, 2: 0.05}, name="combined_sewer" )

Groundwater¶

Groundwater nodes implement a simple residence time to determine baseflow

In [29]:

gw = Groundwater(capacity=3.2e9, area=3.2e8, name="gw", residence_time=20)

gw = Groundwater(capacity=3.2e9, area=3.2e8, name="gw", residence_time=20)

Create a nodelist¶

To keep all the nodes in one place, we put them into a list

In [30]:

nodelist = [
    thames_above_abingdon,
    evenlode,
    thames,
    ray,
    cherwell,
    oxford,
    distribution,
    farmoor,
    oxford_fwtw,
    oxford_wwtw,
    combined_sewer,
    oxford_land,
    gw,
    farmoor_abstraction,
    evenlode_thames,
    cherwell_ray,
    cherwell_thames,
    thames_mixer,
]

print(nodelist)

nodelist = [ thames_above_abingdon, evenlode, thames, ray, cherwell, oxford, distribution, farmoor, oxford_fwtw, oxford_wwtw, combined_sewer, oxford_land, gw, farmoor_abstraction, evenlode_thames, cherwell_ray, cherwell_thames, thames_mixer, ] print(nodelist)

[<wsimod.nodes.waste.Waste object at 0x7f85c5ee5a10>, <wsimod.nodes.catchment.Catchment object at 0x7f85c5ed36d0>, <wsimod.nodes.catchment.Catchment object at 0x7f85c5ed35d0>, <wsimod.nodes.catchment.Catchment object at 0x7f85c5ed2810>, <wsimod.nodes.catchment.Catchment object at 0x7f85c5ed1850>, <wsimod.nodes.demand.ResidentialDemand object at 0x7f85c5e71d50>, <wsimod.nodes.nodes.Node object at 0x7f85c5e72910>, <wsimod.nodes.storage.Reservoir object at 0x7f85c5e72b90>, <wsimod.nodes.wtw.FWTW object at 0x7f85c5ecaed0>, <wsimod.nodes.wtw.WWTW object at 0x7f85c5e5ffd0>, <wsimod.nodes.sewer.Sewer object at 0x7f85c65867d0>, <wsimod.nodes.land.Land object at 0x7f85c5ebd810>, <wsimod.nodes.storage.Groundwater object at 0x7f85c658c290>, <wsimod.nodes.nodes.Node object at 0x7f85c5ee4f50>, <wsimod.nodes.nodes.Node object at 0x7f85c5ee4250>, <wsimod.nodes.nodes.Node object at 0x7f85c5edea10>, <wsimod.nodes.nodes.Node object at 0x7f85c5eddc90>, <wsimod.nodes.nodes.Node object at 0x7f85c5edced0>]

Arcs¶

Arcs link nodes. An example arc is the link between a FWTW and the distribution node

In [31]:

# Standard simple arcs
fwtw_to_distribution = Arc(
    in_port=oxford_fwtw, out_port=distribution, name="fwtw_to_distribution"
)
print(fwtw_to_distribution)

# Standard simple arcs fwtw_to_distribution = Arc( in_port=oxford_fwtw, out_port=distribution, name="fwtw_to_distribution" ) print(fwtw_to_distribution)

<wsimod.arcs.arcs.Arc object at 0x7f85c6215410>

As with nodes, even though we only gave it a few parameters, the arc comes with a lot built in

In [32]:

print(dir(fwtw_to_distribution))

print(dir(fwtw_to_distribution))

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'apply_overrides', 'arc_mass_balance', 'blend_vqip', 'capacity', 'compare_vqip', 'concentration_to_total', 'copy_vqip', 'ds_vqip', 'ds_vqip_c', 'empty_vqip', 'empty_vqip_predefined', 'end_timestep', 'extract_vqip', 'extract_vqip_c', 'flow_in', 'flow_out', 'get_excess', 'in_port', 'mass_balance', 'mass_balance_ds', 'mass_balance_in', 'mass_balance_out', 'name', 'out_port', 'preference', 'reinit', 'send_pull_check', 'send_pull_request', 'send_push_check', 'send_push_request', 'sum_vqip', 'total_to_concentration', 'v_change_vqip', 'v_change_vqip_c', 'v_distill_vqip', 'v_distill_vqip_c', 'vqip_in', 'vqip_out']

We can see that the arc links the two nodes

In [33]:

print(fwtw_to_distribution.in_port)

print(fwtw_to_distribution.in_port)

<wsimod.nodes.wtw.FWTW object at 0x7f85c5ecaed0>

In [34]:

print(fwtw_to_distribution.out_port)

print(fwtw_to_distribution.out_port)

<wsimod.nodes.nodes.Node object at 0x7f85c5e72910>

And that it has updated the nodes that it is connecting.

In [35]:

print(oxford_fwtw.out_arcs)

print(oxford_fwtw.out_arcs)

{'fwtw_to_distribution': <wsimod.arcs.arcs.Arc object at 0x7f85c6215410>}

In [36]:

print(distribution.in_arcs)

print(distribution.in_arcs)

{'fwtw_to_distribution': <wsimod.arcs.arcs.Arc object at 0x7f85c6215410>}

We use arcs to send checks and requests..

In [37]:

print(fwtw_to_distribution.send_pull_check({"volume": 20}))

print(fwtw_to_distribution.send_pull_check({"volume": 20}))

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

In [38]:

reply = fwtw_to_distribution.send_pull_request({"volume": 20})
print(reply)

reply = fwtw_to_distribution.send_pull_request({"volume": 20}) print(reply)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 15.0}

They convey this information to the nodes that they connect to, which update their state variables

In [39]:

print(oxford_fwtw.service_reservoir_tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 0.0}

In turn, the arcs update their own state variables.

In [40]:

print(fwtw_to_distribution.flow_in)
print(fwtw_to_distribution.flow_out)

print(fwtw_to_distribution.flow_in) print(fwtw_to_distribution.flow_out)

15.0
15.0

Arc parameters¶

Besides the in/out ports and names, arcs can have a parameter for their capacity, to limit the flow that may pass through it each timestep. A typical example would be on river abstractions to a reservoir

In [41]:

abstraction_to_farmoor = Arc(
    in_port=farmoor_abstraction,
    out_port=farmoor,
    name="abstraction_to_farmoor",
    capacity=5e4,
)

abstraction_to_farmoor = Arc( in_port=farmoor_abstraction, out_port=farmoor, name="abstraction_to_farmoor", capacity=5e4, )

A bit more sophisticated is the 'preference' parameter. We use preference to express where we would prefer the model to send water. In this example, the sewer can send water to both the treatment plant and directly into the river. Of course we would always to prefer to send water to the plant, so we give it a very high preference. Discharging into the river should only be done if there is no capacity left at the WWTW, so we give the arc a very low preference.

In [42]:

sewer_to_wwtw = Arc(
    in_port=combined_sewer, out_port=oxford_wwtw, preference=1e10, name="sewer_to_wwtw"
)
sewer_overflow = Arc(
    in_port=combined_sewer,
    out_port=thames_mixer,
    preference=1e-10,
    name="sewer_overflow",
)

sewer_to_wwtw = Arc( in_port=combined_sewer, out_port=oxford_wwtw, preference=1e10, name="sewer_to_wwtw" ) sewer_overflow = Arc( in_port=combined_sewer, out_port=thames_mixer, preference=1e-10, name="sewer_overflow", )

Create arcs¶

Arcs are a bit less interesting than nodes because they generally don't capture complicated physical behaviours. So we just create all of them below.

In [43]:

evenlode_to_thames = Arc(
    in_port=evenlode, out_port=evenlode_thames, name="evenlode_to_thames"
)

thames_to_thames = Arc(
    in_port=thames, out_port=evenlode_thames, name="thames_to_thames"
)

ray_to_cherwell = Arc(in_port=ray, out_port=cherwell_ray, name="ray_to_cherwell")

cherwell_to_cherwell = Arc(
    in_port=cherwell, out_port=cherwell_ray, name="cherwell_to_cherwell"
)

thames_to_farmoor = Arc(
    in_port=evenlode_thames, out_port=farmoor_abstraction, name="thames_to_farmoor"
)

farmoor_to_mixer = Arc(
    in_port=farmoor_abstraction, out_port=thames_mixer, name="farmoor_to_mixer"
)

cherwell_to_mixer = Arc(
    in_port=cherwell_ray, out_port=thames_mixer, name="cherwell_to_mixer"
)

wwtw_to_mixer = Arc(in_port=oxford_wwtw, out_port=thames_mixer, name="wwtw_to_mixer")

mixer_to_waste = Arc(
    in_port=thames_mixer, out_port=thames_above_abingdon, name="mixer_to_waste"
)

distribution_to_demand = Arc(
    in_port=distribution, out_port=oxford, name="distribution_to_demand"
)

reservoir_to_fwtw = Arc(in_port=farmoor, out_port=oxford_fwtw, name="reservoir_to_fwtw")

fwtw_to_sewer = Arc(in_port=oxford_fwtw, out_port=combined_sewer, name="fwtw_to_sewer")

demand_to_sewer = Arc(in_port=oxford, out_port=combined_sewer, name="demand_to_sewer")

land_to_sewer = Arc(in_port=oxford_land, out_port=combined_sewer, name="land_to_sewer")

land_to_gw = Arc(in_port=oxford_land, out_port=gw, name="land_to_gw")

garden_to_gw = Arc(in_port=oxford, out_port=gw, name="garden_to_gw")

gw_to_mixer = Arc(in_port=gw, out_port=thames_mixer, name="gw_to_mixer")

evenlode_to_thames = Arc( in_port=evenlode, out_port=evenlode_thames, name="evenlode_to_thames" ) thames_to_thames = Arc( in_port=thames, out_port=evenlode_thames, name="thames_to_thames" ) ray_to_cherwell = Arc(in_port=ray, out_port=cherwell_ray, name="ray_to_cherwell") cherwell_to_cherwell = Arc( in_port=cherwell, out_port=cherwell_ray, name="cherwell_to_cherwell" ) thames_to_farmoor = Arc( in_port=evenlode_thames, out_port=farmoor_abstraction, name="thames_to_farmoor" ) farmoor_to_mixer = Arc( in_port=farmoor_abstraction, out_port=thames_mixer, name="farmoor_to_mixer" ) cherwell_to_mixer = Arc( in_port=cherwell_ray, out_port=thames_mixer, name="cherwell_to_mixer" ) wwtw_to_mixer = Arc(in_port=oxford_wwtw, out_port=thames_mixer, name="wwtw_to_mixer") mixer_to_waste = Arc( in_port=thames_mixer, out_port=thames_above_abingdon, name="mixer_to_waste" ) distribution_to_demand = Arc( in_port=distribution, out_port=oxford, name="distribution_to_demand" ) reservoir_to_fwtw = Arc(in_port=farmoor, out_port=oxford_fwtw, name="reservoir_to_fwtw") fwtw_to_sewer = Arc(in_port=oxford_fwtw, out_port=combined_sewer, name="fwtw_to_sewer") demand_to_sewer = Arc(in_port=oxford, out_port=combined_sewer, name="demand_to_sewer") land_to_sewer = Arc(in_port=oxford_land, out_port=combined_sewer, name="land_to_sewer") land_to_gw = Arc(in_port=oxford_land, out_port=gw, name="land_to_gw") garden_to_gw = Arc(in_port=oxford, out_port=gw, name="garden_to_gw") gw_to_mixer = Arc(in_port=gw, out_port=thames_mixer, name="gw_to_mixer")

Again, we keep all the arcs in a tidy list together.

In [44]:

arclist = [
    evenlode_to_thames,
    thames_to_thames,
    ray_to_cherwell,
    cherwell_to_cherwell,
    thames_to_farmoor,
    farmoor_to_mixer,
    cherwell_to_mixer,
    wwtw_to_mixer,
    sewer_overflow,
    mixer_to_waste,
    abstraction_to_farmoor,
    distribution_to_demand,
    demand_to_sewer,
    land_to_sewer,
    sewer_to_wwtw,
    fwtw_to_sewer,
    fwtw_to_distribution,
    reservoir_to_fwtw,
    land_to_gw,
    garden_to_gw,
    gw_to_mixer,
]

arclist = [ evenlode_to_thames, thames_to_thames, ray_to_cherwell, cherwell_to_cherwell, thames_to_farmoor, farmoor_to_mixer, cherwell_to_mixer, wwtw_to_mixer, sewer_overflow, mixer_to_waste, abstraction_to_farmoor, distribution_to_demand, demand_to_sewer, land_to_sewer, sewer_to_wwtw, fwtw_to_sewer, fwtw_to_distribution, reservoir_to_fwtw, land_to_gw, garden_to_gw, gw_to_mixer, ]

Mapping¶

Remember, WSIMOD is an integrated model. Because it covers so many different things, it is very easy to make mistakes. Thus it is always good practice to plot your data!

Below we load the node location data and create arcs from the information in the arclist.

In [45]:

location_fn = os.path.join(data_folder, "raw", "points_locations.geojson")
nodes_gdf = gpd.read_file(location_fn).set_index("name")
arcs_gdf = []

for arc in arclist:
    arcs_gdf.append(
        {
            "name": arc.name,
            "geometry": LineString(
                [
                    nodes_gdf.loc[arc.in_port.name, "geometry"],
                    nodes_gdf.loc[arc.out_port.name, "geometry"],
                ]
            ),
        }
    )

arcs_gdf = gpd.GeoDataFrame(arcs_gdf, crs=nodes_gdf.crs)

location_fn = os.path.join(data_folder, "raw", "points_locations.geojson") nodes_gdf = gpd.read_file(location_fn).set_index("name") arcs_gdf = [] for arc in arclist: arcs_gdf.append( { "name": arc.name, "geometry": LineString( [ nodes_gdf.loc[arc.in_port.name, "geometry"], nodes_gdf.loc[arc.out_port.name, "geometry"], ] ), } ) arcs_gdf = gpd.GeoDataFrame(arcs_gdf, crs=nodes_gdf.crs)

Because we converted the information as GeoDataFrames, we can simply plot them below

In [46]:

f, ax = plt.subplots()
arcs_gdf.plot(ax=ax)
nodes_gdf.plot(color="r", ax=ax, zorder=10)

f, ax = plt.subplots() arcs_gdf.plot(ax=ax) nodes_gdf.plot(color="r", ax=ax, zorder=10)

Out[46]:

<Axes: >

Orchestration¶

Orchestration is making the simulation happen by calling functions in the nodes. These functions simulate physical behaviour within the node, and cause pulls/pushes to happen which in turn triggers physical behaviour in other nodes.

Orchestrating an individual timestep¶

We will start below by manually orchestrating a single timestep.

We start by setting the date, so that every node knows what forcing data to read for this timestep.

In [47]:

date = dates[0]

for node in nodelist:
    node.t = date

print(date)
print(oxford_fwtw.t)

date = dates[0] for node in nodelist: node.t = date print(date) print(oxford_fwtw.t)

2009-03-03 00:00:00
2009-03-03 00:00:00

We can see the service reservoirs are empty but the supply reservoir is not!

In [48]:

print(oxford_fwtw.service_reservoir_tank.storage)
print(farmoor.tank.storage)

print(oxford_fwtw.service_reservoir_tank.storage) print(farmoor.tank.storage)

{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 0.0}
{'temperature': 0, 'silicon': 0, 'fluoride': 0, 'chloride': 0, 'nitrate': 0, 'sulphate': 0, 'nitrogen': 0, 'sodium': 0, 'potassium': 0, 'calcium': 0, 'magnesium': 0, 'boron': 0, 'volume': 10000000.0}

If we call the FWTW's treat_water function it will pull water from the supply reservoir and update its service reservoirs

In [49]:

oxford_fwtw.treat_water()

print(oxford_fwtw.service_reservoir_tank.storage)
print(farmoor.tank.storage)

oxford_fwtw.treat_water() print(oxford_fwtw.service_reservoir_tank.storage) print(farmoor.tank.storage)

{'temperature': 0.0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 43641.0}
{'temperature': 0, 'silicon': 0.0, 'fluoride': 0.0, 'chloride': 0.0, 'nitrate': 0.0, 'sulphate': 0.0, 'nitrogen': 0.0, 'sodium': 0.0, 'potassium': 0.0, 'calcium': 0.0, 'magnesium': 0.0, 'boron': 0.0, 'volume': 9955000.0}

This information is tracked in the arcs that enter the FWTW

In [50]:

print(oxford_fwtw.in_arcs)

print(oxford_fwtw.in_arcs)

{'reservoir_to_fwtw': <wsimod.arcs.arcs.Arc object at 0x7f85c658ed10>}

In [51]:

print(reservoir_to_fwtw.flow_in)
print(reservoir_to_fwtw.flow_out)

print(reservoir_to_fwtw.flow_in) print(reservoir_to_fwtw.flow_out)

45000.0
45000.0

Although none of that water has yet entered the distribution network (only some small flow from the earlier demonstration)

In [52]:

print(fwtw_to_distribution.flow_in)

print(fwtw_to_distribution.flow_in)

15.0

That is because no water consumption demand had yet been generated.

If we call the demand node's create_demand function we see that the distribution arc becomes utilised.

In [53]:

oxford.create_demand()
print(fwtw_to_distribution.flow_in)

oxford.create_demand() print(fwtw_to_distribution.flow_in)

30015.0

We also see that this gets pushed onwards into the sewer system

In [54]:

print(demand_to_sewer.flow_in)

print(demand_to_sewer.flow_in)

30000.0

Many nodes have functions intended to be called during orchestration. These functions are described in the documentation. For example, we see in the Land node API reference that the 'run' function is intended to be called from orchestration.

In [55]:

oxford_land.run()

oxford_land.run()

Below we call the functions for other nodes

In [56]:

# Discharge GW
gw.distribute()

# Discharge sewers (pushed to other sewers or WWTW)
combined_sewer.make_discharge()

# Run WWTW model
oxford_wwtw.calculate_discharge()

# Make abstractions
farmoor.make_abstractions()

# Discharge WW
oxford_wwtw.make_discharge()

# Route catchments
evenlode.route()
thames.route()
ray.route()
cherwell.route()

# Discharge GW gw.distribute() # Discharge sewers (pushed to other sewers or WWTW) combined_sewer.make_discharge() # Run WWTW model oxford_wwtw.calculate_discharge() # Make abstractions farmoor.make_abstractions() # Discharge WW oxford_wwtw.make_discharge() # Route catchments evenlode.route() thames.route() ray.route() cherwell.route()

Ending a timestep¶

Because mistakes happen, it is essential to carry out mass balance testing. Each node has a mass balance function that can be called. We see a mass balance violation resulting from the demonstration with the FWTW earlier.

In [57]:

for node in nodelist:
    in_, ds_, out_ = node.node_mass_balance()

for node in nodelist: in_, ds_, out_ = node.node_mass_balance()

mass balance error for volume of 15.0 in oxford_distribution
mass balance error for volume of -15.0 in oxford_fwtw

We should also call the end_timestep function in nodes and arcs. This is important for mass balance testing and capturing the behaviour of some dynamic processes in nodes.

In [58]:

for node in nodelist:
    node.end_timestep()

for arc in arclist:
    arc.end_timestep()

for node in nodelist: node.end_timestep() for arc in arclist: arc.end_timestep()

Model object¶

Of course it would be a massive pain to manually orchestrate every timestep. So instead we store node and arc information in a model object that will do the orchestration for us.

Because we have already created the nodes/arcs above, we simply need to add the instantiated lists above.

In [59]:

my_model = Model()
my_model.add_instantiated_nodes(nodelist)
my_model.add_instantiated_arcs(arclist)
my_model.dates = dates

my_model = Model() my_model.add_instantiated_nodes(nodelist) my_model.add_instantiated_arcs(arclist) my_model.dates = dates

The model object lets us reinitialise the nodes/arcs, and run all of the orchestration with a 'run' function.

In [60]:

my_model.reinit()

flows, _, _, _ = my_model.run()

my_model.reinit() flows, _, _, _ = my_model.run()

  0%|          | 0/1456 [00:00<?, ?it/s]

  2%|▏         | 34/1456 [00:00<00:04, 335.60it/s]

  5%|▍         | 68/1456 [00:00<00:04, 335.84it/s]

  7%|▋         | 102/1456 [00:00<00:04, 332.35it/s]

  9%|▉         | 136/1456 [00:00<00:03, 334.61it/s]

 12%|█▏        | 170/1456 [00:00<00:03, 333.88it/s]

 14%|█▍        | 204/1456 [00:00<00:03, 333.57it/s]

 16%|█▋        | 238/1456 [00:00<00:03, 332.84it/s]

 19%|█▊        | 272/1456 [00:00<00:03, 328.68it/s]

 21%|██        | 306/1456 [00:00<00:03, 329.21it/s]

 23%|██▎       | 339/1456 [00:01<00:03, 329.27it/s]

 26%|██▌       | 372/1456 [00:01<00:03, 329.27it/s]

 28%|██▊       | 405/1456 [00:01<00:03, 328.72it/s]

 30%|███       | 439/1456 [00:01<00:03, 329.65it/s]

 32%|███▏      | 472/1456 [00:01<00:02, 329.74it/s]

 35%|███▍      | 506/1456 [00:01<00:02, 330.72it/s]

 37%|███▋      | 540/1456 [00:01<00:02, 328.72it/s]

 39%|███▉      | 574/1456 [00:01<00:02, 329.39it/s]

 42%|████▏     | 608/1456 [00:01<00:02, 329.89it/s]

 44%|████▍     | 641/1456 [00:01<00:02, 329.84it/s]

 46%|████▋     | 675/1456 [00:02<00:02, 330.41it/s]

 49%|████▊     | 709/1456 [00:02<00:02, 329.67it/s]

 51%|█████     | 742/1456 [00:02<00:02, 329.56it/s]

 53%|█████▎    | 775/1456 [00:02<00:02, 329.23it/s]

 55%|█████▌    | 808/1456 [00:02<00:01, 328.11it/s]

 58%|█████▊    | 841/1456 [00:02<00:01, 328.65it/s]

 60%|██████    | 874/1456 [00:02<00:01, 328.62it/s]

 62%|██████▏   | 908/1456 [00:02<00:01, 329.27it/s]

 65%|██████▍   | 942/1456 [00:02<00:01, 329.79it/s]

 67%|██████▋   | 976/1456 [00:02<00:01, 330.46it/s]

 69%|██████▉   | 1010/1456 [00:03<00:01, 330.93it/s]

 72%|███████▏  | 1044/1456 [00:03<00:01, 329.39it/s]

 74%|███████▍  | 1077/1456 [00:03<00:01, 328.21it/s]

 76%|███████▌  | 1110/1456 [00:03<00:01, 327.15it/s]

 79%|███████▊  | 1144/1456 [00:03<00:00, 328.10it/s]

 81%|████████  | 1177/1456 [00:03<00:00, 327.78it/s]

 83%|████████▎ | 1210/1456 [00:03<00:00, 327.74it/s]

 85%|████████▌ | 1243/1456 [00:03<00:00, 327.82it/s]

 88%|████████▊ | 1276/1456 [00:03<00:00, 327.79it/s]

 90%|████████▉ | 1310/1456 [00:03<00:00, 328.97it/s]

 92%|█████████▏| 1343/1456 [00:04<00:00, 326.82it/s]

 95%|█████████▍| 1376/1456 [00:04<00:00, 327.54it/s]

 97%|█████████▋| 1409/1456 [00:04<00:00, 327.01it/s]

 99%|█████████▉| 1442/1456 [00:04<00:00, 327.04it/s]

100%|██████████| 1456/1456 [00:04<00:00, 329.31it/s]

The model outputs flows as a dictionary which can be converted to a dataframe

In [61]:

flows = pd.DataFrame(flows)

print(flows.sample(10))

flows = pd.DataFrame(flows) print(flows.sample(10))

                          arc           flow       time  temperature  \
14804             gw_to_mixer    9962.619356 2011-02-05     8.842734   
29684  distribution_to_demand   30000.000000 2013-01-14    10.981650   
23629       thames_to_farmoor  499392.000000 2012-04-01    11.119822   
12441          mixer_to_waste  551779.132154 2010-10-16    11.642158   
3964     fwtw_to_distribution   30000.000000 2009-09-07     7.086445   
27929             gw_to_mixer   28198.367226 2012-10-22    13.178160   
30127           land_to_sewer       0.000000 2013-02-04     0.000000   
22550       reservoir_to_fwtw   30934.213240 2012-02-09    10.790209   
10748       reservoir_to_fwtw   30934.213240 2010-07-27    10.282667   
12156              land_to_gw   18387.106974 2010-10-02    14.353861   

           silicon   fluoride      chloride      nitrate      sulphate  \
14804     0.090084   0.257382      0.077215     0.025738     90.083596   
29684     2.009045   0.041834      9.286433     1.748716     14.224264   
23629   525.012192  66.748937  17837.470080  2700.024519  28560.408686   
12441  4165.754591  88.927764  20853.981290  2918.989808  30592.891463   
3964      0.729897   0.018252      4.605352     0.747359      6.490232   
27929     0.159771   0.456488      0.136946     0.045649    159.770708   
30127     0.000000   0.000000      0.000000     0.000000      0.000000   
22550   190.982542   4.136400   1158.995759   170.814055   1691.804214   
10748   144.287270   3.162487    742.101520   146.725914   1139.780090   
12156     0.157380   0.449658      0.134898     0.044966    157.380434   

          nitrogen        sodium    potassium       calcium    magnesium  \
14804     5.147634      0.386073     9.008360     90.083596     7.721451   
29684     1.892480      5.907127     1.265702     31.677048     1.429036   
23629  2755.005943  11171.211429  2135.622857  51139.382400  2511.857829   
12441  3001.439846  13677.642542  2869.421236  50826.417091  2808.494594   
3964      0.780465      2.963568     0.598848     14.010095     0.678831   
27929     9.129755      0.684732    15.977071    159.770708    13.694632   
30127     0.000000      0.000000     0.000000      0.000000     0.000000   
22550   185.332987    781.018656   155.110589   3032.347802   149.880913   
10748   152.887695    460.785335    94.720714   2483.624695   115.381934   
12156     8.993168      0.674488    15.738043    157.380434    13.489751   

           boron  
14804   0.128691  
29684   0.013883  
23629  23.589014  
12441  32.629925  
3964    0.007544  
27929   0.228244  
30127   0.000000  
22550   1.567645  
10748   1.151882  
12156   0.224829  

Validation plots¶

Of course we shouldn't trust a model without proof, and this is doubly true for an integrated model whose errors may easily propagate.

Thankfully we have rich in-river water quality sampling in Oxford that we can use for validation

We load and format this data for dates and pollutants that overlap with what we have simulated

In [62]:

mixer_val_df = pd.read_csv(os.path.join(data_folder, "raw", "mixer_wims_val.csv"))
mixer_val_df.date = pd.to_datetime(mixer_val_df.date)
mixer_val_df = mixer_val_df.loc[mixer_val_df.date.isin(dates)]
val_pollutants = mixer_val_df.variable.unique()
mixer_val_df = mixer_val_df.pivot(index="date", columns="variable", values="value")

mixer_val_df = pd.read_csv(os.path.join(data_folder, "raw", "mixer_wims_val.csv")) mixer_val_df.date = pd.to_datetime(mixer_val_df.date) mixer_val_df = mixer_val_df.loc[mixer_val_df.date.isin(dates)] val_pollutants = mixer_val_df.variable.unique() mixer_val_df = mixer_val_df.pivot(index="date", columns="variable", values="value")

We get rid of the first day of flows because our tanks were initialised empty and will not be informative

In [63]:

flows = flows.loc[flows.time != dates[0]]

flows = flows.loc[flows.time != dates[0]]

We convert our flows, which are simulated in kg/d into a concentration value (kg/m3/d).

In [64]:

flows_plot = flows.copy()

# Convert to KG/M3
for pol in set(val_pollutants):
    if pol != "temperature":
        flows_plot[pol] /= flows_plot.flow

flows_plot = flows.copy() # Convert to KG/M3 for pol in set(val_pollutants): if pol != "temperature": flows_plot[pol] /= flows_plot.flow

We pick the model outlet as a validation location and make a pretty plot - fantastic!

In [65]:

plot_arc = "mixer_to_waste"
f, axs = plt.subplots(val_pollutants.size, 1)
for pol, ax in zip(val_pollutants, axs):
    ax.plot(
        flows_plot.loc[flows_plot.arc == plot_arc, [pol, "time"]].set_index("time"),
        color="b",
        label="simulation",
    )
    ax.plot(mixer_val_df[pol], ls="", marker="o", color="r", label="spot sample")
    ax.set_ylabel(pol)
plt.legend()

plot_arc = "mixer_to_waste" f, axs = plt.subplots(val_pollutants.size, 1) for pol, ax in zip(val_pollutants, axs): ax.plot( flows_plot.loc[flows_plot.arc == plot_arc, [pol, "time"]].set_index("time"), color="b", label="simulation", ) ax.plot(mixer_val_df[pol], ls="", marker="o", color="r", label="spot sample") ax.set_ylabel(pol) plt.legend()

Out[65]:

<matplotlib.legend.Legend at 0x7f85c6542350>