Providing Valid Inputs#
PyProbe uses Pydantic for input validation. This exists to ensure that the data provided is in the correct format to prevent unexpected errors. Most of the time, this will happen behind-the-scenes, so you will only notice it if there is a problem. This example is written to demonstrate how these errors may come about.
RawData Validation#
The RawData class is a specific variant of the Result object which only stores data in the standard PyProBE format. Therefore, validation is performed when a RawData object is created to verify this.
If you follow the standard method for importing data into PyProBE, you should never experience these errors, however it is helpful to know that they exist.
We will start with a normal dataset, printing the type that the procedure data is stored in:
[1]:
import pyprobe
import polars as pl
info_dictionary = {'Name': 'Sample cell',
'Chemistry': 'NMC622',
'Nominal Capacity [Ah]': 0.04,
'Cycler number': 1,
'Channel number': 1,}
data_directory = '../../../tests/sample_data/neware'
# Create a cell object
cell = pyprobe.Cell(info=info_dictionary)
cell.add_procedure(procedure_name='Sample',
folder_path = data_directory,
filename = 'sample_data_neware.parquet')
print(type(cell.procedure['Sample']))
<class 'pyprobe.filters.Procedure'>
The Procedure class inherits from RawData, which has a defined set of required columns (the PyProBE standard format):
[2]:
print(pyprobe.rawdata.required_columns)
['Time [s]', 'Step', 'Event', 'Current [A]', 'Voltage [V]', 'Capacity [Ah]']
Whenever a RawData class (or any of the filters module classes, that inherit from it) are created, the dataframe is checked against these required columns. We will create an example dataframe that is missing columns, which will be identified by the error that is returned.
[3]:
incorrect_dataframe = pl.DataFrame({'Time [s]': [1, 2, 3],
'Voltage [V]': [3.5, 3.6, 3.7],
'Current [A]': [0.1, 0.2, 0.3],
})
pyprobe.rawdata.RawData(base_dataframe=incorrect_dataframe,
info = {})
2024-12-23 16:57:36,506 - pyprobe.rawdata - ERROR - Missing required columns: ['Step', 'Event', 'Capacity [Ah]']
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[3], line 5
1 incorrect_dataframe = pl.DataFrame({'Time [s]': [1, 2, 3],
2 'Voltage [V]': [3.5, 3.6, 3.7],
3 'Current [A]': [0.1, 0.2, 0.3],
4 })
----> 5 pyprobe.rawdata.RawData(base_dataframe=incorrect_dataframe,
6 info = {})
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/main.py:193, in BaseModel.__init__(self, **data)
191 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
192 __tracebackhide__ = True
--> 193 self.__pydantic_validator__.validate_python(data, self_instance=self)
ValidationError: 1 validation error for RawData
base_dataframe
Value error, Missing required columns: ['Step', 'Event', 'Capacity [Ah]'] [type=value_error, input_value=shape: (3, 3)
┌──...───────┘, input_type=DataFrame]
For further information visit https://errors.pydantic.dev/2.8/v/value_error
You will also see a validation error if you try to create one of these classes with a data object that is not a Polars DataFrame or LazyFrame:
[4]:
incorrect_data_dict = {'Time [s]': [1, 2, 3],
'Voltage [V]': [3.5, 3.6, 3.7],
'Current [A]': [0.1, 0.2, 0.3],
}
pyprobe.rawdata.RawData(base_dataframe=incorrect_data_dict,
info = {})
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[4], line 5
1 incorrect_data_dict = {'Time [s]': [1, 2, 3],
2 'Voltage [V]': [3.5, 3.6, 3.7],
3 'Current [A]': [0.1, 0.2, 0.3],
4 }
----> 5 pyprobe.rawdata.RawData(base_dataframe=incorrect_data_dict,
6 info = {})
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/main.py:193, in BaseModel.__init__(self, **data)
191 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
192 __tracebackhide__ = True
--> 193 self.__pydantic_validator__.validate_python(data, self_instance=self)
ValidationError: 2 validation errors for RawData
base_dataframe.is-instance[LazyFrame]
Input should be an instance of LazyFrame [type=is_instance_of, input_value={'Time [s]': [1, 2, 3], '...t [A]': [0.1, 0.2, 0.3]}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.8/v/is_instance_of
base_dataframe.is-instance[DataFrame]
Input should be an instance of DataFrame [type=is_instance_of, input_value={'Time [s]': [1, 2, 3], '...t [A]': [0.1, 0.2, 0.3]}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.8/v/is_instance_of
Analysis Module Validation#
You are much more likely to experience validation errors when dealing with the functions and classes in the analysis module. These may require a particular PyProBE object to work.
As an example, the Cycling class requires an Experiment input. This is because it provides calculations based on the cycle() method of the experiment class:
[5]:
experiment_object = cell.procedure['Sample'].experiment('Break-in Cycles')
print(type(experiment_object))
<class 'pyprobe.filters.Experiment'>
The experiment object should return no errors:
[6]:
from pyprobe.analysis.cycling import Cycling
cycling = Cycling(input_data = experiment_object)
However, if I were to filter the object further, I would get an error:
[7]:
cycling = Cycling(input_data = experiment_object.cycle(1))
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[7], line 1
----> 1 cycling = Cycling(input_data = experiment_object.cycle(1))
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/main.py:193, in BaseModel.__init__(self, **data)
191 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
192 __tracebackhide__ = True
--> 193 self.__pydantic_validator__.validate_python(data, self_instance=self)
ValidationError: 1 validation error for Cycling
input_data
Input should be a valid dictionary or instance of Experiment [type=model_type, input_value=Cycle(base_dataframe=<Laz...hours']}, cycle_info=[]), input_type=Cycle]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
Functions in the analysis module also contain type validation. This occurs on two levels. First, the inputs to the function are checked. E.g. for the gradient function of the differentiation module, input_data is required as a PyProBE object, and string column names are required for x and y:
[8]:
from pyprobe.analysis import differentiation
gradient = differentiation.gradient(input_data = cell.procedure['Sample'].experiment('Break-in Cycles').discharge(-1),
x = "Capacity [Ah]",
y = "Voltage [V]")
But if I provide an array to input_data, I will get an error to say that input_data should be one of many PyProBE objects:
[9]:
import numpy as np
gradient = differentiation.gradient(input_data = np.zeros((10, 2)),
x = "Capacity [Ah]",
y = "Voltage [V]")
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[9], line 2
1 import numpy as np
----> 2 gradient = differentiation.gradient(input_data = np.zeros((10, 2)),
3 x = "Capacity [Ah]",
4 y = "Voltage [V]")
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/validate_call_decorator.py:60, in validate_call.<locals>.validate.<locals>.wrapper_function(*args, **kwargs)
58 @functools.wraps(function)
59 def wrapper_function(*args, **kwargs):
---> 60 return validate_call_wrapper(*args, **kwargs)
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py:96, in ValidateCallWrapper.__call__(self, *args, **kwargs)
95 def __call__(self, *args: Any, **kwargs: Any) -> Any:
---> 96 res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
97 if self.__return_pydantic_validator__:
98 return self.__return_pydantic_validator__(res)
ValidationError: 6 validation errors for gradient
input_data.RawData
Input should be a valid dictionary or instance of RawData [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
input_data.Procedure
Input should be a valid dictionary or instance of Procedure [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
input_data.Experiment
Input should be a valid dictionary or instance of Experiment [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
input_data.Cycle
Input should be a valid dictionary or instance of Cycle [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
input_data.Step
Input should be a valid dictionary or instance of Step [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
input_data.Result
Input should be a valid dictionary or instance of Result [type=model_type, input_value=array([[0., 0.],
[..., 0.],
[0., 0.]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.8/v/model_type
Analysis functions will also check that the columns you require for the computation are present in the PyProBE objects provided. As an example, we will call the gradient() method, requesting to differentiate a column that does not exist in the underlying data:
[10]:
gradient = differentiation.gradient(input_data = cell.procedure['Sample'].experiment('Break-in Cycles').discharge(-1),
x = "Temperature [C]",
y = "Voltage [V]")
2024-12-23 16:57:36,976 - pyprobe.units - ERROR - Name Date does not match pattern.
2024-12-23 16:57:36,977 - pyprobe.units - ERROR - Name Step does not match pattern.
2024-12-23 16:57:36,977 - pyprobe.units - ERROR - Name Event does not match pattern.
2024-12-23 16:57:36,978 - pyprobe.result - ERROR - Column with quantity 'Temperature'
2024-12-23 16:57:36,978 - pyprobe.analysis.utils - ERROR - Missing required columns: ['Temperature [C]']
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[10], line 1
----> 1 gradient = differentiation.gradient(input_data = cell.procedure['Sample'].experiment('Break-in Cycles').discharge(-1),
2 x = "Temperature [C]",
3 y = "Voltage [V]")
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/validate_call_decorator.py:60, in validate_call.<locals>.validate.<locals>.wrapper_function(*args, **kwargs)
58 @functools.wraps(function)
59 def wrapper_function(*args, **kwargs):
---> 60 return validate_call_wrapper(*args, **kwargs)
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py:96, in ValidateCallWrapper.__call__(self, *args, **kwargs)
95 def __call__(self, *args: Any, **kwargs: Any) -> Any:
---> 96 res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
97 if self.__return_pydantic_validator__:
98 return self.__return_pydantic_validator__(res)
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pyprobe/analysis/differentiation.py:38, in gradient(input_data, x, y)
22 """Differentiate smooth data with a finite difference method.
23
24 A wrapper of the numpy.gradient function. This method calculates the gradient
(...)
35 calculated gradient.
36 """
37 # 2. Validate the inputs to the method
---> 38 validator = AnalysisValidator(
39 input_data=input_data,
40 required_columns=[x, y],
41 # required_type not neccessary here as type specified when declaring
42 # input_data attribute is strict enough
43 )
44 # 3. Retrieve the validated columns as numpy arrays
45 x_data, y_data = validator.variables
File /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/pydantic/main.py:193, in BaseModel.__init__(self, **data)
191 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
192 __tracebackhide__ = True
--> 193 self.__pydantic_validator__.validate_python(data, self_instance=self)
ValidationError: 1 validation error for AnalysisValidator
Value error, Missing required columns: ['Temperature [C]'] [type=value_error, input_value={'input_data': Step(base_...re [C]', 'Voltage [V]']}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.8/v/value_error