DEPARTMENT OF INFRASTRUCTURE ENGINEERING UNIVERSITY OF MELBOURNE ENEN90031Quantitative Environmental Modelling
Modelling Assignment 2
Due Midnight on Friday 28th May 2021.
Introduction
This assignment concentrates on the use of global model analysis techniques that can be applied to optimisation, sensitivity analysis and model uncertainty analysis. The final step involves applying your model results to determine the volume-reliability relationship for an irrigation supply reservoir and to estimate the impact of climate change on that relationship. The key parts of the assignment are:
1 Calibration and validation of your model using the global optimization technique Differential Evolution;
2 Undertaking a regional sensitivity analysis to understand the effects of different parameters on the model performance;
3 Undertaking an uncertainty analysis using GLUE; and
4 Assessing the volume-reliability relationship of an irrigation supply reservoir and the impact of climate change on that relationship.
The assessment for this assignment is as follows.
• Model calibration and validation (part 1): (Technical 10%, Discussion 10%)
• Regional sensitivity analysis (part 2) (Technical 10%, Discussion 10%)
• Parameter uncertainty analysis (part 3) (Technical 10%, Discussion 10%)
• Irrigation assessment (part 4) (Technical 10%, Discussion 10%)
• Discussion and conclusions (part 5) (Discussion 10%)
• Quality of report presentation 10%
Your report should contain the following: aims, introduction, results, discussion, conclusions, references and an appendix containing your Python code. The report must be no more than 12 pages (including graphs but excluding the appendix) and any additional pages will not be assessed. In writing your report, try to demonstrate to the reader your knowledge of the methodologies and their limitations using your understanding of the techniques used and modelling results. Please submit your report using the LMS assignment link.
This assignment uses data from station 401203 Mitta Mitta River at Hinnomunjie, which flows into Dartmouth Reservoir in eastern Victoria.
1 /6
Engineering Practice Hurdle
This assignment can be used as the final piece of your Engineering Practice Hurdle Written Communication submission. STEP workshops and online lessons are available to help you further develop your writing skills.
See the Skills Towards Employment Program community for more details on the Engineering Practice Hurdle.
Code notes
We have provided partial code in a Jupyter notebook. You may need to install the pyDOE library using pip install pyDOE from Anaconda command. You also need WapabaModel.py
You have, or soon will have, undertaken a range of similar analyses in tutorial classes and should refer back to that code and adapt it for various parts of this assignment.
Part 1: Global Calibration and Validation using Differential Evolution
The aim of this part is to calibrate and validate the Wapaba model for the study catchment. Note, in using Differential Evolution you can assume it is reliably converging to the global optimum (we have check this for you). Using the Differential Evolution algorithm and approaches learnt in class, you should:
a) Use a split-sample calibration and validation analysis to assess how well the model predicts runoff behaviour for periods that are independent of the calibration period. In making this assessment consider both mean squared error and coefficient of efficiency. Also consider the model bias in different periods of calibration and validation.
b) Assess whether the parameters depend on the calibration period and how different the model predictions are for different parameter sets.
For this section, your report should:
1.1. outline your methods for the split sample analysis;
1.2. provide evidence to support your assessment of the impact of calibration period;
1.3. provide any comments you may have on the adequacy and limitations of your analysis; and 1.4. summarise your findings.
Part 2: Regional sensitivity analysis (RSA)
The aim of Regional Sensitivity Analysis is to understand the relationships between the model predictions and the parameters. This is based on a Monte Carlo analysis of the model. Here you should:
a) Assess the sensitivity of Wapaba to each parameter;
b) Assess the parameter interactions; and
c) Assess to what degree your answer depends on the threshold you use to define behavioural runs
For this section, your report should provide an interpretation of results of the Regional Sensitivity 2
/6
Analysis in terms of:
2.1. how sensitive the objective function value is to each parameter;
2.2. whether there are significant parameter interactions;
2.3. which parameters are most likely to be reliably identified through calibration; 2.4. whether these results help you interpret your part 1 results, and if so, how; and 2.5. what limitations there might be for this analysis.
Part 3: Prediction uncertainty analysis
Having now undertaken both the global optimisation and the sensitivity analysis, it is now time to focus in on how uncertain the model predictions are. To estimate the prediction uncertainty, please use the Generalised Likelihood Uncertainty Estimation (GLUE) methodology. To do this you should adapt the code from Tutorial 9.
For this section of the assignment you should:
a) Choose an appropriate threshold to define behavioural runs; and
b) Assess the resulting 90 percent prediction intervals (i.e. 5 to 95%) for runoff and the frequency with which observed flows fall within, above and below that prediction interval.
Your report should:
3.1. Justify your choice of behavioural threshold, including through a discussion of point b above;
3.2. Present the resulting flow prediction intervals; and
3.3. Critique your results and the overall analysis approach.
Note the similarities in RSA and GLUE and take this into account when writing your report. For example, there is no need to repeat figures that have very similar content and your discussion should not repeat the same/similar things.
Part 4: Irrigation storage volume-reliability trade-off and climate change
assessment
Now that the model has been calibrated and evaluated, the model can be used to make predictions. By also using the parameter uncertainties, the prediction error can be quantified. This part of the assignment looks at the volume and reliability of an irrigation storage dam. A timeseries of demand is calculated within the Jupyter notebook provided for the assignment. We have provided a simple model of the storage (assuming the only inflows are from the stream and the only outflows are to meet demand or spills to the downstream river when the storage is completely full). For our purposes here, you should assess reliability as:
Reliability = volume of water supplied / volume of water demanded
where the volumes are summed for the simulation period, excluding the warmup period. Exclude the warm-up period from this assessment.
We also provide the irrigation area, irrigation demand, and catchment area in the Jupyter notebook. Such a facility would have a long life so it is also important to estimate future reliability for each
3 /6
system. Wapaba can be used to estimate changes in flows with changed climate and these can then be used to see how reliable the water supply is.
There are two tasks here:
a) Using the model to develop a volume-reliability curve for an irrigation storage by calculating reliability for different storage volumes using the behavioural models and the simulation period, excluding the warmup period. You should indicate the uncertainty in reliability.
b) Repeat a) for a climate change scenario where there is a 20% reduction in rainfall, a 5% increase in PET (i.e. run Wapaba with rainfall multiplied by 0.85 and PET by 1.05) and a 5% increase in irrigation demand (multiply demand by 1.05). Also indicate an estimate of the uncertainty resulting from Wapaba.
In writing up your assessment of the impact of climate change on the irrigation reliability, you should evaluate how well you think these uncertainties have been captured and comment on potential improvements in the method that you can think of. Comment on other potential sources of uncertainty.
Part 5: Discussion and conclusions
Lastly, in your report you need to discuss the strengths and weakness of your modelling for this assignment and summarise the overall findings from each part of the model analysis.
4 /6