Data standardization for Smart-regalional and inter-kommunal water-quality monitoring on Mjøsa

INFORMATION ARCHITECTURE

Gjøvik Kommune is promoting a Smart-City program of development aimed at the integration of Smart (IoT) technologies into municipal planning. To meet the expectations and mandate of smart governance, there is a need for a greater degree of cooperation and collaboration between academic institutions, governmental agencies at the municipal (kommune) and county (fylke) levels to monitor environmental pollution and water-quality. Digital standards, research criteria standards, quality control (QC), quality assurance (QA) and verification of standards are especially critical.  

Data standardization refers in the macro- sense to considerations that are inter-organizational and interdisciplinary and that hold bearing with regard to research planning and design as information architecture protocols. On the project-detail and database-management level we can look closer at standardization of labeling, naming conventions and tracking or archiving practice. Information architectures of several governmental and research organizations can be informative to the research and activity of student efforts and will be most useful if they are approached in a coordinated way.

Smart-by survey

The Royal Norwegian Ministry of Municipal Modernization published a November 2019 report on the results of a survey seeking perceptions about Smart-city programs. Employees of municipalities were asked to provide input on what contributions the state can make in context with the further development of Smart-city integrations in Norway. The top rated categories are related to technical standardization (74%), ensuring secure common technical components (74%) and dissemination of knowledge and experience. In fact, these rated higher than the perceived need for support of research and development.

It could be further interpreted that respondents feel the proper role for state interventions fall within the area of standardization, quality assurance (QA) quality control (QC) for the enforcement of regulatory standards. Research and development and innovation is meanwhile considered the domain of private business and academic institutions. Business, community and educational sectors or zones are where research and development should be free to explore. 

Environmental standards are often contested. The criteria recognizing and importance of particular instances of pollution are weighted differently. Testing methods and even what to measure and regulate becomes a political issue and so we look to government to hold the standard.

Standard Bearers

In Norway standards for water-safety are the responsibility of :

NIPH Norwegian Institute of Public Health : oversight and regulation for water quality standards
Mattilsynet Norwegian Food Safety Authority

International Standard Bearers

ISO  - International Organization for Standardization
WHO -  World Health Organization:  GLAAS WHO drinking-water sanitation and hygiene
CEN - European Committee for Standardization
GRI - Global Reporting Initiative
EPA  - Environmental Protection Agency (USA)

Norway is not a member participant in the World Health Organization GLAAS 2018 / 2019 cycle, but it commissions reporting by research foundations within Norway. National standards are upheld to environmental scrutiny from different perspectives: 

Water resources are managed first and foremost as they relate to hydro-electric and transportation infrastructure by the The Norwegian Water Resources and Energy Directorate. This regulatory body does not directly monitor water-quality, but monitors safety of dams, natural disasters due to flooding, power disruptions, earthquakes and landslides, and climate change. The integration of smart technology (IoT) sensors and monitoring is well established for this kind of use and has been promoted with such popular movies as the 2015 drama Bølgen (The Wave). The family-oriented drama/natural-disaster film, starring Kristoffer Joner centers on a team of researchers working at a geological monitoring station. The star forewarned of an impending flood caused by a geological fault disturbance and takes action to save his community (Narrowband Internet of Things (NB-IoT) is an emerging standard for sensors.)

Health and safety standards for water-quality

Somewhat different to water-resources management is alignment of national regulation of water-quality with health and safety standards.

Folkehelseinstituttet and NIPH reports on health and disease related contamination and prioritizes bacterial and directly pathogenic outbreaks: coliform bacteria, intestinal enterococci and Clostridium perfringens as top priorities. Their most recent water quality assessment; Drikkevann.Rapport til Mattilsynet 2016 (Vannrapport 124) emphasizes and reports the rare incidence of outbreaks of disease. The reported incidents in this release is from data collected in 2014. 

The report minimizes the recognition of categorized chemical pollutants that are not actively pathogenic. 

I cannot see that the NIPH enforces regulatory standards for monitoring of non-pathogenic chemical pollutants to a consistent standard or routine. The 124 report does report  “anomalies to the target formulations” in just a handful of locations where high levels of fluoride, iron, aluminum, manganese and total organic compounds (TOC) exceeded target levels. 

Is there a lack of national standardization and regulated enforcement for the inorganic chemical categories? The data is apparently provided by an implicit expectation of local "self-reporting", presumably by the local waste water treatment plants (WWPS) instead of according to a nationally enforced standard. The Folkehelseinstitutt (NIPH) report says as much; “It should be mentioned that most of these parameters do not pose a problem in the Norwegian water supply, and that therefore many water utilities have not done analysis for some parameters or that very few tests have been performed.” (p. 7)

These are the other categories that are referred to as monitored by way of an unregulated “targeting”:

1,2-dichloroethane, Ammonium, Antimony, Arsenic, Benzene, Benzo (a) Pyrene, Lead, Boron, Bromine, Cyanide, Glycols, Hydrocarbons and mineral oils, cadmium, chloride, copper, chromium, mercury, sodium, nickel, nitrate (NO3 -N), Nitrite (NO2 -N), Pesticides, Polycyclic aromatic hydrocarbons (PAH), Radon, Selenium, Sulfate, Tetrachloroethene and Trichloroethene, , Trihalomethane and Tritium.

Chemical and inorganic pollutants are the subject of several of the international safety standards such as WHO GLAAS.

The Vann-Nett Search 
accesses water source data
from all Water regions in Norway

Norwegian law establishes Water Region Committees composed of representatives of the water region authority and other county municipalities, county officials, as well as other relevant sector authorities and municipalities. Gjøvik and Mjøsa lay within the Glomma Water region. Reporting and research is collated via the Norwegian Environment Agency; Miljødirektoriat and some very detailed information and data about local water quality monitoring can be searched on Vann-Nett. Vann-Nett is owned by the environmental administration and the Norwegian Water Resources and Energy Directorate (NVE). The detail mapping collected data from all of Norway are searchable from the category levels of Water Region, Water Area, Watercourse Area in a sidebar menu. There is also a definition of a Catchment region which is not functioning. These categories represent different ways of searching the source data collection which includes data from various research institutes. The site's information architecture allows for customizable report generation.

Of particular interest to Mjøsa and Gjøvik is water quality info currently available about Hunnselva which runs through NTNU campus at Mustad. Mjosa is bounded by 7 komune and as of 2020 2 fylker *where it had been 3.

The Search window on Vann-Nett prompts for entries of either Place names or Names of waterways. The back-end directory also produces results for a unique waterway ID. Results produce the first five Place Names and then Waterways associated with the place names and their ID number. Selecting a Place or waterway immediately produces detailed results including a global menu of dropdown indexed data stats; including an interactive map zoomed in to the area, place or indicating the waterway, general information, water type, ecological status, pressure, measure, environmental target & archived data on the selected item. This is an example of a geographical organizational scheme aligned with indexed relational database resources. Waterways with a Waterway ID produce a controlled index of statistics. This is more accurately a relational database formatted for web browsing.

A query builder allows for a PDF report on water quality based on customized criteria.

There is a great deal of information here about water quality affecting Mjøsa, particularly with regard to the monitoring of nutrient pollution and ratings for Phosphorus and Nitrogen. Ratings for Hunnselva are especially concerning:

Meanwhile the data on chemical pollutants is less definitive:

The need for data standardization is especially critical to coordinate the sharing and confirming of research data between local entities. While it is presumed that all stakeholders have an interest in keeping local reservoirs pristine as a source for drinking water, recreational activities, irrigation and maritime traffic. The various stakeholders also hold different attitudes and opinions about the criteria and methods used to assess the water quality. Farmers may accept a different standard than fishermen or scientists.

Norwegian Institute for Water Research (NIVA)

The Norsk institutt for vannforskning (NIVA) is a leading foundation that conducts applied research certified by ISO standards and their research is often contracted for by governmental agencies. 

The Research organization NIVA monitors:

• Biological, physical and chemical parameters in watercourses and coastal areas.
• The effects of eutrophication, contaminants, acidification and physical interventions such as watercourse regulations, road construction and mining.
• The effects of climate changes on aquatic environments.
• The flow of nutrients and contaminants from rivers to the coast

AquaMonitor is a NIVA  pilot project utilizing Smart technologies to monitor waterways on Inner Oslo Fjord, Outer Oslo Fjord and Langtjern in Buskerud County.

Data collection waypoints are linked to Vannmiljo interactive maps include samples collected by NIVA in 2011 on Mjøsa. The system features many levels of navigability in a interface based on a geographical organizational scheme centered on zoomable maps and detail that can display many aspects of the a selected map area.

Vannmiljø interactive zoomable map view of Mjøsa water sample collection points near Gjøvik.

Data collection points are GPS located on Mjøsa with detail for where and when the samples were collected.  Categories for data labeling is a back-end or research database convention. Collaborating researchers can compile and share data with the Miljødirektoratet according to their controlled vocabulary metadata conventions. Excel based tables with the following category codes have an established precedence on research collected by NIVA and would be a smart vocabulary convention to follow for collaborative research.

Vannlokalitetskode   (WaterLocationCode)
Vannlokalitetsnavn   (WaterLocationName)
Aktivitetsnavn           (EventName)
Oppdragsgiver         (Principal)
Oppdragstaker         (Contractor)
ParameterID              (ParameterID)
Parameternavn         (ParameterName)
Mediumnavn             (MediumName)
Vitenskapelignavn    (Scientific Name)
vetakingsmetodeID 
AnalysemetodeID       (AnalysisID)
vetakingstidspunkt    (DeviceName)
Enhetsnavn                (PlaceName)
Ovre_dyp                    (UpperDepth)
Nedre_dyp                  (LowerDepth)
Deteksjonsgrense     (DetectionLimit)
Kvantifiseringsgrense  (quantification) (UnitOfMeasure)
Opprinelse                 (Origin)
Kommentarer             (Comments)
SistEndretDato          (LastChangedDate)

My research makes the case for corresponding standardization of more of the widely recognized chemical pollutant parameters in the back-end or research data-collection efforts. Controlled vocabularies are a strategy for collaboration based on data standardization. Syntax of metadata labeling is a consideration at the time of research data collection. Data analysis programs such as SPSS sort data fields with labeling as in the examples above. This kind of classification scheme is a backend tool used by authors and indexers for organizing and tagging documents (Rosenfeld, p. 279)

Gjøvik Kommune

Water quality in Gjøvik  is presented with aggregated and specific ratings of 60 water bodies within the boundaries of the municipal district. The data comes from both high precision instruments and  intermediate to low precision sensors.  The aggregated data presents a generally good picture, while the high precision data shows results that show water quality that includes all levels.  Specifically,  Hunnselva running through Raufoss and Gjøvik to Mjøsa currently exhibits the most consistently "Very Bad" levels of Nitrogen and phosphorus, conditions for fish and bottom fauna, and high levels of Polycyclic aromatic hydrocarbons and mercury.  The aggregation of results from all 60 sources tends to dilute or mask the sources of the most serious problems. Point source measurements are monitored and precise data is available but collected inconsistently.

Smart-city Technology

Smart technology is understood as enabling more efficient use of resources and instrumental for monitoring negative consequences of waste, runoff and pollution. Private sector business development is rated lower than environmental protection as a success criteria for the Kommune’s smart-city program. I think this can be interpreted as reflective of the attitudes about the proper place of government and state intervention.

Collaboration with local educational institutions was identified as among the most important criteria for succeeding with a smart-city effort. The results reflect that the local governance sees an important differentiation of roles between state support for standardization and the vital part that open and independent research and development can take place. Lack of digital standards is stated as the most important barrier or challenge in connection with the implementation of the smart city- initiative.

"Observations on water resources concern a wide variety of parameters, covering both supply and quality. However, it is conventional for each agency, programme or project to manage its own list of terms denoting observation parameters, usually with little or no reference to those maintained by other organizations."(Cox, Simons and Yu, 2014)

Criteria for measuring program gains were typically tracked for each project if not for more long term program level gains. Gains were not only attributed to revenue generating profit centers such as parking or fee collection, but also environmental metrics.Environmental consequences rated #3, after user satisfaction and efficiency gains.

A Serious Incident at Skreia

Untreated sewage right out into Mjøsa

The waste-water treatment plant at Skreia was completely destroyed by a fire on December 18, 2018.  All sewage, estimated at about 50 m3 per hour, was then sent uncleaned into Mjøsa. 3 months after the accident 100,000 m3 of uncleaned sewage has been discharged into the lake. Locally, this is a very serious incident that has affected the environment and water quality in Totenvika and adjacent areas.

It may seem conservative to introduce quality assurance (QA) standardization in research practices at a time when some would consider an investigatory, activist or whistleblowing effort more appropriate. What is the proper response of researchers to a serious incident of non-compliance or negligence?

In fact the local authorities are taking investigation very seriously. The need for quality and rational assessment in the case of non-compliance or failure of safety-critical systems remains. At the same time public disclosure and accountability are needed.

Stricter water standards also protect consumers against fraud. An example of consumer fraud is unreasonable fear spread about contamination as a ploy to sell expensive home filtration systems. (CBC, 2020)

https://www.totensblad.no/2020/nyheter/fant-bakterier-ber-alle-i-ostre-toten-koke-drikkevannet/

Water Quality Report Mjøsa 2019

Freshwater microplastic in Mjøsa

A comprehensive study Freshwater microplastic in Norway was made by NIVA - Norwegian Institute of Water Quality in 2018 (before the accident) which established 20 sample monitoring locations at key places on the waterway. The study is an excellent model for data collection and standardization for several reasons. The data is relatively unambiguous and therefore the operationalization of the research is of a high quality.  Solid plastic fragments found in sediment samples are more readily quantifiable or countable and identifiable than diluted soluble chemical concentrations that are often offset against flow rates or presented as time dependent risk variables.

Experimental design for youth activities need to be somewhat rudimentary or illustrative about research design. A variety of basic experimental designs based on latin squares and limited defined variables will be most instructive. The Mjøsa water quality study on microplastics is a good model research design because the material data collection and sample collection points are stable (defined).  Interpretation requires some calculation and work with tables, but (hopefully) not overly difficult or murky quantifications or analysis.

Information Security and Verification

At different stages of the research data security will be more and/or less critical. Data credibility is related to its integrity and stability. The extent that collection of sample and research data is handled methodically, carefully and according to high security standards, the measure of protection of these efforts. At different levels of an (Iot) enabled implementation there are protocols and propriety to be observed with regard to security. A sensor on a farm irrigation system is a different concern than a home security system or a water quality monitoring system at a water treatment facility. Responsibility for data breaches or maintenance of data security at the level of device manufacturer, connectivity provider, integrator, private business, consumer or organization could all be relevant to varying degrees.

(RDBMS) Relational Database Management Systems

File and directory based systems are often based on custom software apps for preforming tasks. Relational Database Management Systems provide for storage, query and data transformation as well as simultaneous access can be made available to relevant data stakeholders.

Design implementations need to be approached in a coordinated way to support interfaces and apps that are current, i.e built on a database architecture that is more stable than the gadget and interface marketplace.

Optical Measures and Electricity

Two surrogate measures that can be used to monitor water quality are conductance and turbidity. Specific conductance; a measure of a water sample's conductivity of electrical current can be used as a measure for dissolved solids and concentrations of ions such as nitrate, sulfate and chloride among others. Turbidity is an optical measure of the scattering of light that can be used to detect phosphorus, total nitrogen, and the presence of fecal coliform bacteria. Dissolved pollutants are detected better with conductance while particulate pollution is detected better with tests of turbidity. Both are indirect analagous or surrogate measures of toxicity or pathogenicity. (Horsburgh et al., 2010, p.2)  Water level, rate of discharge are also variables of measure that could be useful.

Surrogate data testing (based on constrained transformations or realizations) is a statistical proof by
contradiction technique and similar to bootstrapping used to detect non-linearity in a time series. The technique basically involves specifying a null hypothesis Ho describing a linear process and then generating several corresponding representative data sets for comparison to the Ho.

"The inverse problem for a non-linear system is to determine the nature of the underlying dynamics (is it in the practical situation where all that is available is a time series of the data" (Testing for nonlinearity in time series, (Theiler, 1992)

Of critical interest here is the the transformation of anomalous incidents (of pollution) into linear profiles.
Transformations and data analysis or data cleaning routines can identify some invalid data spikes due to faulty sensors, or "noise". Linear re-modeling of results will tend to mask discrete incidents of high discharges from point sources (or unidentified dumping), while the transformations ar arguably better suited for long term monitoring of non-point source pollution.

Data Standardization : some measures are more meaningful than others

The determination of water quality levels is standardized to formulations for individual substances and in some cases proportional and aggregated levels. The Canadian Water Quality Index (WQI) consists of three measures of variance including Scope, Frequency and Amplitude. Synthesized and simplified indexes can obscure or dilute the seriousness of incidents of pollution. Meanwhile, a simple scale can communicate results in an easy-to-read to people who may not be scientists. 

Data Standardization generally deals with the transformation of datasets after the data is pulled from source systems and before it’s loaded into target systems. In many cases it will be necessary to adhere to data processing workflow that converts the structure of disparate datasets into a Common Data Format. In this case we are likely considering how we will handle measurement of pollutants. The amount of a pollutant needs to be standardized to an acceptable health risk level. (In other words, the specific tolerance or toxicity varies from substance to substance, and so standardization is the establishes a baseline for acceptable health levels for each substance)

Data standardization also refers to some data coding and formatting practices, for example p-value standardization. The terms normalization and standardization are sometimes used interchangeably, but they usually refer to different things. Normalization usually means to scale a variable to have a values between 0 and 1, while standardization transforms data to have a mean of zero and a standard deviation of 1

References

Cox, S., Simons, B. and Yu, J. (2014). A Harmonized Vocabulary for Water Quality. Environmental Information Systems, CSIRO, Highett, Australia.

Engebretsen, M. (2017). Levende diagrammer og zoombare kart. Norsk medietidsskrift, 24(02), pp.1-27.

Halpern, O., Mitchell, R. and Geoghegan, B. (2017). The Smartness Mandate: Notes toward a Critique. Grey Room, 68, pp.106-129.

Horsburgh, J., Spackman Jones, A., Stevens, D., Tarboton, D. and Mesner, N. (2010). A sensor network for high frequency estimation of water quality constituent fluxes using surrogates. Environmental Modelling & Software, 25(9), pp.1031-1044.

Haugen, C. (2019). Urenset kloakk rett ut i Mjøsa. [online] Eub.no. Available at: http://www.eub.no/meninger/urenset-kloakk-rett-ut-i-mjosa.

Kommunal- og moderniseringsdepartementet (2019). Smarte byer og kommuner i Norge - en kartlegging. R1020566.

Mae.gov.nl.ca. (2019). Drinking Water Quality Index | Water Resources Management. 

Rosenfeld, L., Morville, P, Arango, J.(2015) Information Architecture for the Web and Beyond, O'Reilly, Sebastapol, CA

CBC Marketplace, Reverse osmosis water purification scam: Hidden camera investigation. (2020). Retrieved 23 January 2020, from https://www.youtube.com/watch?v=GcjhRBflVGQ

Solheim, A., Thrane, J., Skjelbred, B., Håll, J., Økelsrud, A. and Kile, M. (2018). Operational monitoring of Lake Mjøsa. Norwegian Institute for Water Research: Annual report for 2018. Norsk institutt for vannforskning.

Tyto (2018). Show 71 - The Internet of Things. [podcast] The csuite podcast. Featured Insights, The Hype Reports. Available at: https://tytopr.com/iot-csuite-podcast/.

Vaagaasar, A., Skyttermoen, T. (2017). Prosjektveilederen. Oslo: Cappelen Damm akademisk

Østerberg, A., Olsen, A., Kurdøl, E., Amundsun, H., Aagaard, H., Winderen, J., Espedal, J., Løkken, L., Stranger, M., Stensrud, M., Jonsbu, T. and Eckhoff, W. (2019). Det Var Jo Ingen Horizont Der : Mjøsa -et kunstprosjekt 2016-2018. Lillehammer: Oppland fylkeskommune and Hedmark fylkeskommune.



#zones #populations #optimization #resiliece