Convert and Validate Input Data

Last updated:  October 21, 2002

Introduction
This document is one of five that describes the basic ICON program (skeleton.icn) which normally provides the framework for all ICON programs. This document describes the second  module type named "Convert and Validate" which takes the exported raw Modbus data from the first module and performs the following operations on each data point:

  1. Convert raw Modbus data to engineering units. (Often times data is read with an A-D converter and the signal will be a value that varies from 0 to 4095 (for a 12 bit converter). This number might represent temperature where a raw reading of 0 represents -40 degrees F and the full scale A-D value of 4095 might represent +125 degrees F. The Convert instruction is used to make this conversion.)
  2. Bounds check the data for proper range. (It is wise to validate readings, for instance, you don't want to turn your furnace on to heat a room if the temperature sensor is reading -40 because the Modbus device or the sensor has failed.)
  3. Create a "Data Invalid" status variable. This variable will be 0 if the data is valid and 1 if invalid. This status is created from the status of the Modbus read operation and the bounds check.
  4. Average (filter) the data to remove spikes and noise. (Sometimes the measurement has noise. So instead of reading 1024, 1024, 1024, ... for different readings in time for a constant temperature the values in time might be 1020, 1030, 1024, ... A simple way to remove the noise is to average the last X readings. If X=1 then there is no averaging. If X=3, then the last 3 readings are averaged. This operation can be done with the "Circbuf" instruction.)
  5. Provide for data override on a point by point basis. (It is often useful to be able to simulate converted data results for program testing. During normal operation it is sometimes desirable to actually force input results to be a certain value to continue operation during a sensor failure. For example, in greenhouse operation you may be controlling vents and heaters to try to maintain temperature and humidity. If the humidity probe Allow the user to override the Modbus inputs for testing (and create valid Modbus status during the override operation).
An HMI interface is created that allows the user to interface to the above program to:
  1. Set raw to engineering unit conversion coefficients.
  2. Set bounds limits.
  3. Set number of readings to filter.
  4. Force an override value.
  5. View raw, converted and filtered data.
  6. View "Data Invalid" status.
This module can process up to 16 data points from a single Modbus read export. Add additional modules if more than 16 points must be processed from a single exported Modbus array or you have multiple Modbus reads. It is not recommended to do more than 16 in one module because the HMI gets too busy. Name the first module "Convert and Validate 1", the second "Convert and Validate 2", etc. Or you may pick other unique names.

In the following sections the program is listed along with the configuration. The first line is the instruction and caption as would be listed when programming. This is followed by the general and table parameter configuration. Finally, a description of the purpose of the program is included if needed. If the instruction needs a reference on the HMI, the configuration for that HMI object is included.

Module Convert and Validate 1

Map Import raw input data
General Parameters
Array: raw
Direction: To array

Table Parameters
Individual
#data1[2]
#data1[3]
#data1[4]
#data1[5]
#data1[6]
#data1[7]
#data1[8]
#data1[9]
You may enter exported data in any order and skip un-needed values from a single exported Modbus array (in this example #data1). The Modbus read validity status is in #data1. If this is a 1-wire read then the scan counter is in #data1[1]. Hence this example starts at #data1[2] which woud be the first data point. Set the table length to the number of elements you wish to process.

Expression Get Modbus status
Table Parameters
Result: Expression: Description
status #data1 Enter the first element of your selected exported Modus data array

Constant Override enable
General Parameters
Parameter Example Description
Variable: override_enable The variable with the override controls.
Data Type: Float
Mode: Normal

Table Parameters

Set the table length to the number of data points to process. Set the values to 0 (overrides disabled) for the initial condition. Descriptions are not needed. This instruction loads the override_enable variable array elements with the user selected values of 0 (disable override) or 1 (enable override) and are changed by the user from the HMI.

Constant Override values
General Parameters
Parameter Example Description
Variable: override The variable array with the override values for the first Modbus instruction.
Data Type: Float
Mode: Normal Set values on every pass.

Table Parameters
Set the table length to the number of data points to process. Values don't matter, they will be set from the HMI. Set each description as appropriate for that input.

The above 4 instructions are only required once for the module. The following 5 instructions are repeated for each block. There must be one block for each parameter you wish to process (the table lengths specified above).

Convert Sensor 1
General Parameters
Parameter Example Description
Input: raw The raw Modbus value.
Result: sensor_inst The converted value in engineering units.
Select function: Normal Normal interpolating polynomial.
Poly degree: 1 Straight line fit.

Table Parameters

Set your table size to two rows. You can calculate your conversion coefficients by entering two pairs of values. Each pair consists of an input value (the raw A-D reading) and the converted value, for example temperature. In the example below an A-D channel varies from 0 to 4095 for a temperature range of -40 degrees F to +125 degrees F.
Row Conversion coefficient Input value Converted value
1 0 0 -40
2 0 4095 125
After entering the input and converted values click "Update". Then click "Refresh". Now your conversion coefficients should be updated to -40 for the first and 0.04029304 for the second. These are the numbers required to do a 1st degree polynomial conversion of raw data to converted by using the equation:

sensor_inst= -40 + (0.04029304 X #mod_data1)

You could change your table size to include more pairs. For instance if you pick a table size of four rows and enter four known Input-Convert pairs, when you click update a least squares best fit of the four points to a straight line will be calculated. If you would like to fit a quadratic you can pick a 2nd degree polynomial. You must have at least one more pair than the degree of the polynomial. Most inputs should be converted with a 1st degree polynomial. But some inputs like thermocouples require a higher degree.

Once in a while the input to output is a log function. For this case select the function as "log". Usually you will still use a 1st degree polynomial. Now enter your raw-convert pairs. Now the program will take the log of the input and use this number for the least squares fit calculation.

Important note: Before you can do a new least squares fit you must set all conversion coefficients to 0 and change at least one number in the input-convert section.

Bounds check the data limits
General Parameters
Parameter Example Description
Input: sensor_inst Data in engineering units
Result: sensor_inst Data in engineering units clamped to High and Low bounds.
Alarm: sensor_inst[1] The alarm status, set to 1 if data is above High bound alarm or below Low bound alarm. Otherwise set to 0
High alarm: 1
Low alarm: 1
High bound alarm: 105 If the value of sensor_inst is above this value, the value will be flagged as invalid ( a 1) on the status variable sensor_inst[1].
High bound: 100 If the value of sensor_inst is above this value, the value of sensor_inst will be set to this value.
Low bound: 0 If the value of sensor_inst is below this value, the value of sensor_inst will be set to this value.
Low bound alarm: -5 If the value of sensor_inst is below this value, the value will be flagged as invalid ( a 1) on the status variable sensor_inst[1].
Sometimes data can go slightly below or above its valid range but still be OK. Relative humidity is a good example, it should only vary between 0 and 100 but depending on the sensor it might actually go to -5 and up to 105. Therefore, setting Low Bound to 0 and High Bound to 100 will cause the readings to be clamped. However, if the reading is above 105 or below -5 then the sensor can be considered in failure mode. Therefore, setting Low Bound alarm to -5 and High Bound alarm to 105 will cause the "validity" variable (sensor_inst[1]) to be set to a 1 if the sensor is bad (remember inverted logic).

Expression get validity status
Table Parameters
Result: Expression: Description
sensor_inst[1] 1 'if' status The purpose of this instruction is to also set the validity flag (sensor_inst[1]) to 1 if the Modbus instruction failed. The expression can be interpreted as follows, set a value of 1 on sensor_inst[1] if the value of status is not 0, otherwise do nothing to change the value of sensor_inst[1].

Circbuf filter data
General parameters:
Parameter Example Description
Input: sensor_inst Input instantaneous data.
Results: sensor_avg Result is the averaged data. 
Control: Null This causes a new average to be calculated every pass.
Max valid status: 000000h If the status in sensor_inst[1] is greater than 0 the value in sensor_inst will not be used in the average calculation.
Circular buffer length: 60 Allow an average of up to 60 elements.

Table parameters:
Parameter Example Description
Calc type: Average Do an average.
Number in calc: 3 Average the last 3 readings.
Minimum valid: 1 There must be at least 1 valid reading to get a valid result.
As configured above the value in sensor_avg will be the average of the last three readings from sensor_inst. If the loop execution time is 1 second then the average will be the last three readings from this scan and the scans 1 and 2 seconds prior. By changing the "Number in calc" parameter you can change the number of averaged elements. If you pick 1 then averaged data will be the same as instantaneous. If you pick 10 then the average will be the last 10 seconds of data. This instruction assumes a validity status at index 1 greater than the input variable (sensor_inst[1] in this example) and was created in the previous  2instructions. And since the Max valid status is set at 0, any 1 values on  sensor_inst[1] will cause those readings to be rejected in the average. The validity status for sensor_avg will be at sensor_avg[1]. This value will also be 0 if there is at least 1 valid data point in the 3 element average, otherwise the value will be 1 to indicate an invalid average.

Expression Get override value
Table Parameters
Result: Expression: Description
sensor_avg override 'if' override_enable Set override data if override_enable has been set to 1 from the HMI.
sensor_avg[1] 'if' override_enable Set valid override status if override_enable has been set to 1 from the HMI.
sensor_avg  1E38 'if' sensor_avg[1] This line optional and only required if you wish to set the data value to 1E38 to indicate invalid data in addition to the having the status in sensor_avg[1].

Additional Sensor Conversions
Repeat the above instructions for each required block. For the next sensor the raw data will be at raw[1]. The next override enable and value will be in variables override_enable[1] and override[1]. Notice these indexes increment by 1. Your converted data will be in sensor_inst[2] and the validity flag will be in sensor_inst[3]. The average value will be in sensor_avg[2] and the validity flag in sensor_avg[3]. Notice these indexes increment by 2 because of the validity variable. Use this pattern to configure your other instruction blocks.

The following instruction is entered only one time at the end of the module to export the data for use by other modules.
Fan Export converted and validated data
General Parameters
Source: sensor_avg Description
Destination: #sensor_avg1 Use #sensor_avg2 for the second Module, etc. This is the exported data that will be used by other modules.
Count index: Null
Time index: Null
Time def: Null
Last period: Null.
Block size: 8  Set to number of blocks configured above.
Number of indexes: 1
Direction: Multiple sources to one destination

HMI Screen
Next you need to configure an HMI screen that will allow the user at run time to configure conversion coefficients, set bounds limits and filter values. Also the raw, converted instantaneous and converted averaged (filtered) values along with validity status needs to be displayed. The screen will have seven columns with each column being the length of the number of sensors. Column 1 will contain the HMI objects to set conversion coefficients. Column 2 will contain the HMI objects to set bounds limits. Column 3 will contain the HMI objects to set filter values. Column 4 will display the validity state. Column 5 will display the raw Modbus (A-D) values. Column 6 will display the instantaneous converted values.  And finally column 7 will display the averaged (filtered) values.