CtrlJsonSerialization - Proposal of Ctrl Extension for JSON serialization and deserialization Gialousis Miltiadis Suder Mariusz BE-ICS-FD - CERN ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CtrlJsonSerialization
Proposal of Ctrl Extension
for JSON serialization and deserialization
Gialousis Miltiadis
Suder Mariusz
BE-ICS-FD
26/05/2020 CtrlJsonSerialization 1
Motivation
Have you ever …
Needed to store complex/compound data structures ?
• fwTrending needs to store vast configurations of trend pages that consist of trend areas, which
in turn contain trend curves, with each of these having numerous properties that need to be
editable.
• New AlarmScreen needs to store the configuration of filters, filtering widgets, layouts of the
alarm table columns, etc.
• New UNICOS HMI needs to store "screen layouts" or "profiles", whereby users could save the
content of each preview window with several properties, and then load/save/review modify
them.
• Need to return a more complex data structure when the only available option is string or
dyn_string (e.g. PanelOffReturn() which returns a dyn_float or dyn_string)
Always a problem, but much more evident as we step into the consolidation of components like
fwTrending, Alarmscreen as well as the integration of CTRL++.
26/05/2020 CtrlJsonSerialization 2Use Case: fwTrending
Without serialization
Proposed object breakdown • DPT fwTrending_Trend
• dpe string title
• dpe dyn_string areas
• DPT fwTrending_TrendArea
• dpe string title
…
• dpe string autoScroll
• dpe dyn_string curves
• DPT fwTrending_Curve
• dpe string title
…
• dpe bool visible
• Custom API to dpSet/Get all those dps.
JsonSerializer
• DPT fwTrending_Trend
• dpe string fwTrendingTrend
• string json = fwJsonSerialize (fwTrending_Trend)
• dpSet(sPlotDp + fwTrendingTrend, json);
26/05/2020 CtrlJsonSerialization 3Current approach - Shortcomings
• Every property is stored in a dedicated dp-element
• API that allows to manipulate the groups of settings lists each property as a separate
argument in the system call
Even though in some cases this solution is sufficient and performant we can clearly see a lot of
issues with this approach
• Great difficulty in extending or altering the underlying saved properties. Ending up with
overblow DP-Types, with huge number of elements that are rarely ever accessed, long API
function signatures.
• Boilerplate repetition. Each different component developer is obliged to come up with a new
small custom API.
• Inability to store Ctrl++ objects and more (vectors, shared_ptr etc) out of the box.
26/05/2020 CtrlJsonSerialization 4JSON – Reaching solution
Having considered XML, we decided JSON was the way to go, providing encoding of data into a string.
• Well known standard, simple and with concise format
• Simplified structure of the DPT (reduced number of elements)
• Clean interface (ease of exchanging data with Ctrl extensions)
• Allows for future extensibility and compatibility without extra cost/overhead
Choosing Ctrl extension over native jsonEncode, jsonDecode functions.
• Limited support of nested structures (need for recursive encoding/decoding)
• Cannot work out of the box with custom Ctrl++ objects (need for extensive annotations).
• Overhead and difficulty of instantiating the proper data structures when decoding.
Decided to provide our own solution to overcome those limitations and have the possibility to extend it in
the future according to the needs of the JCOP community.
26/05/2020 CtrlJsonSerialization 5API
string fwJsonSerialize (const type &obj|shared_ptr obj, [,
mapping options = makeMapping()],
bool compact = true)
int fwJsonDeserialize (const string &json,
&obj|shared_ptr obj [,
mapping options = makeMapping()])
• Problems reported with exceptions, use try…catch statement
try{
fwJsonDeserialize("{\"a\":4,\"b\":true}", obj);
}catch{
DebugTN(getLastException());
}
• Options mapping
• Already included in functions signatures, to be implemented later
26/05/2020 CtrlJsonSerialization 6Current State – Highlights
• Implemented support for serialization and deserialization of:
• All built-in WinCC OA types used in JCOP framework
• New CTRL++ types shared_ptr, vector and user defined types: classes/structs/enums
• Nested objects of different types and at different depth
• Format of serialized data compatible with jsonEncode() output
• Few exceptions for less common types
• Selection of output JSON format
• Human readable or concise
26/05/2020 CtrlJsonSerialization 7Planned Features
• Implementation of options mapping for serialization and deserialization
• Serialization • Deserialization
• "depth" int • "depth" int
• "noSerialize" dyn_string
• "serializePtrTarget" bool
• Support for type annotation
• shared_ptr of base class with object of derived class
• anytype/mixed variable, array or mapping element
• Function for querying particular value/object stored in JSON document
• int fwJsonPeek(const string &json, const string &query,
&obj|shared_ptr obj, mapping options = makeMapping())
26/05/2020 CtrlJsonSerialization 8Summary & Roadmap Proposal
If accepted by JCOP users community
• First 'preview' version ready to be included in the JCOP framework
• Basic set of features provided – functionality of WinCC OA built-in functions + serialization of complex
structures and deserialization to the output argument type
• Covers use cases of storing/exchanging data structures of known type
• For evaluation in development environment
• API relatively stable, but it may be a subject to change if requested and approved at JCOP forum
• Further development
• Implementing planned features
• Users requests – we count on your feedback!
• Production-ready version to be delivered in autumn
26/05/2020 CtrlJsonSerialization 9home.cern
Considered features
• Other options:
• Deserialization
• "stopAtFirstError" bool
• "strictType" bool
• Possibility of declaring members excluded from serialization inside a class
static const dyn_string _no_serialize = makeDynString("member1", "member2", "member3",
"member4.nested1");
• Class methods for serialization and deserialization – if defined, used by
fwJson[De]Serialize() instead of default behaviour
class Path{
dyn_string path_elements;
public string _serialize(){return strjoin(path_elements, "/");}
};
26/05/2020 CtrlJsonSerialization 11Remarks on storing serialized data
• For member of complex data structure, that needs to be accessed frequently (for
example with dpQuery() or in a dpQueryConnect() callback) it is better not to serialize
it, but rather store it in a separated DPE
• Developer is responsible for proper design
• Use case for noSerialize option
26/05/2020 CtrlJsonSerialization 12Quality assurance
• Automated unit tests in gitlab CI pipeline
• Serialization and deserialization of different types (including nested ones)
• Code coverage: ~75% (for now testing only happy path)
• Manually triggered unit tests, runned with tool to detect memory issues (valgrind)
• Performance tests to be prepared
26/05/2020 CtrlJsonSerialization 13Use Case: ScreenLayout
This is a simplified form of the UnHMI_ScreenLayout
data structure.
The properties of those classes can easily be
extended or modified at any time.
On the right, we can see the debug
output of an instance of the above class
Below we can see the same instance
serialized, in json string format, after
using our CTRL extension.
26/05/2020 CtrlJsonSerialization 14Use Case: Alarm Screen
Alarm Screen Configuration Grouping rows in the table Filter Configuration
{
{ "GroupingConfiguration": { {
"ProxyModelConfiguration": { "columnWidth": 250, "filterRulesSets": [
"AlarmModelConfiguration": { "groupHeader": "Group", {
"AlarmSourceConfiguration": { "groupingRules": [ "filters": [
"fromTemplate": "", { {
"whereTemplate": "“ "alarmPropId": "application", "pattern": "*CAEN*",
}, "groupingRole": "display", "source": "this",
"blockingTime": 2000, "sortAscending": true, "valueId": "dpElement“
"plugins": [ "AlarmPropJCOP" ], "textFormat": "“ },
"embeddedFilters": { }, {
} { "pattern": "*My device description*",
}, "alarmPropId": "domain", "source": "this",
"columns": [ "groupingRole": "display", "valueId": "deviceDescription“
{ "sortAscending": true, },
"id": "Abbreviation", "textFormat": "“ {
"title": "Short Sign", }, "singleValue": {
"backColor": { { "QString": "Local“
"id": "AlertColor“ "alarmPropId": "nature", },
} "groupingRole": "display", "source": "this",
}, "sortAscending": true, "valueId": "scope“
{ "textFormat": "“ }
"id": "dpElement", } ],
"title": "Device DP Element“ ] "include": true
}, } },
…. } {
] "filters": [
}, {
"UserSettings": { "pattern": "*Other interesting devices*",
"showFilterWidget": true, "source": "this",
"showFooterWidget": true, "valueId": "deviceDescription“
"showFilterView": "none", },
"filterSetDp": "_NgAsFilterSet000001", {
"visibleColumns": [ "pattern": "*With an interesting logical name*",
{ "source": "this",
"columnTitle": "Short Sign", "valueId": "alias“
"width": 90, }
"visible": true }, ],
… "include": true
{ }
"columnTitle": "Time", ]
"width": 177 }
}
]
},
"Filter": {
"filterName": "Default filter",
"structureFilter": {
}, "textFilter": "*“
}
}
26/05/2020 CtrlJsonSerialization 15Use Case: Alarm Screen
Filter Configuration for New Alarm Screen Serialized vector
[
{
"filters": [
{
"pattern": "*CAEN*",
"source": "this",
"valueId": "dpElement"
},
{
"pattern": "*My device description*",
"source": "this",
"valueId": "deviceDescription"
}
],
"include": true
},
{
"filters": [
{
"pattern": "*Other interesting devices*",
"source": "this",
"valueId": "deviceDescription"
}
],
"include": true
}
]
26/05/2020 CtrlJsonSerialization 16You can also read
