From this form, you can create or change a WebHook.
General information
If the number of errors exceeds the configured threshold for the given time frame, the WebHook will be automatically disabled and a warning email will be sent to the designated manager.
All failures will be plotted and visible in the WebHook Failures tab.
The WEBHOOK_MAIL_TMPL setting of the "Data exchange" module allows you to define the message template used to send the email.
The WEBHOOK_ERROR_HORIZON setting of the "Data exchange" module allows you to define the failure conservation time.
Currently, failures that are too old are deleted via an automatic job.
It is only initiated when new failures are added.
Failures may therefore remain present over a longer period than foreseen in the horizon.
The purpose of the mechanism is above all to avoid an indefinite accumulation of failure traces.
However, they can then be deleted manually in the Failures tab.
"Entity" type WebHooks
This WebHook type is used to send a message in JSON format to an external system during a CRUD (Create, Change, Delete) event on entities.
Definition
The WebHook definition is a JSON-formatted description that makes it possible to indicate:
Since a WebHook can relate to several types of entities, the definition is a list of descriptions containing this information: which entities, for which events, send a message with which attributes.
The definition has the following format:
{
"entities": [
{ <entité / évènements / attributs> },
{ <entité / évènements / attributs> },
{ <entité / évènements / attributs> }
]
}
Each entity descriptor has the following format:
{
"entity": [
<criteria>, ...
],
"events": [ <events>, ... ],
"attributes": {
"attribut1": "${expression1}",
...
}
}
Entity
The "entity" first part serves to select the entities for which you want to be notified.
It is a list of criteria ("matchers") which are assessed in the stated order and which must all be validated for the notification to be sent.
There are 3 currently types of criteria:
| Type | Example | Description |
| typeHierarchy | { "match": "typeHierarchy", "value": "box" } | Is used to select all entity types in a type hierarchy with a given root. For example, for "box": structure points (box), and FM types: space, floor, room, etc. |
| exactType | { "match": "exactType", "value": "box" } | Is used to accurately select only the entities of the given type. For example, "box" will only select structure points, but not spaces, floors, rooms, etc. |
| attributeValue | { "match": "attributeValue", "attribute": "use", "value": "TECHNICAL" } |
Is used to select the entities for which the indicated attribute has the specified value. The attribute name must be a single attribute name supported by the entity. The value must be a character string in the internal format used by CARL Source to represent values of various types (e.g. in XML loading files). For the time being, the only available operator is the equality. |
Events
This part allows you to indicate the type of events on the selected entities for which you want to be notified: entity creation (created), entity change (updated), entity deletion (deleted).
If this part is not specified ("events" not included or null), all event types will then be notified.
Attributes
This part is used to list the attributes of the entity that will be included in the notification message in the following form:
"<attribute name in the JSON message>": "38013"
The attribute name is any string.
The expression must be of the following form "${entity.attr}".
It is possible to chain the expression on several levels: "${entity.attrA.attrB}".
To obtain information on the type of event that produces the notification, a dedicated attribute with the following expression can be used: "${event}", which will have the values: "created", "updated", or "deleted".
It is also possible not include any expression and to indicate directly the value that will be included in the JSON message. For example:
"attrA": true,"attrB": 5,"attrC": null
Security
On "Entity" WebHooks, the generation of a single authentication token can be requested.
This token acts as a secret shared with the external system receiving the query from the WebHook.
If a token is defined on a WebHook, an X-CS-HMAC-SHA256 http header is added to the query.
This header contains the hash (encrypted element) of the JSON message sent.
Upon receipt, the external system is free to produce a hash of the received JSON using the token and the same algorithms (HMAC + SHA256) and compare the result with the one stored in the header.
If the results are the same, the origin of the received JSON is then authenticated, as otherwise it may be rejected.
HTTP headers
Some details on the definition of HTTP headers:
Definition example
{
"headers": [
{ "name": "some_header", "value": "something" },
{ "name": "my_header", "value": "value_1" },
{ "name": "my_header", "value": "value_2" }
],
"entities": [
{
"entity": [
{ "match": "typeHierarchy", "value": "mr" }
],
"events": ["created", "updated", "deleted"],
"attributes": {
"event": "${event}",
"code": "${entity.code}",
"description": "${entity.description}",
"lastModTime": "${entity.modifyDate}",
"eqptBroken": "${entity.eqptBroken}",
"expectedAmount": "${entity.amount}",
"delay": "${entity.delay}"
}
}, {
"entity": [
{ "match": "exactType", "value": "room" },
{ "match": "attributeValue", "attribute": "use", "value": "TECHNICAL" }
],
"events": [ "deleted" ],
"attributes": {
"code": "${entity.code}",
"description": "${entity.description}",
"lastModTime": "${entity.modifyDate}",
"projectGuid": "${entity.bimGuid}",
"area": "${entity.area}"
}
}
]
}
Example of a message sent
{
"data": [
{
"type": "mr",
"id": "183368af405-c1",
"attributes": {
"event": "updated",
"code": "000003",
"description": "test",
"modifyDate": "2022-09-13T15:01:28.851+02:00",
"eqptBroken": false,
"amount": 0.0,
"delay": null,
"symptom": "ALERT",
}
},
{
"type": "mr",
"id": "18014fc38127-5a",
"attributes": {
"event": "created",
"code": "000007",
"description": "test 2",
"modifyDate": "2022-09-13T15:01:29.583+02:00",
"eqptBroken": true,
"amount": 5864.0,
"delay": null,
"symptom": "ALERT",
}
}
]
}
"ArcGIS" type WebHooks
This WebHook type goes further than a simple notification and allows the data of CARL Source entities to be synchronized with those of a Geodatabase of an ArcGIS server through a FeatureService and its REST API applyEdits.
As with a WebHook on entities, CARL Source will send notifications (synchronization) as entities are changed but there are a number of differences.
For these WebHooks:
Definition
The definition in JSON format is very similar to that of an entity WebHook and describes a set of "layers" of the FeatureService and for each layer, which entities will be synchronized with these layers.
{
"layers": [
{ <entité / layer / attributs> },
{ <entité / layer / attributs> },
{ <entité / layer / attributs> }
]
}
Each entity descriptor has the following format:
{
"entity": [
<criteria>, ...
],
"layer": { "id": <digital id>, "name": "<layer name>", "geometryType": "POINT|LINESTRING|POLYGON" },
"attributes": {
"attribut1": "${expression1}",
...
}
}
The parts relating to the selection of entities and the list of exported attributes are the same as those of conventional entity WebHooks.
The part "layer" is used to indicate the layer identifiers with which the selected entities should be synchronized.
These identifiers are important for CARL Source to ensure synchronization. CARL Source keeps a synchronization status for each synchronized entity, allowing it to determine whether the entity has already been synchronized, and with which layer. This is needed in order to appropriately translate entity events (create entity, add geometry, delete geometry, etc.) into appropriate operations for the ArcGIS REST API.
If the FeatureService on which a WebHook is based is republished, the layer.id numbers may change. It is then essential to update them in the WebHook definition to avoid errors. However, the same layer.name must always used since it is this name that is kept by CARL Source to manage the synchronization status of entities. If the names change, then complete resynchronization must be forced for the WebHook via an automatic job.
When republishing a FeatureService, it is best to disable the WebHook before, update it after republishing, re-enable it, and finally use the automatic job to synchronize the entities changed in the time interval when it was inactive.
Note that layers can only contain one geometry type, and therefore the geometry type is also used to filter the entities synchronized with the layer. An asset with a "polygon" geometry type will not be synchronized with a layer targeting assets if this layer has a "line" geometry type. If no layer matches the filter and geometry type, the entity will not be synchronized even if it has a geometry. If an entity type can have several geometry types, several declarations for several target layers are required.
Security
On "ArcGIS" WebHooks, the login / password pair can be indicated.
This information is used to request the generation of a token from the ArcGIS server.
This token is then associated with queries to the mapping server, which can check its validity and ensure the legitimacy of the flow from CARL Source.
ArcGIS FeatureService layers
To be able to automatically initialize a WebHook from the definition of a FeatureService, layers must be named with the following prefix, in the same way as for the automatic initialization of maps: CARL_<entity type>_... . These layers will be automatically imported, and the others will be ignored.
Layers must also have following fields:
| Field name | Type |
| ENTITYID | Text (33) |
| ENTITYCLASS | Text (255) |
| GlobalID | GlobalID / with a unique index. |
Finally, layers and the FeatureService must have the following capability (which can be seen in the FeatureService description): "Supports ApplyEdits With Global Ids: true". While this capability is often dependent on the inclusion of the GlobalID field with a unique index, other restrictions may apply for this capability to be active. For this purpose, you have to refer to the ArcGIS documentation.
Definition example
{"layers": [
{
"entity": [ {"match": "typeHierarchy", "value": "mr"} ],
"layer": { "id": 0, "name": "CARL_MR_POINT", "geometryType": "POINT" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "wo"} ],
"layer": { "id": 1, "name": "CARL_WO_POINT", "geometryType": "POINT" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "wo"} ],
"layer": { "id": 2, "name": "CARL_WO_LINE", "geometryType": "LINESTRING" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "wo"} ],
"layer": { "id": 3, "name": "CARL_WO_POLYGON", "geometryType": "POLYGON" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [
{"match": "typeHierarchy", "value": "material"},
{"match": "attributeValue", "attribute": "eqptType", "value": "VHL"}
],
"layer": { "id": 4, "name": "CARL_MATERIAL_POINT", "geometryType": "POINT" },
"attributes": {
"xtraTxt01": "${entity.description}",
"xtraNum01": "${entity.replacementAge}",
"xtraDate01": "${entity.lastInventory}"
}
},
{
"entity": [ {"match": "typeHierarchy", "value": "material"} ],
"layer": { "id": 5, "name": "CARL_MATERIAL_LINE", "geometryType": "LINESTRING" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "material"} ],
"layer": { "id": 6, "name": "CARL_MATERIAL_POLYGON", "geometryType": "POLYGON" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "box"} ],
"layer": { "id": 7, "name": "CARL_BOX_POINT", "geometryType": "POINT" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "box"} ],
"layer": { "id": 8, "name": "CARL_BOX_LINE", "geometryType": "LINESTRING" },
"attributes": { "xtraTxt01": "${entity.description}" }
},
{
"entity": [ {"match": "typeHierarchy", "value": "box"} ],
"layer": { "id": 9, "name": "CARL_BOX_POLYGON", "geometryType": "POLYGON" },
"attributes": { "xtraTxt01": "${entity.description}" }
}
]}
Automatic synchronization job with an ArcGIS server
When a WebHook is active, it synchronizes the entities with the ArcGIS service. However, in the event of an error (server unavailability, network outages, etc.), some entities may not be synchronized correctly. Similarly, when a synchronization is set up on an existing database, it must be possible to synchronize the existing entities before it is enabled.
For these use cases, CARL Source can be used to add an automatic job (job class: ARCGISWEBHOOKJOB) to initiate (manually or programmed) synchronization of certain entities. The entities that will be (re)synchronized are those that have been changed between the two dates indicated in the job settings, or if none is specified, all the entities (for example for an initial synchronization).
In order to function, the synchronization job uses the definition of a WebHook, which must be specified and be in the "Active" or "Inactive" state (in this state the synchronization is disabled but the WebHook is valid and usable by the job).
In addition to the dates, the entities synchronized by the job can also be restricted by listing the types of entities to be synchronized. This is a comma separated list. If it is empty, all entity types specified in the WebHook will be synchronized.
As default, the job assumes that the entities' synchronization states are valid. The entities in question have simply not been synchronized (e.g. WebHook inactive). If there are inconsistencies, then "Force resynchronization". In this case, the synchronized data is deleted (synchronization status in CARL Source, and entity data in the ArcGIS FeatureService), so as to start from scratch again for these entities, and these entities are then synchronized again.
Job time
The job time is obviously proportional to the number of entities to be synchronized. For this purpose, it is recommended to limit their number by specifying a date interval, or by limiting the types of entities to be synchronized.
The "Force resynchronization" setting also has a very significant impact on the job time since it requires deletions to be made beforehand for each entity.
Job periodic programming
From CARL Source version 7.1.0 onwards, a new setting entitled "Entities changed since this automatic job was last run successfully" is available for automatic jobs associated with the ArcGisWebHookJobBeanclass.
The purpose of this setting is to facilitate the periodic scheduling of the job.
Indeed, in the context of automatic running, it is generally not desirable to set a time range or to resynchronize all the entities; however, it is useful to be able to synchronize only the entities changed since the last successful running of the job.
This setting, if checked, allows the system to set the appropriate start date corresponding to the date of the last successfully run job. In this way, the job time window shifts as the runs take place.
If a start date is already specified for the job, checking this setting will cause it to be replaced by the automatically calculated date.
Effect of WebHooks on performance
WebHooks have been developed with the aim of minimizing the effect on the application's performance. When they are active, however, they necessarily add additional jobs.
A few measures have been taken to avoid negative impacts:
In all cases, WebHooks are monitored by performance indicators that are available in the "performance report".
Debugging WebHooks
A trace profile has been added to enable detailed traces on WebHooks.
Its name is "WH_DEBUG.
To facilitate diagnostics, a webhooks.json file containing WebHooks definitions has also been added to the Performance Report with the following indicators: