Cloud Data Push¶

Cloud data push is a cloud-to-cloud API that enables API clients to subscribe to real-time value changes, alarm state changes and events from push sources in SMIP.

Push Sources¶

This documentation focuses on things, which are the primary data push source in SMIP.

API clients may call READ all thing types info API to obtain information about which types of things are available and what data is available for push.

API clients may create subscriptions, start listening to subscriptions, stop listening to subscriptions and close subscriptions during their push life-cycle. This is done in real-time by calling appropriate Push Data APIs and possibly by connecting to available push transports as described in the following chapters.

While SMIP supports different transports for data push (with more possibly added in the future), the main transport is the SMIP’s internal smip_mqtt transport. Most examples in the following chapters reference this transport.

Push Sessions¶

A push session is a “container” that contains multiple push subscriptions. Each API client should create just a few (possibly just one) sessions with multiple subscriptions.

One may call READ session creation info API to obtain all available session types, message types and transport types.

Push session facts:

has a unique id - acts as a secret token and it is used in all API calls and in push transport connections (see also MQTT Client API Access).
has a strict session_type - there may be more session types available to the API client. Sessions may be created empty (see default session type) or with preconfigured subscriptions.
has a message_type - defines the type of push message formats the session uses. See Push Messages for details about default formats.
has a transport_type - defines the transport type that is used to push messages to the API client. There may be multiple available transports in SMIP, starting from the default data push MQTT transport. Other types could for example include various REST transports, external (non-SMIP) MQTT transports etc. Please see Push Transports for details.
has possible push wildcard addresses or topics - these are push addresses or topics that the API client may listen to (depending on the used transport). If for example data push MQTT transport is used, then there will be at least one generic “listen-to-all-messages” MQTT topic wildcard available in the session.
may have many subscriptions - but each push source (thing) may have only 1 subscription per session. Each subscription contains absolute push addresses/topics where data is pushed. API clients may listen to each individual address/topic or preferably to a wildcard address/topic that listens to all of them at once. See Push Subscriptions for details.
has various metadata, metrics and status (OPEN, CLOSE) information
has a well defined life-cycle (see next chapter for details)

A Push Session’s Life-cycle¶

Action/state	API	Description
`CREATE`	CREATE push session API	A new push session is created. Depending on the transport type, sessions may be created in `OPEN` or `CLOSED` state.
`OPEN`	The API client connects to push transport (if applicable).	This depends on the transport type. For REST transports a session is always `OPEN`. When using data push MQTT transport or similar transports, a session becomes `OPEN` when the API client actually connects to MQTT broker. See also MQTT Client API Access.
Subscription `CRUDs`	Push Subscription APIs	Push Subscriptions may be Created, Read, Updated and Deleted in real-time while a push session is `OPEN` or `CLOSED`.
`CLOSED`	The API client disconnects from push transport (if applicable).	This depends on the transport type. For REST transports a session is never `CLOSED`. When using data push MQTT transport or similar transports, a session is always created `CLOSED` and must be `OPENed` first by connecting to appropriate MQTT broker. When API client disconnects from MQTT broker, the session becomes `CLOSED` again. `CLOSED` sessions may expire (are automatically `DELETEd`) if they are not `OPENed` in due time.
`DELETE`	DELETE push session API	Push session with all of its subscriptions is removed from SMIP. This may happen automatically when a session expires.

The “default” session type¶

The default session type defines sessions that are created empty by default and API clients are expected to add subscriptions by calling appropriate APIs.

Subscriptions may be added, altered and removed in real-time while the session is OPEN or CLOSED.

Push Subscriptions¶

A push subscription contains information about a push source, which data is being pushed and to which destination address/topic is being pushed. Push subscriptions may be updated/changed in real-time depending on API client’s requirements.

Each push subscriptions belongs to exactly one push source. There may be only one (or no) subscription for the same push source in a push session. The same push source may have multiple subscriptions, but each one must be in a different push session. Such subscriptions may have different push metadata (different listeners may listen to different variables, alarms and events from the same push source).

It is important to understand the relation among SMIP push subscriptions and MQTT (and similar) subscriptions. Push subscriptions are SMIP “constructs” that are always created by calling appropriate Push Subscription APIs . However, depending on the push transport used, a push subscription will expose “native” MQTT (or similar) addresses / topics that a subscription actually uses to push messages to.

Push subscription facts (thing push source scenario):

it is contained in a session
it represents one thing (push source) and contains the thing’s namespace and local id like:
- TI-f6ab19b0-20ed-41ef-99cc-40bf287f25b7 and rgb_light_01
contains a list of thing’s variable names that are being pushed like:
- temperature, consumed_power, on_off_state etc.
defines the push address/topic for real-time variables like:
- p/data_api/v3_0/{SESSION_ID}/t/TI-f6ab19b0-20ed-41ef-99cc-40bf287f25b7/rgb_light_01/rt_data
contains a list of thing’s alarms that are being pushed like:
- temperature_high, temperature_low etc.
defines the push address/topic for alarms like:
- p/data_api/v3_0/{SESSION_ID}/t/TI-f6ab19b0-20ed-41ef-99cc-40bf287f25b7/rgb_light_01/alarms
contains a list of thing’s events that are being pushed like:
- door_open, motion_detected, key_used etc.
defines the push address/topic for events like:
- p/data_api/v3_0/{SESSION_ID}/t/TI-f6ab19b0-20ed-41ef-99cc-40bf287f25b7/rgb_light_01/events
has internal metrics and status (OPEN, CLOSED) information

A Push Subscription’s Life-cycle¶

SMIP provides different APIs for subscription CRUD operations. While it is possible to add separate subscriptions by calling the API per each desired thing, the preferred approach is to add multiple subscriptions by making fewer calls to appropriate bulk API’s instead.

One may create subscriptions by directly providing absolute thing ids (namespace + local id). This requires the API client to know the exact absolute id of each thing, which may not be feasible in many cases.

If things are not known in advance, one may use known instance ids and find things by using thing selectors, e.g., calling the appropriate API with a query like: subscribe to all RGB lights in smart-home instance 123. Please see Examples and Thing Selectors on how to do that.

Life-cycle:

Action/state	API	Description
`UPSERT`	see Push Subscription APIs	A new push subscription is created or updated. Depending on the transport type, subscriptions may be created in OPEN or CLOSED state.
`OPEN`	The API client subscribes to appropriate MQTT topic (if applicable).	This depends on the transport type. For REST transports a subscription is always `OPEN`. When using data push MQTT transport or similar transports, a subscription becomes `OPEN` when the API client actually subscribes to an appropriate MQTT topic (wildcard or absolute). Push source (thing) starts pushing requested data to subscribed API client. See also Topic Subscription Authorization.
`CLOSED`	The API client unsubscribes from the related MQTT topic(s) (if applicable).	This depends on the transport type. For REST transports a subscription is never `CLOSED`. When API client unsubscribes from the related topic(s), the subscription becomes `CLOSED`. Push source (thing) stops pushing data to the unsubscribed API client.
`DELETE`	see Push Subscription APIs	Subscription is `CLOSED` and removed from the session.

Push Transports¶

A push transport defines the native “transport/protocol” trough which data is pushed from a push source (thing) to its destination, i.e., the API client. Examples of possible transports that could be used for push are: REST, MQTT, AMQP, various JMS solutions, Apache Kafka, DB SQL writes etc.

Some transports must be preconfigured while other transports work out of the box. The default integrated transports in SMIP are smip_mqtt and preconfigured REST push transports.

A REST transport for pushing to host www.some_cloud.com for example, must be preconfigured in SMIP (host, port, access credentials etc.) and given a new id like rest_to_some_cloud_com. On the other hand, the smip_mqtt transport works out of the box, since API clients connect to SMIP’s push MQTT broker as documented in MQTT Client API Access.

When a session is being created, the API client chooses among supported transports. One can obtain all available transports by calling READ session creation info API.

The “smip_mqtt” transport¶

The smip_mqtt transport uses a SMIP hosted MQTT broker that API clients may connect to and receive push messages from. It may be used out of the box with the same API credentials as used for SMIP REST Data APIs.

Authentication and authorization to smip_mqtt is described in MQTT Client API Access.

The smip_mqtt transport provides the following global wildcard MQTT topics:

p/data_api/v3_0/{SESSION_ID}/# - all messages from all subscribed things (includes also wildcards below)
p/data_api/v3_0/{SESSION_ID}/t/+/+/rt_data - all real-time data messages from all subscribed things
p/data_api/v3_0/{SESSION_ID}/t/+/+/alarms - all real-time alarms messages from all subscribed things
p/data_api/v3_0/{SESSION_ID}/t/+/+/events - all real-time event messages from all subscribed things
where {SESSION_ID} in above topics represents the actual session id.

These MQTT topics are also returned when calling CREATE push session API and READ push session info API.

The smip_mqtt transport provides the following per-thing MQTT topics:

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/rt_data - all real-time variable changes per specific thing
p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/alarms - all real-time alarm changes per specific thing
p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/events - all real-time events per specific thing
where {SESSION_ID}, {THING_NAMESPACE} and {THING_ID} in above topics represent actual values from session and thing subscription.

These MQTT topics are also returned when calling various Push Subscription APIs.

Push Messages¶

Each push message originates from a single push source (thing) and contains only data from the said push source. Each message is pushed to an appropriate push address (e.g., MQTT topic), which belongs to its push source (see smip_mqtt transport for topic examples).

Kinds of push messages:

real-time messages - contain values of real-time variables, e.g., temperature=22.5, switched_onoff=1
alarm messages - contain real-time alarms, e.g., temperature_high
event messages - contain real-time events, e.g., motion_detected, key_used

SMIP supports multiple push message types (formats). The API client chooses the desired type when it creates a new push session by calling CREATE push session API.

By default SMIP uses predefined message types named default and default_verbose which are described in the following chapters. Other formats may be developed and added to SMIP as needed.

To obtain all available message types, one may call READ session creation info API. When creating a new push session, one must add the desired message type in the request of CREATE push session API.

“default” messages¶

Default messages are JSON messages that contain pushed data and additional metadata. Commonly used properties are described below:

id - the unique message id
timestamp - the ISO 8601 message timestamp with added milliseconds, i.e., yyyy-MM-dd'T'HH:mm:ss.SSS'Z'.
data_state - the state of the pushed data which may be:
- ALL - all subscribed variables or alarms are present in the message. This is usually sent right after the subscription becomes OPEN or right after the subscription has been changed. SMIP will make best effort to push the current state of all subscribed data before it starts pushing only data changes.
- CHANGES - message contains only subscribed variables or alarms that have actually changed.
event.type - the type of event which may be:
- SYSTEM - a system event like: SYS_PUSH_SOURCE_OFFLINE, SYS_PUSH_SOURCE_ONLINE
- CONTENT - a content related event that depends on the actual push source an specific project definition
via_instance - the id of the instance that was used to create the subscription.

Each message is associated to its push source. The id of the push source is present in the destination address/topic that the message has been pushed to. However, one may use default_verbose messages instead. They additionally contain push source ids inside each message body.

“default” real-time data message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/rt_data

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "data_state": "ALL|CHANGES",
   "via_instance": "the_instance_id",
   "rt_data": {
      "var1": "value1",
      "var2": "value2"
   }
}

Examples of hypothetical variable names: temperature, light_onoff, power, energy, key_usage_counter

To obtain actual thing types and pushable variables one may call READ all thing types info API or READ per-instance thing types info API.

“default” alarm message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/alarms

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "data_state": "ALL|CHANGES",
   "via_instance": "the_instance_id",
   "alarms": [{
      "id": "the_alarm_id",
      "raised": true
   }]
}

Examples of hypothetical alarm ids: temperature_high, power_meter_error, device_error

To obtain actual thing types and pushable alarms one may call READ all thing types info API or READ per-instance thing types info API.

“default” event message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/events

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "via_instance": "the_instance_id",
   "event": {
      "id": "the_event_id",
      "type": "SYSTEM|CONTENT",
      "properties": {
         "prop1": "value1",
         "prop2": "value2"
      },
      "body": {
         "something": "contents of body depend on each event / project ...",
         "something_else": "... any valid JSON body may be present here."
      }
   }
}

Examples of hypothetical event ids: motion_detected, key_used, configuration_changed

To obtain actual thing types and pushable events one may call READ all thing types info API or READ per-instance thing types info API.

“default_verbose” messages¶

Default_verbose messages are JSON messages which are very similar to “default” messages, but have additional push source ids embedded in the message body. This is redundant information as it is already present in the push address/topic where the message has been pushed. It is added for reasons of convenience when API clients prefer to read body messages rather than parse topics.

“default_verbose” real-time data message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/rt_data

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "data_state": "ALL|CHANGES",
   "via_instance": "the_instance_id",
   "push_source": {
      "type": "THING",
      "namespace": "{THING_NAMESPACE}",
      "id": "{THING_ID}"
   },
   "rt_data": {
      "var1": "value1",
      "var2": "value2"
   }
}

Examples of hypothetical variable names: temperature, light_onoff, power, energy, key_usage_counter

To obtain actual thing types and pushable variables one may call READ all thing types info API or READ per-instance thing types info API.

“default_verbose” alarm message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/alarms

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "data_state": "ALL|CHANGES",
   "via_instance": "the_instance_id",
   "push_source": {
      "type": "THING",
      "namespace": "{THING_NAMESPACE}",
      "id": "{THING_ID}"
   },
   "alarms": [{
      "id": "the_alarm_id",
      "raised": true
   }]
}

Examples of hypothetical alarm ids: temperature_high, power_meter_error, device_error

To obtain actual thing types and pushable alarms one may call READ all thing types info API or READ per-instance thing types info API.

“default_verbose” event message¶

Message is pushed to the following topic (smip_mqtt transport usage scenario):

p/data_api/v3_0/{SESSION_ID}/t/{THING_NAMESPACE}/{THING_ID}/events

{
   "id": "the_message_id",
   "timestamp": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
   "via_instance": "the_instance_id",
   "push_source": {
      "type": "THING",
      "namespace": "{THING_NAMESPACE}",
      "id": "{THING_ID}"
   },
   "event": {
      "id": "the_event_id",
      "type": "SYSTEM|CONTENT",
      "properties": {
         "prop1": "value1",
         "prop2": "value2"
      },
      "body": {
         "something": "contents of body depend on each event / project ...",
         "something_else": "... any valid JSON body may be present here."
      }
   }
}

Examples of hypothetical event ids: motion_detected, key_used, configuration_changed

To obtain actual thing types and pushable events one may call READ all thing types info API or READ per-instance thing types info API.

Cloud Data Push¶

Push Sources¶

Push Sessions¶

A Push Session’s Life-cycle¶

The “default” session type¶

Push Subscriptions¶

A Push Subscription’s Life-cycle¶

Push Transports¶

The “smip_mqtt” transport¶

Push Messages¶

“default” messages¶

“default” real-time data message¶

“default” alarm message¶

“default” event message¶

“default_verbose” messages¶

“default_verbose” real-time data message¶

“default_verbose” alarm message¶

“default_verbose” event message¶

SMIP Data API 3.X Details

Navigation

Related Topics