Skip to content

Plugin

Plaid ships with a default Plugin that POSTs JSON formatted messages over http or https. The messages provide interaction points to use the OpenADR services required for use, without having to expose the whole protocol prematurely.

Plaid Diagram highlighting Plugin

The default plugin is used as-is in 90% of implementations, but for customers that want to update to a different messaging system or customize the way they interact with the plugin, it can be edited directly in the C++ code base.

Source for the sample plugin can be found in the following path: <root>/pluginimpl/plugin/notifier/http/.

Default Plugin Overview

The plugin has a series of callback functions that, if configured, post JSON messages to a specified endpoint. Initiation of these callback functions is determined by either a message from the VTN, or Plaid's internal scheduling system.

  • event functions contain information related to upcoming events
  • report functions contain and request information about reports
  • status functions provide information about your Plaid instance's operation

The remainder of this page will provide information on what the different functions do, and how to configure the plugin and enable functions.

Plugin Configuration

Config for the plugin is nested in the plugin section of the Plaid config file. It specifies a number of endpoints for callback functions that represent the core interfaces into Plaid. If left as a blank string (""), the message will not be POSTed when the function is ready. If an http or https endpoint is specified, a message will be POSTed to that endpoint.

For example, the startEvent function is triggered at the start of each event.

With the following configuration the startEvent function will be ignored:

  "startEvent": ""

In the following configuration the startEvent function will POST it's payload to https://api.awesomedemandresponse.com/startEvent every time an OpenADR event starts:

  "startEvent": "https://api.awesomedemandresponse.com/startEvent"

Plugin http message headers

The plugin will include Authorization headers if the httpAuth configuration is enabled. Other, custom headers may be created but will require customization of the C++ code. The code base to create these is exposed, so deep knowledge of C++ should not be required, and support is available as needed.

Overall message structure

Event and Report messages have a header field that includes information consistent across messages - this can be used to parse and direct messages.

Header fields reference

  • instanceId mirrors the instanceId defined in the config file
  • messageTypes enumerates the message type
  • pluginApiVersion version number of the API
  • pluginVersion version number of the plugin code
  • venId mirrors venId defined in config file or assigned by VTN during registration
  • vtnId mirrors venId defined in config file or assigned by VTN during registration

Plugin Function Reference

Message schema files as well as sample message for each callback function can be found here in the /quicktype/messages/ directory in the source code.

This reference will describe the different functions available. Some functions have an expected JSON payload response that will be sent in the body. For functions that do not require a response payload, it will be ignored. If a completely blank payload is returned, default settings will be used. If a response is returned that does not conform to schema, Plaid will throw an error.

Event Functions

startDistributeEvent

Message Name OnDistributeEventStart
Schema reference on_distribute_event_start.json
Description Includes all events sent by the VTN to the VEN. This may include past events, current events, active events, etc. JSON payload will include all information contained in event, most importantly: start time, payload, signal types and intervals.
Trigger VTN sends a new distribute event payload, which happens every time an event is added, updated or deleted
Response reference on_distribute_event_start_response.json
Response description Instructs the VEN to opt in or opt out of events by default (will be used, unless overridden by specific event opt in / opt out)

completeDistributeEvent

Message Name OnDistributeEventComplete
Schema reference on_distribute_event_complete.json
Description Contains the current unix timestamp. May be used to inform your system that no more OnEvent and CancelEvent messages are coming.
Trigger When all of the events event and cancelEvent callbacks has been sent
Response reference n/a

event

Message Name OnEvent
Schema reference on_event.json
Description Sends information about an individual event. Includes all information sent by VTN => VTN. Most importantly, event start, duration, intervals and payloads, event ID and targets.
Trigger Sent for every non-cancelled event every time the VTN sends a list of events.
Response reference on_event_response.json
Response description Instructs the VEN to opt in or opt out of this event

startEvent

Message Name OnEventStart
Schema reference on_event_start.json
Description Includes full information set about the event. Most implementations will skip this and use either `event` or `startEventInterval` instead.
Trigger Sent when event control should be started (when event start time is scheduled, adjusted for randomization window if event is randomized).
Response reference n/a

startEventInterval

Message Name OnEventIntervalStart
Schema reference on_event_interval_start.json
Description Includes information about the interval - start time, duration, payload and signal type, alongside full information about the event.
Trigger Sent at the start of each interval in the event. In valid OpenADR, the first interval will start at the same time as the event starts, so a OnEventIntervalStart message will always be sent simultaneously to the startEvent interval.
Response reference n/a

cancelEvent

Message Name OnEventCancel
Schema reference on_event_cancel.json
Description Includes full information about the event that has been cancelled
Trigger Sent for every cancelled event sent when a VTN distributes a list of events
Response reference on_event_cancel_response.json
Response description Instructs the VEN to opt in or opt out of this event

archiveEvent

Message Name OnEventArchive
Schema reference on_event_archive.json
Description Includes full information about the event to be archived
Trigger When an event is no longer in the list sent by the VTN (e.g. was deleted in the VTN or is too old to send)
Response reference n/a

endEvent

Message Name OnEventComplete
Schema reference on_event_complete.json
Description Includes full information on the event that has just completed and the ending time
Trigger When an active event has just ended (adjusted for randomization period, if applicable). Also triggered if event is cancelled while active (again, after the randomization period, if applicable).
Response reference n/a

rampUp

Message Name OnEventRampUp
Schema reference on_event_ramp_up.json
Description Includes full information on the event
Trigger If the event has a ramp up period, will trigger when the ramp up period starts.
Response reference n/a

Reporting

registerReports

Message Name OnRegisterReports
Schema reference on_register_reports.json
Description Requests information to register reports with the VTN. This is where the VEN represents to the VTN what reports it has available
Trigger On registration with VTN, or when a re-registration is requested by the VTN.
Response reference on_register_reports_response.json
Response description Includes an array where each element is a report description to be registered with the VTN, including report name, specifier ID, and individual rIDs to be registered with the VTN. Multiple intervals can be registered per report using the intervalDescriptions array. Intervals must have a unique rId. Note that usage intervals should include the usageIntervalProperties field, while status intervals should not include it.

startPeriodicReport

Message Name OnPeriodicReportStart
Schema reference on_periodic_report_start.json
Description Contains payload that describes the periodic report, including the report ID & rIds requested, as well as the timing and period of the data requested.
Trigger When a new periodic report is requested by the VTN
Response reference n/a

completePeriodicReport

Message Name OnPeriodicReportComplete
Schema reference on_periodic_report_complete.json
Description Contains both report request ID and report specifier ID of the periodic report that has been cancelled or completed.
Trigger When a periodic report request is either cancelled by the VTN, or has completed.
Response reference n/a

queryIntervals

Message Name OnQueryIntervals
Schema reference on_query_intervals.json
Description Requests data for a specific report
Trigger When a report is due to the VTN. Always initiated by VTN, but this can either be a specific one-time report request, or be request for a periodic report that Plaid will parse, and trigger OnQueryIntervals when reports are due.
Response reference on_query_intervals_response.json
Response description An array of reports to send to the VTN, each report including one or more data points. In most cases, a response will just include a single report in the array, but this offers the flexibility to send multiple reports in response to one request. To use the default report information (as in the request), do not include reportIdentifier information. To override the report name, specifierId, etc., use the reportIdentifier fields. This is unusual, but some VTNs will want reports sent in a custom manner.

Status

heartbeat

Message Name OnHeartbeat
Schema reference on_heartbeat.json
Description A POST with no body to the endpoint specified
Trigger Triggered on an interval as specified in the config file. Will run on the top of the interval, (e.g. if the interval is 300 seconds, will run on 00:00:00, 00:05:00, 00:10:00, etc.)
Response reference n/a

onRegister

Message Name OnRegister
Schema reference on_register.json
Description Includes current unix timestamp
Trigger When Plaid has successfully registered with the VTN
Response reference n/a

onError

Message Name OnError
Schema reference on_error.json
Description Error message sent by Plaid plugin
Trigger When plugin encounters an error
Response reference n/a

Sample messages

To create sample plugin messages in json files, run the plaid binary with the --samples flag, followed by the path to the directory you would like. If the directory path is not included, the files will be exported to the current directory.

$ ./plaidven --samples sample/message/directory
[Date] [plaid] [info] [main] Exporting samples to sample/message/directory

Type casting

The default plugin is set up to provide support for quicktype.

A quicktype file can be created from the schema files in the source code. This file provides functions for serializing and deserializing JSON strings into javascript objects. These functions will throw errors if your objects do not align to the schema, allowing for a faster debugging feedback loop.

Quicktype will need to be added as a command line tool: installation instructions can be found here: https://github.com/quicktype/quicktype

Once it is added, from the plugin messages schema directory (hosted in oadr/oadrapi/oadrapi/pluginmessages) you can run the following command in the CLI to generate the quicktype file.

For example, to create a simple javascript quicktype file, use the following command:

quicktype -l javascript --src-lang schema schema/*.json > lang/quicktype.js

Other quicktype file generation codes are in the generate.sh file in the source code repository.

Reference JS implementation of http server to receive messages

A reference implementation in Javascript of an http server that receives POST messages from the plugin can be found here: https://github.com/gridfabric/nodejs-sample-middleware.

The reference implementation also includes a basic example of using the quicktype output to serialize the JSON response.

Creating a plugin

It is possible to create a plugin from scratch. Documentation for this is not yet complete.

A pointer to an IVENManager is passed to the plugin's initialize function. This object shouldn't be needed under most circumstances as messages sent through the http API (see the HTTP API section) are forwarded to the VENManager. This object is offered as a convenience in the event that NOVA lacks functionality required for integration.

FUNCTIONS ON THE IVENManager OJBECT SHOULD ONLY BE CALLED WHEN INITIATED FROM THE HTTP API. THEY SHOULD NOT BE CALLED FROM ANY OF THE CALLBACKS.

The function OnRegisteredMessage in the sample plugin demonstrates calling the forEach event function on IVENManager.

Error Handling Tips

Plaid executes as a series of jobs which follow two simple rules:

  1. Only one job may execute at a time
  2. If a job fails, retry it at some interval

Each job is a series of steps. If a failure is detected in any step, the entire job will be retried. Failures mainly occur at the boundaries: when communicating to external systems. There are two external systems involved: the VTN and the customer system. Jobs that fail when communicating to the customer system will be retried at whatever retry interval is set in the config file. Jobs that fail when communicating to the VTN use an exponential back-off algorithm to determine when the next retry will occur.

Retries in a micro-service are an essential component of proper operation. Even when everything is up and running as expected, unexpected failures can occur at the network layer. Rebooting systems and services can also cause issues. Failures which occur from these and other circumstances are mitigated with retries.

Retries are most useful for coping with short outages. When long outages occur, the best strategy may be to restart Plaid. It is always safe to restart.

The following sections do not cover the intention of the plugin callbacks. See the header file INovaNotifierPlugin.h for an up-to-date description of the callbacks. Instead, the following sections provide tips for handling error conditions.

Idempotent Functions

Any function that can be retried must be designed to be idempotent. A function is idempotent if it can be executed multiple times with the same information and the system is left in a correct state.

Let's take a customer order system as an example. If the same exact order is charged to a customer account multiple times (e.g. as defined by a unique orderID), the customer should only be charged once.

When failures occur, Plaid may call callback functions with the same information. The callback function and any functions called on the external system must be idempotent.

Error Handling in the Plugin

Plaid automatically handles retries to the VTN and may also handle retries to the external system. This section discusses when and how this retry mechanism can be used.

The custom plugin communicates to the customer system through a library which implements some protocol such as http, message queues, database, etc. When a failure occurs, the client code detects the failure in 1 of 3 ways:

  1. An error code returned from a function call
  2. An exception thrown from a function call
  3. A negative response in the message from the external system

Assuming the micro-service code and API code on the external system are correct, all 3 errors are an indication that the system is temporarily down and the message should be retried at some point in the future. The 3rd error type might also indicate an error in the micro-service or the external system so it's good practice to log the 3rd error type and send a notification.

Error handling in the plugin can be done in two ways:

  1. Catch the errors and handle retries in the plugin. The sample plugin does this. This strategy works OK for event handling, but does not work for report handling.
  2. Don't catch exceptions and let Plaid do the retries. If an error is detected, throw a std::runtime_error("With a useful message") and let Plaid retry the job.

Plaid detects failures through C++ exceptions. As long as the exception can be caught with std::exception, Plaid will catch the error and retry the job at whatever interval is specified in the config. When an exception occurs, Plaid calls the OnException callback with details of the job that failed and the exception message.

You are not required to choose one strategy over the other. Choose whichever strategy works best for the given callback.

Some jobs communicate to both the VTN and to the customer system. In these instances, failures in the customer system affect what data is sent to the VTN. The following discussions on Event and Report handling in the plugin offer options for handling failures specific to those callbacks.

These discussions assume some familiarity of the callbacks. Before digging into the details below, it will help to read through and start implementing the callbacks.

Concurrency

Plaid is single threaded and operation will pause while a callback has been sent and it is waiting for a response. For example, Plaid API will not respond to requests while it is waiting for a callback function to complete. It will eventually timeout and then continue operation.

Events

The plugin callbacks startDistributeEvent, completeDistributeEvent, event, and cancelEvent all occur while Plaid is collecting data that will be sent to the VTN. These callbacks allow the plugin to opt in or out of events and that information is relayed back to the VTN. If an error is thrown from one of these functions, Plaid won't have the information needed to respond to the VTN and will delay sending the message to the VTN.

Here are some options for handling errors in these functions:

  1. Catch errors and default the opt status: The default opt status should probably be optOut to the let the VTN know the VEN is not available for control. Defaulting to optIn is also an option. Under this strategy, the external system would presumably not have event information stored in the customer system so a strategy is needed to sync the event information when the external system comes back online. Options to sync event information include restarting Plaid and writing a custom HTTP API function to retrieve the information. If the opt status of an event needs to be changed, the external system will also need to make calls to the http API function Created Event and/or Opt Event. If Plaid is restarted, the opt status will automatically be updated since Plaid will retrieve the list of events and redo the event callbacks.
  2. Throw errors: This is the easier strategy to implement. Plaid will keep retrying event processing and when the job completes successfully, Plaid and the external system will be in sync so there's no need for a strategy to resync event information once the external system is available again.

The other event callbacks occur between Plaid and the plugin only - the VTN does not receive any communication from these callbacks. Since the VTN isn't involved, the plugin can handle retries (as the sample plugin does). The plugin can also detect problems and throw errors. Throwing errors is the easier strategy.

Reports

The callbacks OnPeriodicReportStart and OnPeriodicReportComplete do not properly handle exceptions. If an error occurs while processing these callbacks, Plaid must be restarted to correct the error. Failures in this callback should be extremely rare. We are working on a fix.

The OnGeneratePeriodicReport callback does properly handle exceptions. The plugin should track what the last upload date was for each reportRequestID and use that date to "widen" the report window when an error occurs. The sample plugin tracks this information in the QueryIntervals object. The only option for handling retries with this callback is to throw errors: retries cannot be handled within the plugin.

Registration

The registration process is a single job but it hits many callbacks for events and reports. If a failure occurs anywhere in the process, the entire job is restarted.

Implementing for Multiple Utilities

Each running instance of Plaid connects to a single VTN. Therefore, to work with multiple utilities, each utility require it's own instance of Plaid.

The Plaid plugin should be designed to work with any utility though some options in the config file will change (such as vtn url) for each utility. This is true whether Plaid is running in the cloud or on device. As much as possible, control logic should be implemented in the external system with pluggable control logic.

When running multiple instances of Plaid on the same machine, each instance will need it's own log file and listening port (if the HTTP API is enabled).

Plaid can be executed from the same location in two ways:

  1. Use different config files for each instance
  2. Use the same config file with different command line parameters to override the parameters that differ.