Plugin
Plaid ships with a default Plugin that POSTs JSON formatted messages over http or https. The messages provide interaction points to use the OpenADR services required for use, without having to expose the whole protocol prematurely.
The default plugin is used as-is in 90% of implementations, but for customers that want to update to a different messaging system or customize the way they interact with the plugin, it can be edited directly in the C++ code base.
Source for the sample plugin can be found in the following path: <root>/pluginimpl/plugin/notifier/http/.
Default Plugin Overview
The plugin has a series of callback functions that, if configured, post JSON messages to a specified endpoint. Initiation of these callback functions is determined by either a message from the VTN, or Plaid's internal scheduling system.
- event functions contain information related to upcoming events
- report functions contain and request information about reports
- status functions provide information about your Plaid instance's operation
The remainder of this page will provide information on what the different functions do, and how to configure the plugin and enable functions.
Plugin Configuration
Config for the plugin is nested in the plugin section of the Plaid config file. It specifies a number of endpoints for callback functions that represent the core interfaces into Plaid. If left as a blank string (""), the message will not be POSTed when the function is ready. If an http or https endpoint is specified, a message will be POSTed to that endpoint.
For example, the startEvent function is triggered at the start of each event.
With the following configuration the startEvent function will be ignored:
"startEvent": ""
In the following configuration the startEvent function will POST it's payload to https://api.awesomedemandresponse.com/startEvent every time an OpenADR event starts:
"startEvent": "https://api.awesomedemandresponse.com/startEvent"
Plugin http message headers
The plugin will include Authorization headers if the httpAuth configuration is enabled. Other, custom headers may be created but will require customization of the C++ code. The code base to create these is exposed, so deep knowledge of C++ should not be required, and support is available as needed.
Overall message structure
Event and Report messages have a header field that includes information consistent across messages - this can be used to parse and direct messages.
Header fields reference
- instanceId mirrors the instanceId defined in the config file
- messageTypes enumerates the message type
- pluginApiVersion version number of the API
- pluginVersion version number of the plugin code
- venId mirrors venId defined in config file or assigned by VTN during registration
- vtnId mirrors venId defined in config file or assigned by VTN during registration
Plugin Function Reference
Message schema files for each callback function can be found here in the /oadr/oadrapi/oadrapi/pluginmessages/schema/ directory in the source code.
This reference will describe the different functions available. Some functions have an expected JSON payload response that will be sent in the body. For functions that do not require a response payload, it will be ignored. If a completely blank payload is returned, default settings will be used. If a response is returned that does not conform to schema, Plaid will throw an error.
Event Functions
startDistributeEvent
| Message Name | OnDistributeEventStart |
| Schema reference | on_distribute_event_start.json |
| Description | Includes all events sent by the VTN to the VEN. This may include past events, current events, active events, etc. JSON payload will include all information contained in event, most importantly: start time, payload, signal types and intervals. |
| Trigger | VTN sends a new distribute event payload, which happens every time an event is added, updated or deleted |
| Response reference | on_distribute_event_start_response.json |
| Response description | Instructs the VEN to opt in or opt out of events by default (will be used, unless overridden by specific event opt in / opt out) |
completeDistributeEvent
| Message Name | OnDistributeEventComplete |
| Schema reference | on_distribute_event_complete.json |
| Description | Contains the current unix timestamp. May be used to inform your system that no more OnEvent and CancelEvent messages are coming. |
| Trigger | When all of the events event and cancelEvent callbacks has been sent |
| Response reference | n/a |
event
| Message Name | OnEvent |
| Schema reference | on_event.json |
| Description | Sends information about an individual event. Includes all information sent by VTN => VTN. Most importantly, event start, duration, intervals and payloads, event ID and targets. |
| Trigger | Sent for every non-cancelled event every time the VTN sends a list of events. |
| Response reference | on_event_response.json |
| Response description | Instructs the VEN to opt in or opt out of this event |
startEvent
| Message Name | OnEventStart |
| Schema reference | on_event_start.json |
| Description | Includes full information set about the event. Most implementations will skip this and use either `event` or `startEventInterval` instead. |
| Trigger | Sent when event control should be started (when event start time is scheduled, adjusted for randomization window if event is randomized). |
| Response reference | n/a |
startEventInterval
| Message Name | OnEventIntervalStart |
| Schema reference | on_event_interval_start.json |
| Description | Includes information about the interval - start time, duration, payload and signal type, alongside full information about the event. |
| Trigger | Sent at the start of each interval in the event. In valid OpenADR, the first interval will start at the same time as the event starts, so a OnEventIntervalStart message will always be sent simultaneously to the startEvent interval. |
| Response reference | n/a |
cancelEvent
| Message Name | OnEventCancel |
| Schema reference | on_event_cancel.json |
| Description | Includes full information about the event that has been cancelled |
| Trigger | Sent for every cancelled event sent when a VTN distributes a list of events |
| Response reference | on_event_cancel_response.json |
| Response description | Instructs the VEN to opt in or opt out of this event |
archiveEvent
| Message Name | OnEventArchive |
| Schema reference | on_event_archive.json |
| Description | Includes full information about the event to be archived |
| Trigger | When an event is no longer in the list sent by the VTN (e.g. was deleted in the VTN or is too old to send) |
| Response reference | n/a |
endEvent
| Message Name | OnEventComplete |
| Schema reference | on_event_complete.json |
| Description | Includes full information on the event that has just completed and the ending time |
| Trigger | When an active event has just ended (adjusted for randomization period, if applicable). Also triggered if event is cancelled while active (again, after the randomization period, if applicable). |
| Response reference | n/a |
rampUp
| Message Name | OnEventRampUp |
| Schema reference | on_event_ramp_up.json |
| Description | Includes full information on the event |
| Trigger | If the event has a ramp up period, will trigger when the ramp up period starts. |
| Response reference | n/a |
Reporting
registerReports
| Message Name | OnRegisterReports |
| Schema reference | on_register_reports.json |
| Description | Requests information to register reports with the VTN. This is where the VEN represents to the VTN what reports it has available |
| Trigger | On registration with VTN, or when a re-registration is requested by the VTN. |
| Response reference | on_register_reports_response.json |
| Response description | Includes an array where each element is a report description to be registered with the VTN, including report name, specifier ID, and individual rIDs to be registered with the VTN. Multiple intervals can be registered per report using the intervalDescriptions array. Intervals must have a unique rId. Note that usage intervals should include the usageIntervalProperties field, while status intervals should not include it. |
startPeriodicReport
| Message Name | OnPeriodicReportStart |
| Schema reference | on_periodic_report_start.json |
| Description | Contains payload that describes the periodic report, including the report ID & rIds requested, as well as the timing and period of the data requested. |
| Trigger | When a new periodic report is requested by the VTN |
| Response reference | n/a |
completePeriodicReport
| Message Name | OnPeriodicReportComplete |
| Schema reference | on_periodic_report_complete.json |
| Description | Contains both report request ID and report specifier ID of the periodic report that has been cancelled or completed. |
| Trigger | When a periodic report request is either cancelled by the VTN, or has completed. |
| Response reference | n/a |
queryIntervals
| Message Name | OnQueryIntervals |
| Schema reference | on_query_intervals.json |
| Description | Requests data for a specific report |
| Trigger | When a report is due to the VTN. Always initiated by VTN, but this can either be a specific one-time report request, or be request for a periodic report that Plaid will parse, and trigger OnQueryIntervals when reports are due. |
| Response reference | on_query_intervals_response.json |
| Response description | An array of reports to send to the VTN, each report including one or more data points. In most cases, a response will just include a single report in the array, but this offers the flexibility to send multiple reports in response to one request. To use the default report information (as in the request), do not include reportIdentifier information. To override the report name, specifierId, etc., use the reportIdentifier fields. This is unusual, but some VTNs will want reports sent in a custom manner. Note that the reportIdentifier field "reportId" will map to "eiReportId" in the oadrReport object sent by Plaid to the VTN. |
Status
heartbeat
| Message Name | OnHeartbeat |
| Schema reference | on_heartbeat.json |
| Description | A POST with no body to the endpoint specified |
| Trigger | Triggered on an interval as specified in the config file. Will run on the top of the interval, (e.g. if the interval is 300 seconds, will run on 00:00:00, 00:05:00, 00:10:00, etc.) |
| Response reference | n/a |
onRegister
| Message Name | OnRegister |
| Schema reference | on_register.json |
| Description | Includes current unix timestamp |
| Trigger | When Plaid has successfully registered with the VTN |
| Response reference | n/a |
onError
| Message Name | OnError |
| Schema reference | on_error.json |
| Description | Error message sent by Plaid plugin |
| Trigger | When plugin encounters an error |
| Response reference | n/a |
Sample messages
To create sample plugin messages in json files, run the plaid binary with the --samples flag, followed by the path to the directory you would like. If the directory path is not included, the files will be exported to the current directory.
$ ./plaidven --samples sample/message/directory
[Date] [plaid] [info] [main] Exporting samples to sample/message/directory
Type casting
The default plugin is set up to provide support for quicktype.
A quicktype file can be created from the schema files in the source code. This file provides functions for serializing and deserializing JSON strings into javascript objects. These functions will throw errors if your objects do not align to the schema, allowing for a faster debugging feedback loop.
Quicktype will need to be added as a command line tool: installation instructions can be found here: https://github.com/quicktype/quicktype
Once it is added, from the plugin messages schema directory (hosted in oadr/oadrapi/oadrapi/pluginmessages) you can run the following command in the CLI to generate the quicktype file.
For example, to create a simple javascript quicktype file, use the following command:
quicktype -l javascript --src-lang schema schema/*.json > lang/quicktype.js
Other quicktype file generation codes are in the generate.sh file in the source code repository.
Reference JS implementation of http server to receive messages
A reference implementation in Javascript of an http server that receives POST messages from the plugin can be found here: https://github.com/gridfabric/nodejs-sample-middleware.
The reference implementation also includes a basic example of using the quicktype output to serialize the JSON response.
Creating a plugin
It is possible to create a plugin from scratch. Documentation for this is not yet complete.
A pointer to an IVENManager is passed to the plugin's initialize function. This object shouldn't be needed under most circumstances as messages sent through the http API (see the HTTP API section) are forwarded to the VENManager. This object is offered as a convenience in the event that NOVA lacks functionality required for integration.
FUNCTIONS ON THE IVENManager OJBECT SHOULD ONLY BE CALLED WHEN INITIATED FROM THE HTTP API. THEY SHOULD NOT BE CALLED FROM ANY OF THE CALLBACKS.
The function OnRegisteredMessage in the sample plugin demonstrates calling the forEach event function on IVENManager.
Error Handling Tips
Plaid executes as a series of jobs which follow two simple rules:
- Only one job may execute at a time
- If a job fails, retry it at some interval
Each job is a series of steps. If a failure is detected in any step, the entire job will be retried. Failures mainly occur at the boundaries: when communicating to external systems. There are two external systems involved: the VTN and the customer system. Jobs that fail when communicating to the customer system will be retried at whatever retry interval is set in the config file. Jobs that fail when communicating to the VTN use an exponential back-off algorithm to determine when the next retry will occur.
Retries in a micro-service are an essential component of proper operation. Even when everything is up and running as expected, unexpected failures can occur at the network layer. Rebooting systems and services can also cause issues. Failures which occur from these and other circumstances are mitigated with retries.
Retries are most useful for coping with short outages. When long outages occur, the best strategy may be to restart Plaid. It is always safe to restart.
The following sections do not cover the intention of the plugin callbacks. See the header file INovaNotifierPlugin.h for an up-to-date description of the callbacks. Instead, the following sections provide tips for handling error conditions.
Idempotent Functions
Any function that can be retried must be designed to be idempotent. A function is idempotent if it can be executed multiple times with the same information and the system is left in a correct state.
Let's take a customer order system as an example. If the same exact order is charged to a customer account multiple times (e.g. as defined by a unique orderID), the customer should only be charged once.
When failures occur, Plaid may call callback functions with the same information. The callback function and any functions called on the external system must be idempotent.
Error Handling in the Plugin
Plaid automatically handles retries to the VTN and may also handle retries to the external system. This section discusses when and how this retry mechanism can be used.
The custom plugin communicates to the customer system through a library which implements some protocol such as http, message queues, database, etc. When a failure occurs, the client code detects the failure in 1 of 3 ways:
- An error code returned from a function call
- An exception thrown from a function call
- A negative response in the message from the external system
Assuming the micro-service code and API code on the external system are correct, all 3 errors are an indication that the system is temporarily down and the message should be retried at some point in the future. The 3rd error type might also indicate an error in the micro-service or the external system so it's good practice to log the 3rd error type and send a notification.
Error handling in the plugin can be done in two ways:
- Catch the errors and handle retries in the plugin. The sample plugin does this. This strategy works OK for event handling, but does not work for report handling.
- Don't catch exceptions and let Plaid do the retries. If an error is detected, throw a
std::runtime_error("With a useful message")and let Plaid retry the job.
Plaid detects failures through C++ exceptions. As long as the exception can be caught with std::exception, Plaid will catch the error and retry the job at whatever interval is specified in the config. When an exception occurs, Plaid calls the OnException callback with details of the job that failed and the exception message.
You are not required to choose one strategy over the other. Choose whichever strategy works best for the given callback.
Some jobs communicate to both the VTN and to the customer system. In these instances, failures in the customer system affect what data is sent to the VTN. The following discussions on Event and Report handling in the plugin offer options for handling failures specific to those callbacks.
These discussions assume some familiarity of the callbacks. Before digging into the details below, it will help to read through and start implementing the callbacks.
Concurrency
Plaid is single threaded and operation will pause while a callback has been sent and it is waiting for a response. For example, Plaid API will not respond to requests while it is waiting for a callback function to complete. It will eventually timeout and then continue operation.
Events
The plugin callbacks startDistributeEvent, completeDistributeEvent, event, and cancelEvent all occur while Plaid is collecting data that will be sent to the VTN. These callbacks allow the plugin to opt in or out of events and that information is relayed back to the VTN. If an error is thrown from one of these functions, Plaid won't have the information needed to respond to the VTN and will delay sending the message to the VTN.
Here are some options for handling errors in these functions:
- Catch errors and default the opt status: The default opt status should probably be optOut to the let the VTN know the VEN is not available for control. Defaulting to optIn is also an option. Under this strategy, the external system would presumably not have event information stored in the customer system so a strategy is needed to sync the event information when the external system comes back online. Options to sync event information include restarting Plaid and writing a custom HTTP API function to retrieve the information. If the opt status of an event needs to be changed, the external system will also need to make calls to the http API function
Created Eventand/orOpt Event. If Plaid is restarted, the opt status will automatically be updated since Plaid will retrieve the list of events and redo the event callbacks. - Throw errors: This is the easier strategy to implement. Plaid will keep retrying event processing and when the job completes successfully, Plaid and the external system will be in sync so there's no need for a strategy to resync event information once the external system is available again.
The other event callbacks occur between Plaid and the plugin only - the VTN does not receive any communication from these callbacks. Since the VTN isn't involved, the plugin can handle retries (as the sample plugin does). The plugin can also detect problems and throw errors. Throwing errors is the easier strategy.
Reports
The callbacks OnPeriodicReportStart and OnPeriodicReportComplete do not properly handle exceptions. If an error occurs while processing these callbacks, Plaid must be restarted to correct the error. Failures in this callback should be extremely rare. We are working on a fix.
The OnGeneratePeriodicReport callback does properly handle exceptions. The plugin should track what the last upload date was for each reportRequestID and use that date to "widen" the report window when an error occurs. The sample plugin tracks this information in the QueryIntervals object. The only option for handling retries with this callback is to throw errors: retries cannot be handled within the plugin.
Registration
The registration process is a single job but it hits many callbacks for events and reports. If a failure occurs anywhere in the process, the entire job is restarted.
Implementing for Multiple Utilities
Each running instance of Plaid connects to a single VTN. Therefore, to work with multiple utilities, each utility require it's own instance of Plaid.
The Plaid plugin should be designed to work with any utility though some options in the config file will change (such as vtn url) for each utility. This is true whether Plaid is running in the cloud or on device. As much as possible, control logic should be implemented in the external system with pluggable control logic.
When running multiple instances of Plaid on the same machine, each instance will need it's own log file and listening port (if the HTTP API is enabled).
Plaid can be executed from the same location in two ways:
- Use different config files for each instance
- Use the same config file with different command line parameters to override the parameters that differ.