Suricatta daemon mode

Introduction

Suricatta is – like mongoose – a daemon mode of SWUpdate, hence the name suricatta (engl. meerkat) as it belongs to the mongoose family.

Suricatta regularly polls a remote server for updates, downloads, and installs them. Thereafter, it reboots the system and reports the update status to the server, based on an update state variable currently stored in bootloader’s environment ensuring persistent storage across reboots. Some U-Boot script logics or U-Boot’s bootcount feature may be utilized to alter this update state variable, e.g., by setting it to reflect failure in case booting the newly flashed root file system has failed and a switchback had to be performed.

Suricatta is designed to be extensible in terms of the servers supported as described in Section The Suricatta Interface. Currently, support for the hawkBit server is implemented via the hawkBit Direct Device Integration API alongside a simple general purpose HTTP server. The support for suricatta modules written in Lua is not a particular server support implementation but rather an option for writing such in Lua instead of C.

Running suricatta

After having configured and compiled SWUpdate with enabled suricatta support for hawkBit,

./swupdate --help

lists the mandatory and optional arguments to be provided to suricatta when using hawkBit as server. As an example,

./swupdate -l 5 -u '-t default -u http://10.0.0.2:8080 -i 25'

runs SWUpdate in suricatta daemon mode with log-level TRACE, polling a hawkBit instance at http://10.0.0.2:8080 with tenant default and device ID 25.

If multiple server support is compiled in, the -S / --server option or a server entry in the configuration file’s [suricatta] section selects the one to use at run-time. For convenience, when having support for just one server compiled-in, this is chosen automatically.

Note that on startup when having installed an update, suricatta tries to report the update status to its upstream server, e.g., hawkBit, prior to entering the main loop awaiting further updates. If this initial report fails, e.g., because of a not (yet) configured network or a currently unavailable hawkBit server, SWUpdate may exit with an according error code. This behavior allows one to, for example, try several upstream servers sequentially. If suricatta should keep retrying until the update status is reported to its upstream server irrespective of the error conditions, this has to be realized externally in terms of restarting SWUpdate on exit.

After an update has been performed, an agent listening on the progress interface may execute post-update actions, e.g., a reboot, on receiving DONE. Additionally, a post-update command specified in the configuration file or given by the -p command line option can be executed.

Note that at least a restart of SWUpdate has to be performed as post-update action since only then suricatta tries to report the update status to its upstream server. Otherwise, succinct update actions announced by the upstream server are skipped with an according message until a restart of SWUpdate has happened in order to not install the same update again.

The Suricatta Interface

Support for servers other than hawkBit or the general purpose HTTP server can be realized by implementing the “interfaces” described in include/channel.h and include/suricatta/server.h, the latter either in C or in Lua. The channel interface abstracts a particular connection to the server, e.g., HTTP-based in case of hawkBit. The server interface defines the logics to poll and install updates. See corelib/channel_curl.c / include/channel_curl.h and suricatta/server_hawkbit.{c,h} for an example implementation in C targeted towards hawkBit.

include/channel.h describes the functionality a channel has to implement:

typedef struct channel channel_t;
struct channel {
    ...
};

channel_t *channel_new(void);

which sets up and returns a channel_t struct with pointers to functions for opening, closing, fetching, and sending data over the channel.

include/suricatta/server.h describes the functionality a server has to implement:

typedef struct {
    server_op_res_t has_pending_action(int *action_id);
    server_op_res_t install_update(void);
    server_op_res_t send_target_data(void);
    unsigned int get_polling_interval(void);
    server_op_res_t start(const char *cfgfname, int argc, char *argv[]);
    server_op_res_t stop(void);
    server_op_res_t ipc(int fd);
    void (*help)(void);
} server_t;

These functions constituting a particular suricatta server implementation have to be registered for being selectable at run-time by calling register_server() (see include/suricatta/server.h) with a name and a server_t struct pointer implemented in a __attribute__((constructor)) marked function, see suricatta/server_hawkbit.c as example.

The type server_op_res_t is defined in include/suricatta/suricatta.h. It represents the valid function return codes for a server’s implementation.

In addition to implementing the particular channel and server, the suricatta/Config.in file has to be adapted to include a new option so that the new implementation becomes selectable in SWUpdate’s configuration. In the simplest case, adding an option like the following one for hawkBit into the menu "Server" section is sufficient.

config SURICATTA_HAWKBIT
    bool "hawkBit support"
    default y
    select JSON
    help
      Support for hawkBit server.
      https://projects.eclipse.org/projects/iot.hawkbit

Having included the new server implementation into the configuration, edit suricatta/Makefile to specify the implementation’s linkage into the SWUpdate binary, e.g., for the hawkBit example implementation, the following lines add server_hawkbit.o to the resulting SWUpdate binary if SURICATTA_HAWKBIT was selected while configuring SWUpdate.

ifneq ($(CONFIG_SURICATTA_HAWKBIT),)
obj-$(CONFIG_SURICATTA) += server_hawkbit.o
endif

Support for general purpose HTTP server

This is a very simple backend that uses standard HTTP response codes to signal if an update is available. There are closed source backends implementing this interface, but because the interface is very simple interface, this server type is also suitable for implementing an own backend server. For inspiration, there’s a simple (mock) server implementation available in examples/suricatta/server_general.py.

The API consists of a GET with Query parameters to inform the server about the installed version. The query string has the format:

http(s)://<base URL>?param1=val1&param2=value2...

As examples for parameters, the device can send its serial number, MAC address and the running version of the software. It is duty of the backend to interpret this - SWUpdate just takes them from the “identify” section of the configuration file and encodes the URL.

The server answers with the following return codes:

HTTP Code	Text	Description
302	Found	A new software is available at URL in the Location header
400	Bad Request	Some query parameters are missing or in wrong format
403	Forbidden	Client certificate not valid
404	Not found	No update is available for this device
503	Unavailable	An update is available but server can’t handle another update process now.

Server’s answer can contain the following headers:

Header’s name	Codes	Description
Retry-after	503	Contains a number which tells the device how long to wait until ask the next time for updates. (Seconds)
Content-MD5	302	Contains the checksum of the update file which is available under the url of location header
Location	302	URL where the update file can be downloaded.

The device can send logging data to the server. Any information is transmitted in a HTTP PUT request with the data as plain string in the message body. The Content-Type Header need to be set to text/plain.

The URL for the logging can be set as separate URL in the configuration file or via –logurl command line parameter:

The device sends data in a CSV format (Comma Separated Values). The format is:

value1,value2,...

The format can be specified in the configuration file. A format For each event can be set. The supported events are:

Event	Description
check	dummy. It could send an event each time the server is polled.
started	A new software is found and SWUpdate starts to install it
success	A new software was successfully installed
fail	Failure by installing the new software

The general server has an own section inside the configuration file. As example:

gservice =
{
        url             = ....;
        logurl          = ;
        logevent : (
                {event = "check"; format="#2,date,fw,hw,sp"},
                {event = "started"; format="#12,date,fw,hw,sp"},
                {event = "success"; format="#13,date,fw,hw,sp"},
                {event = "fail"; format="#14,date,fw,hw,sp"}
        );
}

date is a special field and it is interpreted as localtime in RFC 2822 format. Each Comma Separated field is looked up inside the identify section in the configuration file, and if a match is found the substitution occurs. In case of no match, the field is sent as it is. For example, if the identify section has the following values:

identify : (
        { name = "sp"; value = "333"; },
        { name = "hw"; value = "ipse"; },
        { name = "fw"; value = "1.0"; }
);

with the events set as above, the formatted text in case of “success” will be:

Formatted log: #13,Mon, 17 Sep 2018 10:55:18 CEST,1.0,ipse,333

Support for Suricatta Modules in Lua

The server_lua.c C-to-Lua bridge enables writing suricatta modules in Lua. It provides the infrastructure in terms of the interface to SWUpdate “core” to the Lua realm, enabling the “business logic” such as handling update flows and communicating with backend server APIs to be modeled in Lua. To the Lua realm, the server_lua.c C-to-Lua bridge provides the same functionality as the other suricatta modules written in C have, realizing a separation of means and control. Effectively, it lifts the interface outlined in Section The Suricatta Interface to the Lua realm.

As an example server implementation, see examples/suricatta/server_general.py for a simple (mock) server of a backend that’s modeled after the “General Purpose HTTP Server” (cf. Section Support for general purpose HTTP server). The matching Lua suricatta module is found in examples/suricatta/swupdate_suricatta.lua. Place it in Lua’s path so that a require("swupdate_suricatta") can load it or embed it into the SWUpdate binary by enabling CONFIG_EMBEDDED_SURICATTA_LUA and setting CONFIG_EMBEDDED_SURICATTA_LUA_SOURCE accordingly.

The interface specification in terms of a Lua (suricatta) module is found in suricatta/suricatta.lua.

suricatta

The suricatta table is the module’s main table housing the exposed functions and definitions via the sub-tables described below. In addition, the main functions suricatta.install() and suricatta.download() as well as the convenience functions suricatta.getversion(), suricatta.sleep(), and suricatta.get_tmpdir() are exposed:

The function suricatta.install(install_channel) installs an update artifact from a remote server or a local file. The install_channel table parameter designates the channel to be used for accessing the artifact plus channel options diverging from the defaults set at channel creation time. For example, an install_channel table may look like this:

{ channel = chn, url = "https://artifacts.io/update.swu" }

where chn is the return value of a call to channel.open(). The other table attributes, like url in this example, are channel options diverging from or omitted while channel creation time, see suricatta.channel. For installing a local file, an install_channel table may look like this:

{ channel = chn, url = "file:///path/to/file.swu" }

The function suricatta.download(download_channel, localpath) just downloads an update artifact. The parameter download_channel is as for suricatta.install(). The parameter localpath designates the output path for the artifact. The suricatta.get_tmpdir() function (see below) is in particular useful for this case to supply a temporary download location as localpath. A just downloaded artifact may be installed later using suricata.install() with an appropriate file:// URL, realizing a deferred installation.

Both, suricatta.install() and suricatta.download() return true, or, in case of error, nil, a suricatta.status value, and a table with messages in case of errors, else an empty table.

The function suricatta.getversion() returns a table with SWUpdate’s version and patchlevel fields. This information can be used to determine API (in-)compatibility of the Lua suricatta module with the SWUpdate version running it.

The function suricatta.sleep(seconds) is a wrapper around SLEEP(3) for, e.g., implementing a REST API call retry mechanism after a number of given seconds have elapsed.

The function suricatta.get_tmpdir() returns the path to SWUpdate’s temporary working directory where, e.g., the suricatta.download() function may place the downloaded artifacts.

suricatta.status

The suricatta.status table exposes the server_op_res_t enum values defined in include/util.h to the Lua realm.

suricatta.notify

The suricatta.notify table provides the usual logging functions to the Lua suricatta module matching their uppercase-named pendants available in the C realm.

One notable exception is suricatta.notify.progress(message) which dispatches the message to the progress interface (see Getting information on running update). Custom progress client implementations listening and acting on custom progress messages can be realized using this function.

All notify functions return nil.

suricatta.pstate

The suricatta.pstate table provides a binding to SWUpdate’s (persistent) state handling functions defined in include/state.h, however, limited to the bootloader environment variable STATE_KEY defined by CONFIG_UPDATE_STATE_BOOTLOADER and defaulting to ustate. In addition, it captures the update_state_t enum values.

The function suricatta.pstate.save(state) requires one of suricatta.pstate’s “enum” values as parameter and returns true, or, in case of error, nil. The function suricatta.pstate.get() returns one of suricatta.pstate’s “enum” values or, in case of error, STATE_ERROR.

suricatta.server

The suricatta.server table provides the sole function suricatta.server.register(function_p, purpose). It registers a Lua function “pointed” to by function_p for the purpose purpose which is defined by suricatta.server’s “enum” values. Those enum values correspond to the functions defined in the interface outlined in the Section on The Suricatta Interface.

In addition to these functions, the two callback functions CALLBACK_PROGRESS and CALLBACK_CHECK_CANCEL can be registered optionally: The former can be used to upload progress information to the server while the latter serves as dwlwrdata function (see include/channel_curl.h) to decide on whether an installation should be aborted while the download phase.

For details on the (callback) functions and their signatures, see the interface specification suricatta/suricatta.lua and the documented example Lua suricatta module found in examples/suricatta/swupdate_suricatta.lua.

The suricatta.server.register() function returns true, or, in case of error, nil.

suricatta.channel

The suricatta.channel table captures channel handling for suricatta Lua modules. The single function suricatta.channel.open(options) creates and opens a channel to a server. Its single parameter options is a table specifying the channel’s default options such as proxy, retries, usessl, strictssl, or headers_to_send. For convenience, options that may change per request such as url, content-type, or headers_to_send may be set as defaults on channel creation time while being selectively overruled on a per request basis. The channel options currently supported to be set are listed in the suricatta.channel.options table. In essence, the options parameter table is the Lua table equivalent of include/channel_curl.h’s channel_data_t.

The suricatta.channel.open(options) function returns a channel table which is either passed to the suricatta.install() and suricatta.download() functions or used directly for communication with a server. More specifically, it has the three functions

get(options) for retrieving information from the server,
put(options) for sending information to the server, and
close() for closing the channel.

The get() and put() functions’ single parameter options is a per-request channel option table as described above.

The functions get() and put() return true, or, in case of error, nil, a suricatta.status value, and an operation result table. The latter contains the fields:

http_response_code carrying the HTTP error code,
format as one of suricatta.channel.content’s options,
raw_reply if options contained format = suricatta.channel.content.RAW,
json_reply if options contained format = suricatta.channel.content.JSON, and
the HTTP headers received in the received_headers table, if any.

The suricatta.channel.content “enum” table defines the “format”, i.e., the response body content type and whether to parse it or not:

NONE means the response body is discarded.
RAW means the raw server’s reply is available as raw_reply.
JSON means the server’s JSON reply is parsed into a Lua table and available as json_reply.

The suricatta.channel.method “enum” table defines the HTTP method to use for a request issued with the put(options) function, i.e., POST, PATCH, or PUT as specified in the options parameter table via the method attribute. In addition to the HTTP method, the request body’s content is set with the request_body attribute in the options parameter table.

As a contrived example, consider the following call to a channel’s put() function

...
local res, _, data = channel.put({
        url          = string.format("%s/%s", base_url, device_id),
        content_type = "application/json",
        method       = suricatta.channel.method.PATCH,
        format       = suricatta.channel.content.NONE,
        request_body = "{ ... }"
    })
...

that issues a HTTP PATCH to some URL with a JSON content without having interest in the response body.

More examples of how to use a channel can be found in the example suricatta Lua module examples/suricatta/swupdate_suricatta.lua.

suricatta.bootloader

The suricatta.bootloader table exposes SWUpdate’s bootloader environment modification functions to suricatta Lua modules.

The enum-like table suricatta.bootloader.bootloaders holds the bootloaders SWUpdate supports, i.e.

suricatta.bootloader.bootloaders = {
    EBG   = "ebg",
    NONE  = "none",
    GRUB  = "grub",
    UBOOT = "uboot",
},

The function suricatta.bootloader.get() returns the currently selected bootloader in terms of a suricatta.bootloader.bootloaders field value.

The function suricatta.bootloader.is(name) takes one of suricatta.bootloader.bootloaders’s field values as name and returns true if it is the currently selected bootloader, false otherwise.

The functions in the suricatta.bootloader.env table interact with the currently selected bootloader’s environment:

The function suricatta.bootloader.env.get(variable) retrieves the value associated to variable from the bootloader’s environment.

The function suricatta.bootloader.env.set(variable, value) sets the bootloader environment’s key variable to value.

The function suricatta.bootloader.env.unset(variable) deletes the bootloader environment’s key variable.

The function suricatta.bootloader.env.apply(filename) applies all key=value lines of a local file filename to the currently selected bootloader’s environment.