My place in space

Introduction to nvlist part 1

Oct. 17, 2017, 11:13 p.m.

Nvlist is an implementation of a general purpose container based on a list which contains pairs (name, value). The idea behind this library is borrowed from the Solaris library which  has the same name. In Solaris nvlist is used for example, by the ZFS. In FreeBSD nvlist exists in kernel and userland. Nvlist was implemented to provide an easy inter-process communication framework for applications that will be privileged separated and later sandboxed, due to this in userland there are functions allowing us to send nvlist over the Unix domain socket, in that case there is also a possibility to send descriptors with nvlist. The nvlist library can also be used to store data on a disk or to send it over the network. Furthermore nvlist is endian-independent. If we will send nvlist over the network from the little-endian architecture to the big-endian architecture, everything will work properly.

The basic functionality of nvlist is to get, take, add, move and check if an element exists. Get’s methods allows access to a read-only element. Secondly there is take which returns the value and removes it from the list. Similar functions are add and move. The first function creates a copy of the element that is added to the list. The move function adds an element to nvlist but doesn’t create a copy of it, after adding the element using this function nvlist is responsible for freeing memory used by this element.

The basic types used by nvlist are:
- null
- bool
- string
- nvlist
- descriptors
- binary
Every type also has an equivalent for the array functions. As we can see on the list above, nvlist provides us the possibility to store nested nvlists, which is often very practical.

One of the main ideas of nvlist is that we don’t need to check every single operation on the list, even its creation. The idea is that one can do several operations like adds/moves and we only check the last operation (often it will be send or save). Then the nvlist will report the error that first occurs. Here’s an example:

fd = open(“somefile”, O_RDONLU);
assert(fd > 0);

nvl = nvlist_create(0);
nvlist_add_string(nvl, "filename", "myfile”);
nvlist_move_descriptor(nvl, "descriptor", fd);
if (nvlist_send(sock, nvl) < 0)
    err(1, "nvlist_send() failed");

First create an nvlist using nvlist_create. Secondly add a string “myfile” under the name “filename” to the list. As mentioned before add function will create a copy of “myfile” text so there is no need to handle copying of it. Next, add a descriptor fd under the name “descriptor”. Here the move function is used. In that case nvlist is responsible for closing the fd descriptor. The last step is to send the nvlist over the sock, and only this operation needs to be checked. If nvlist would fail on any of the above steps the information about this will be reported after the send operation. Finally if everything was successful the nvlist can be freed using nvlist_destroy, this function will also close the fd descriptor.

It’s a different story with get/take functions. If an element doesn't exist the program will be aborted. So due to that before getting/taking an element it is necessary to check if an element exists or use dnvlist for this purpose. The description of the dnvlist interface will be explained in a separate post.

The downside of nvlist is that it doesn’t support any standard formats such as: JSON or JAML. It would be very interesting to see an implementation of parsers for those formats which converts them to nvlist. One of the solutions would be to integrate nvlist with libucl, but this is blue sky thinking.

Nvlist has a very user-friendly interface. All of the functions are described in the man page (nv(9)) with a few examples as well. If you are looking for an IPC for your application please give nvlist a try. If you are interested in more details about nvlist stay tuned.