About speculo

speculo is a tiny library meant to facilitate the usage of shared memory in POSIX systems. It allows developers to easily implement a design where one writer process shares chunks of data with many reader processes, with the guarantee that at any point in time readers have a consistent view of the data, and without the need of locks. It's a thin layer on top of the POSIX shm_open()/mmap() APIs, and as such it should preserve all the speed of SHM while hiding some of the shortcomings.

speculo in a few bullet points:

• small C library for IPC over shared memory
• good test coverage
• one writer, many readers (per speculo_area)
• zerocopy
• data is written and read in chunks of arbitrary size
• a data chunk becomes visible to the readers as soon as the writer commits it
• data chunks can have an expiration time
• data chunks can be obsoleted by a newer copy
• with the exception of a few checkpoints guarded by memory barriers, bytes written to the SHM object won't ever change their value
• when needed, the memory area is copied into a new SHM object with expired and obsoleted chunks removed to make room for new data

Use cases

speculo has been designed to mainly cover these two use-cases:

• sharing a stream of data packets
• sharing one or more data blocks whose value can change over time

Sharing a stream of data packets

This use-case can be implemented by allocating and writing chunks of data into a SHM object (which we call speculo_area) and setting an expiration time on each of them: once the SHM object is full (or a certain percentage of expired chunks has been reached) the data chunks which are still valid get copied over into a new SHM file, while the expired ones are discarded.

Sharing a changeable data block

Any data chunk written in a SHM object can be updated with new data: this is implemented in speculo by marking the old chunk as obsoleted and adding a new chunk with the new data, and remembering the association between the two.

Get involved

The source code of speculo is hosted at GitLab. There one can also find the issue tracker.

Discussions about the usage of speculo and its development should primarily take place in the speculo mailing list. The main developer can also be reached in IRC: mardy @ freenode.net

Author

Alberto Mardegan mardy.nosp@m.@use.nosp@m.rs.so.nosp@m.urce.nosp@m.forge.nosp@m..net

Copyright

GNU Lesser Public Licence v2.1