Yes, there is some overlap but iceoryx and threadprocs could complement each other. Currently, we achieve true zero copy only with data types that are shared memory compatible, i.e. no heap, no self references, etc., basically the data type must be memcopyable. Cap'n Proto or the Apache Arrow columnar format would also work. For other data, e.g one that uses the heap, there would still a serialization step be required but at least there is no further copy between the processes.
With threadprocs, it would be possible to also send data structures that use the heap or are self-referential. iceoryx2 already works in the same way as the proposed solution for the deallocation. There is a submission channel to provide the data and a completion channel where the pointer are sent back to the producer. Currently, we do not call any d'tor, since it is not necessary, but that should not be too difficult to implement.
Btw, with the next release iceoryx2 will have Python bindings. They are already on main and we will make it available via PIP. This should make it easier to use with Pytorch.
I'm not sure I fully understand what you mean? Do you assume we implemented the same approach for shared memory communication like described in the blog post?
If that’s the case, I want to reassure you that we don’t use locks. Quite the contrary, we use lock-free[1] algorithm to implement the queues. We cannot use locks for the reason you mentioned and also for cases when an application dies while holding the lock. This would result in a deadlock and cannot be used in a safety critical environment. Btw, there are already cars out there which are using a predecessor of iceoryx to distribute camera data in an ECU.
For hard realtime systems we have a wait-free queue. This gives even more guarantees. Lock-free algorithms often have a CAS loop (compare and swap), which in theory can lead to starvation but it's practically unlikely as long as your system does not run at 100% CPU utilization all the time. As a young company, we cannot open source everything immediately, so the wait-free queue will be part of a commercial support package, together with more sophisticated tooling, like teased in the blog post.
Regarding memory guarantees. There are essentially the same guarantees like what you have when sharing an Arc<T> via a Rust channel. After publishing the producer releases the ownership to the subscriber and they have read-only access for as long as they hold the sample. When the sample is dropped by all subscriber, it will be released back to the shared memory allocator.
Btw, we also have an event signalling mechanism to not poll the queue but wait until the producer signals that new data is available. But this requires a context switch and it is up to the user to decide if it is desired to have.
> In my experience shared memory is really hard to implement well and manage:
I second that. It took us quite some time to get the correct architecture. After all, iceoryx2 is the third incarnation of this piece of software, with elfepiff an me working on the last two.
> 1. Unless you're using either fixed sized or specially allocated structures, you end up paying for serialization anyhow (zero copy is actually one copy).
Indeed, we are using fixed size structures with a bucket allocator. We have ideas on how to enable the usage on types which support custom allocators and even with raw pointers but that is just a crazy idea which might not pan out to work.
> 2. There's no way to reference count the shared memory - if a reader crashes, it holds on to the memory it was reading. You can get around this with some form of watchdog process, or by other schemes with a side channel, but it's not "easy".
>
> 3. Similar to 2, if a writer crashes, it will leave behind junk in whatever filesystem you are using to hold the shared memory.
Indeed, this is a complicated topic and support from the OS would be appreciated. We found a few ways on how to make this feasible, though.
The origins of iceoryx are in automotive and there it is required to split functionality up into multiple processes. When one process goes down, the system can still operate in a degraded mode or just restart the faulty process. With this, one needs an efficient and low-latency solution else the CPU is spending more time on copying data than on doing real work.
Of course there are issues like the producer mutating data after delivery, but here are also solutions for this. It will of course affect the latency but should still be better than using e.g. unix domain sockets.
Fun fact. For iceoryx1 we supported only 4GB memory chunks and some time ago someone came and asked if we could lift this limitation since he wanted to transfer a 92GB large language model via shared memory.
Thanks for the tips. We have a comparison with message queues and unix domain sockets [1] on the repo on github [2].
~~It's nice to see that independent benchmarks are in the same ballpark than the one we perform.~~
Edit: sorry, I confused your link with another one which also has ping-pong in its title
We provide data types which are shared memory compatible, which means one does not have to serialize/deserialize. For image or lidar data, one also does not have to serialize and this is where copying large data really hurts. But you are right, if your data structures are not shared memory compatible, one has to serialize the data first and this has its cost, depending on what serialization format one uses. iceoryx is agnostic to this though and one can select what's the best for a given use case.
Besides being written in Rust, the big difference is the decentralized approach. With iceoryx1 a central daemon is required but with iceoryx2 this in not the case anymore. Furthermore, more fine grained control over the resources like memory and endpoints like publisher. Overall the architecture is more modular and it should be easier to port iceoryx2 to even more platforms and customize it with 3rd party extension.
With this release we have initial support for C and C++. Not all features of the Rust version are supported yet, but the plan is to finish the bindings with the next release. Furthermore, with an upcoming release we will make it trivial to communicate between Rust, C and C++ applications and all the other language bindings we are going to provide, with Python being probably the next one.
I've been looking around for some kind of design documents that explain how you were able to ditch the central broker, but I haven't found much. Do you have breadcrumbs?
This is a longer story, but I'll try to provide the essence.
* All IPC resources are represented in the file system and have a global naming scheme. So if you would like to perform a service discovery, you take a look at the `/tmp/iceoryx2/services` list all service toml files that you are allowed to access and handle them.
* Connecting to a service means, under the hood, opening a specific shared memory identified via a naming scheme, adding yourself to the participant list, and receiving/sending data.
* Crashing/resource cleanup is done decentrally by every process that has the permissions to perform them.
* In a central/broker architecture you would have the central broker that checks this in a loop.
* In a decentralized architecture, we defined certain sync points where this is checked. These points are placed so that you check the misbehavior before it would affect you. For instance, when a sender shall send you a message every second but you do not receive it, you would actively check if it is still alive. Other sync points are, when an iceoryx2 node is created or you connect or disconnect to a service.
The main point is that the API is decentralized but you can always use it in a central daemon if you like - but you don't have to. It is optional.
Same here. Shared memory is one of those things where the kernel could really help some more with reliable cleanup (1). Until then you're mostly doomed to have a rock solid cleanup daemon or are limited to eventual cleanup by restarting processes. I have my doubts that it isn't possible to get into a situation where segments are being exhausted and you're forced to intervene
(1) I'm referring to automatic refcounting of shm segments using posix shm (not sys v!) when the last process dies or unmaps
Now I'm curious. It's seems you are not the father I'm still drinking beer with. This means there is only one person left that fits this attribute :) ... we should meet for some beer with the other father ;)
Yes, there is some overlap but iceoryx and threadprocs could complement each other. Currently, we achieve true zero copy only with data types that are shared memory compatible, i.e. no heap, no self references, etc., basically the data type must be memcopyable. Cap'n Proto or the Apache Arrow columnar format would also work. For other data, e.g one that uses the heap, there would still a serialization step be required but at least there is no further copy between the processes.
With threadprocs, it would be possible to also send data structures that use the heap or are self-referential. iceoryx2 already works in the same way as the proposed solution for the deallocation. There is a submission channel to provide the data and a completion channel where the pointer are sent back to the producer. Currently, we do not call any d'tor, since it is not necessary, but that should not be too difficult to implement.
reply