Danger: multiple processors processing the same capture
When working with @pguetschel with the ProcessorLoopClosure
we realized that there is an important issue that we didn't consider before.
When there are more than one processor for the same sensor, these processors will have a pointer to the same capture. This has two dangers:
- Frame/key-frame linking: Due to a KF callback or a KF creation, one processor can link the (shared) capture to a KF. This is followed by the corresponding factors emplacement. Then, the other processor will process the corresponding KF callback and try to link the capture with this new KF (error since it was already linked to a KF). Still, the processor has to emplace the factors corresponding to its nature...
-
Removal: Now, in the processor trackers, when a KF is not created in
last
, it is removedlast->remove()
. It removes all its features (of any type) and unlink from the frame, unregister the corresponding state blocks... However, another processor may still want to work with this capture. The capture should be removed if the processor is the only one processing it but not otherwise..
I imagine 2 different options to handle this issue:
A. Duplicate captures: It is the easiest way. Each processor has its own copy of the capture. However, then we duplicate extrinsics and intrinsics.. And eventually we could have different frames containing two copies of the same capture...
B. Consider possible changes and CaptureBase::processing_by_list
: It means considering 1 when dealing with KF callbacks. So, checking if the linking to the new KF should be done or it was already done by another processor. Also, adding a list of processors that have a pointer of the capture (something like CaptureBase::processing_by_list
) with the corresponding machinery to handle the removal of the capture. It is less multi-threading-friendly than A but I think it is the proper way to go.
Is there any other solution? Opinions?