Integration of ipfixcol2 with DPDK

[Anant897]

I would like to request guidance or support for integrating ipfixcol2 with DPDK.
Currently, ipfixcol2 operates using standard Linux network stack interfaces, but for my use case I need to handle high-speed traffic capture.

[sedmicha]

Hello! Are you sure integrating ipfixcol2 with DPDK is really what you'd like to achieve? Ipfixcol2 is only the collector part of the monitoring infrastructure that collects flow records and stores them/processes them further, the part that handles the actual traffic and creates those flow records is a probe, not the collector itself. We do also have a probe that you can use with ipfixcol2 and it already has DPDK support: https://github.com/CESNET/ipfixprobe

[Anant897]

Thank you for the clarification.
Suppose on the machine running the collector (ipfixcol2) we start receiving flow records at a rate of millions per second — what would be the recommended way to scale the collector’s capacity to handle such high throughput?

[sedmicha]

That really depends on the specifics of your setup. First I'd try out the simplest configuration to see if you even need to do anything special -- in our testing, ipfixcol2 could handle a few millions of records per second on a pretty basic machine, but it highly depends on the structure of your flow records and output plugins. If you find out that the throughput is not sufficient, you could for example scale by having multiple instances of input plugins on different ports, multiple instances of output plugins each processing different ODIDs, or even going up a level and having multiple collectors with one serving as a "load balancer proxy" using the forwarder plugin. To go into more detail and give you a more specific recommendation I'd need to know more about your intended setup.