This is a continuation of our blog series on the advanced functions for network visibility solutions with fully programmable data pipelines. Read our introduction to the series here.
The biggest elephant in the network visibility room has always been the fact that you don’t know what you are missing. If there are packets with structures that fall outside of your fixed-function ASIC-based chip/solutions capabilities (and trust me, there always are), you are not only going to drop those packets, but you also have zero indication that you just missed a packet. The packets simply fail to parse, and you will not know that you are missing that information. You can’t control, what you can’t see.
Know the unknown packets
A fully programmable data pipeline (managed by software that controls the chipset) provides such an unprecedented level of control and change in processing mentality that this concern is no longer valid. For starters, you now have the tools at your disposal to directly determine what can be seen, and how it should look on the wire. If you have a fairly solid understanding of the complex packet types found on your network, you are already in a position to never miss a packet. However, if there is traffic that falls outside of your defined parameters for processing, this data is no longer “lost without a trace”.
Dynamic filtering with fully programmable data pipelines
Fully programmable data pipelines alert you if there is a “parse fail” for any particular packet- a capability that simply did not exist in any previously released network visibility solution. Better yet, you can set rules in place so that any packet that fails to parse is sent to another port on the device, which can be connected to a packet recorder or any 3rd party storage platform. You now have a means to collect the data you are missing, pull it up in wireshark (or another analytic tool) to inspect it, and reverse engineer the packet structure and then adjust the capabilities of your device (with some quick changes to the programmable software), and no longer miss that traffic type.
Unique header stripping and keying
Aside from having the ability to capture the packets you are missing, a programmable chipset allows you to accomplish additional unique network visibility functions. One example is being able to strip unique header values in order to “normalize” network traffic. Imagine you have recently purchased a firewall from one of the leading vendors, only to find that once you pipe data to it, nothing can be seen at all- it is showing that there isn’t any traffic coming in. This is the exact situation an organization faced where programmable chips were able to solve a problem in a unique way.
Header stripping
This organization was utilizing the Shortest Path Bridging (SPB) network structure, which is based on the 802.1aq network protocol. The problem was that the recently purchased firewall didn’t understand how to deal with the header structures within 802.1aq traffic, and therefore was unable to gain visibility in to the data it was meant to protect. Their solution? They leveraged a programmable chip-based MantisNet RFP-NG to normalize the traffic for the firewall.
The RFP-NG was placed in-line to the network firewall, with touch points to the data stream directly before, and directly following the firewall itself. The network traffic would pass through the RFP-NG which in turn removed the “problem” header values, and then presented the traffic to the firewall as standard ethernet packets that it is used to seeing. After doing what the firewall is meant to do, the egressed traffic once again passed through the MantisNet RFP-NG, which reinserted the “problem” headers back in to the packet structure before passing the information on back in to the network- with all of this occurring at true line rate.
Keying
Another example of unique functions that can be performed with a programmable chip is that of keying off a specific header value for load balancing or filtering purposes- essentially being able to identify a header value within a complex or overly encapsulated packet type, and taking action based on that value. Standard ASICs come with a fixed set of packet headers and graphs they can identify, and their forwarding logic is cemented into the chip itself. This does not lend itself well to load balancing and filtering complex packet traffic.
Programmable chips allow for forwarding logic to reside in a program (code) which the network operator loads onto the chip; it is not hard-wired into the silicon. This turns the chip found within your network visibility solution into a fully programmable interface. These devices now can parse (and load balance based on any type of hashing) on any type of packet found within a network- for example, it is straight forward to now load-balance traffic on an inner IP value, say for monitoring GTP traffic.
Get a handle on unknown packet identification, complex parsing, match-action and de-parsing with with MantisNet’s RFP-NG.
You can read more about our list of advanced functions made possible with fully programmable data pipeline in our 'Can your packet broker do this?' series:
- Complex parsing / identifying unknown packets / de-parsing of packets
- Identify packets that don't match existing header definitions and parse graph (e.g. packets that don't parse) and direct packets out to a follow-on system for reverse engineering and further characterization
- Unique header / network overlay stripping capabilities (multiple VLANs, 802.1aq)
- Can hash and load balance based on innermost IP information in complex packet formats
- Can generate time-series data metrics for all applied ACLs (via ONT)
- Ability to set ACLs (filters) and generate octets and packet stats natively
- Cloud native instrumentation via Prometheus and Grafana
- INT (in-band network telemetry): ability to insert metadata headers with specific data of interest
- ONT (out-of-band network telemetry): ability to export metadata with specific data of interest
- Software-based, extensible, architecture; dynamic addition of protocols for full programmability of the pipeline
