Although the PCI-bus computers remains a reasonable solution for many low end I/O problems, the PCI-bus immediately shows its weaknesses should we need to cope with high end data acquisition systems. Let’s take the problem of interfacing high performance camera detectors, such as FReLoN, Dalsa or Sarnoff, spiced with some future architectural expectations:
high_speed_challenge.pngWith the theoretical data transfer rate of 133 MB/s, the PCI-bus is even theoretically over its limits for this type of problem. Even more so in the real world where the shared bus structure is used to connect Ethernet controllers, disk controllers and other high speed peripherals.

Although the server level PCI-X bus would double the available bandwidth it does not resolve the problem of sharing the same bus between high speed devices.

The PCI Express moves the highly successful PCI specification as it is on a serial, switched bus system. Comparable to switched Ethernet in certain aspects, the PCI Express allows non-shared, point-to-point connections between devices with data rates of 2.5 Gbit/s and 5 Gbit/s in both directions, simultaneously. Each pair of wire is called a lane. You can have 16 lanes for a graphics card, 8 lanes for a storage controller, 1 lane for a Gigabit Ethernet controller. More lanes multiply the theoretical maximum data transfer rate (ex. 16 lanes gives aggregated data transfer rate of 8 GB/s - 8-to-10 bit encoding applied).
pci_express_fabric.png
The PCI Express allows a star like system architecture with strong resemblance to the modern switched Ethernet fabric. Point to point connection is established between the PCI Express connected devices and the CPU. Also the direct memory access (DMA) and other resources are available to PCI Express devices without having to share the data bus with other devices in the same system.

The PCI-bus protocol is still present, contained in a transmission protocol, encapsulated in data transmission packets. The packets can be prioritized and routed. From the PCI Express device’s point of view it “owns” the data connection. The performance is limited to the quality of service that the communication switch is able to deliver to different, simultaneously connected devices. We can observe again the similitude with the telecommunications terminology. The existing PCI-bus based devices can be connected to the PCI Express based over a PCI Express to PCI-bus. The below picture illustrates an example of a PCI Express switched fabric.
 

Read the following report (pdf) for more interesting information about PICMG 1.3 industrial SHB computers: