Understanding PCIe lanes

Got a shopping cart of parts that you want opinions on? Get advice from members on your planned or existing system (or upgrade).

Moderators: NeilBlanchard, Ralf Hutter, sthayashi, Lawrence Lee

Post Reply
object88
Posts: 13
Joined: Wed Jan 11, 2006 4:23 pm
Location: Denver, CO

Understanding PCIe lanes

Post by object88 » Wed Dec 30, 2015 9:43 am

This isn't so much a question about cool & quiet computing, but rather shows my lack of understanding of modern computer architecture. It's been a long while since I built a machine and cared this much about its overall assembly, so I have some catching up to do.

The short version: considering an i7-6700 & z170 combination..., how are PCIe lanes used & assigned?

In more detail, I want to build a machine that uses an M2 boot SSD over PCIe x4 on an mATX MB, and either one or two GPUs (over SLI / Crossfire if two, obviously). The MB I'm looking at has (in addition to the M2 slot) two PCIe x16s and one PCIe x4. I know that the i7 itself has 16 PCIe lanes, then (up to?) another 20 are available from the southbridge, over DMI.

How are these lanes assigned? Is this automatic, or do modern BIOS's allow for custom assignment?

If I have two GPUs, both on x16, does it matter if one is on the CPU's native lanes and the other on southbridge? Or does it make more sense to assign them as a pair of x8? Is there any penalty for going over the DMI bridge?

Are there any potential complications from adding secondary SSDs over more PCIe slots / lanes, so to avoid the slower SATA route?

Thanks for any education!

CA_Steve
Moderator
Posts: 7650
Joined: Thu Oct 06, 2005 4:36 am
Location: St. Louis, MO

Re: Understanding PCIe lanes

Post by CA_Steve » Wed Dec 30, 2015 1:13 pm

Take a look at the first diagram in SPCR's i7-6700K review. The 16 PCIe 3.0 graphics lanes come out of the CPU. The accompanying Z170 chip has another potential 20 lanes (your motherboard model may vary) split into five sets of 4x 3.0.

So,
- You put the gfx cards in their assigned slots and they get their lanes from the CPU.
- Your storage stuff doesn't affect your gfx cards..unless maybe if you were to get a PCIe card NVMe SSD and throw it into one of the gfx card slots.
- I've seen at most two M.2 sockets on a mobo.
- gfx cards and PCIe bandwidth. Two fun articles: Puget Systems Nov 2013 and Guru 3D Feb 2015

That said, there really only a few applications out there that can and will make use of PCIe's bandwidth for SSDs. For most cases (including gaming builds), SATA is just as fast in real life (as opposed to write and read speed benchmarks).

object88
Posts: 13
Joined: Wed Jan 11, 2006 4:23 pm
Location: Denver, CO

Re: Understanding PCIe lanes

Post by object88 » Wed Dec 30, 2015 1:27 pm

Well, yeah... I know about the origins of the various PCIe lanes. :) What I don't know is what maps the lanes to their usage or physical slots. You said "put the gfx cards in their assigned slots", but... what determines what that assignment is? Is this hardwired on the MB (and hence, the MB manufacturer should dictate which those slots are)? Is this a BIOS option?

Thanks!

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Re: Understanding PCIe lanes

Post by washu » Wed Dec 30, 2015 3:29 pm

While there are exceptions, in most consumer level MBs the slots will be hard wired to either the CPU lanes or the chipset lanes with no way of changing it. On lower end boards it is probably safe to assume the x16 slot is the one connected to the CPU. On higher end boards with multiple physical x16 slots then you will need to check with the manual to be sure how they are connected. Some boards split the x16 from the CPU into two or more slots, but change the lane assignment depending on how many cards are installed. Some boards may have physical x16 slots that are only 4x or 1x and connected to the chipset.

object88
Posts: 13
Joined: Wed Jan 11, 2006 4:23 pm
Location: Denver, CO

Re: Understanding PCIe lanes

Post by object88 » Thu Dec 31, 2015 7:37 am

Thank you, washu!

Post Reply