Fibre Channel FAQ, written by Chris Kalisiak,
last updated 18-NOV-03. Copyright 2003 Chris Kalisiak, except where noted.
All rights reserved.
Are you looking for Fibre Channel products? Visit our
store or our listings on
eBay.
Q: I don’t know much about Fibre Channel. Help?
A: Here’s a pretty good link on Fibre Channel in general:
For additional questions about how to use my T-Cards, please read on.
Q: What are the different versions of T-Cards that are offered?
A: There are currently three types of T-Cards. A "standalone" T-Card (previously
more often commonly used as a "screw terminals" T-Card), a "dual screw terminals"
T-Card, a "start" T-Card, and a "loop" T-Card.
The "standalone" T-Card with a DB9 connector and a set of screw terminals
can be used by itself, without any patch cables. It's ideal for when a single
T-Card is required, and can be expanded to a loop using short bits of Cat5
wire, connecting them together via the screw terminals.
The "dual screw terminals" T-Card is the most appropriate solution when the
user wishes to set up a dual-loop. Fibre Channel disk drives are dual-ported,
meaning that two Fibre Channel interfaces can connect to each drive, where
an interface can be two PCI cards in the same computer, or in two different
computers, for example. The dual-porting feature allows potentially increased
performance and/or port failover support.
The "start" T-Card is the first T-Card in a typical loop. It may be used
by itself by installing a short Cat5e patch cable between the "out" RJ45
and "in" RJ45. There is no "terminator" used with any of my T-Cards. The
"out" from the start T-Card connects to the "in" of the next T-Card, and
so on until the patch cable connects from the "out" of the last T-Card in
the loop to the "in" of the start T-Card.
The "loop" T-Cards can only be used with a start T-Card.
Q: How many drives can I connect together with
the T-Cards?
A: Using the T-Cards with RJ45 connectors and short Cat5e cables, you
can connect up to 8 T-Cards in a single loop, using 3ft cables, or up to
12 T-Cards in a single loop, using 1ft cables. If you want to connect more
than a dozen drives, then you're better off breaking up loops across multiple
Fibre Channel PCI cards (also known as an HBA, or Host Bus Adapters) or,
if that's not practical, then breaking up loops across ports on a Fibre
Channel hub (to increase signal integrity) is a good idea.
The key here with the Cat5e patch cables is to keep the total length of wire down to below 20' if you can help it.
The T-Cards with screw terminals can also be used in loops of up to 8 at
a time, using approximately 1ft pieces of Cat5 wire. These T-Cards seem to
work best with less than 12' of wire, although that might be because of the
Cat5 wire that was used, rather than the Cat5e that I currently use with
the RJ45 T-Cards.
Q: How do I wire up the T-Cards I received, using
Cat5 wire?
A: This question is applicable to T-Cards sold with screw terminals.
T-Cards that are sold with RJ45 connectors are simple to connect together
-- just use short straight-through (_not_ crossover) Cat5e patch cables from
"OUT" to "IN" until you reach the T-Card you started with, and complete the
loop.
So, now on to answer the question about wiring the T-Cards with screw terminals.
Here are a few suggestions I can offer with regards to wiring the T-Cards.
I've made enough minor mistakes with wiring to know that since this is all
low voltage, low current, so you can't really hurt anything. It usually just
takes a couple minutes to diagnose a miswiring problem, when using bits of
standard Cat5 wire.
There's really only three things you can do wrong -- swap the polarity
of the in/out wires, swap the in/out pairs, and not tighten the screw terminals
enough.
1) The polarity is fairly easy, if you always use the white/whatever
wire as the negative, and the whatever/white as the positive. The four
screw terminals are laid out as + - - +, so just make sure that the white
wires are always in the middle.
2) Swapping the in/out pairs is easy to visually confirm, too. Screw
terminal positions 1 and 2 on one T-Card go to positions 4 and 3 on the
next T-Card, so just make sure the colors alternate as you check the loop.
3) Well, if you forget to tighten one of the screws,
then, well you’ll see intermittent problems.
Regarding the DB9 to tinned ends that I offer, basically,
you have two pair of wires exposed at the one end of the section of Cat5.
It's going to be blue and brown, or blue and orange, or some other combination.
You can't hurt anything by miswiring, so just try both combinations until
it works, remembering that the white wires always connect to pins 2 and
3.
Let's say, for sake of argument, it's a blue pair and an orange pair.
Try installing the blue/white in socket number 1, the white/blue in socket
number 2, the white/orange in 3, and the orange/white in 4. If that doesn't
work, then reverse the pairs: orange/white in 1,
white/orange in 2, white/blue in 3, and blue/white in 4.
Just remember that both wires of the pair need to be next to each other,
and that the white wires of each pair need to be in the middle connectors
(2 and 3).
Q: What about spin-up at power-on vs. spin-up
at command?
A: The T-Cards can be configured to spin up all of the drives at the
same time. This might end up putting too much of a strain on the power
supply you'll be using, depending on how many drives, how much current
they draw at spin-up, and what the power supply rating is. If the power
supply is overloaded, that might cause some drives to spin down as others
are trying to spin up.
If this is the case, you can change the "always spin up" setting to "spin
up on command" by removing the start jumper labeled "ST2". It's the "start"
jumper further from the screw terminals. If ST1 is present on your T-Card,
the "start" jumper that's closer to the screw terminals, it always needs
to be installed. If ST1 is not installed, he drive basically won't spin up
at all.
The spin up on command setting specifies that the drives spin up one
at a time, as the HBA instructs each drive to spin up during the discovery
process.
Q: I have a QLA2100F, QLA2200F, LP7000E w/SC GLM, or some other optic card
with an SC connector. Will the T-Cards help?
A: Unfortunately, that card has an optic connector, and disk drives
have copper interfaces. There is no trivial solution to this problem. All
possibilities involve additional purchases of hardware. Please note that the following prices are only valid
in conjunction with the purchase of an eBay auction.
You have a few options at this point. One is to purchase a "media interface
adapter". They are not very common on eBay, but you might be able to find
one up at any given time. The T-Cards with screw terminals need a custom
cable to power the adapter, which I could provide for $15. The T-Cards with
RJ45 connectors have MIA support built into the "START" T-Card. See the description
of what a MIA is elsewhere in this document.
A second option is to put your recently-purchased QLA2100F on the shelf,
and buy another PCI card, but with a copper connector. I can offer a card
based on an LSI Fibre Channel chipset, the FC40909, and is “fabric aware”,
so it will work with fairly modern Fibre Channel switches. The cable included
with each card exposes a DB9 male, with the ability to connect directly to
a T-Card. I can offer these PCI cards, guaranteed to work with my T-Cards,
for $50 each, including the cable. This LSI card does have Win2k support,
and the Win2k driver works fine under WinXP.
So, the third option is to swap PCI cards, trading your optic card in
exchange for a copper card. Let me know in advance that you'd like to swap,
and let me know what kind of card you have, and we should be able to work
something out.
Q: I have a copper PCI card but no cable. How do I connect
it to the T-Card?
A: Depends on what kind of card you have -- there are two standard Fibre
Channel connectors for 1Gbit Fibre Channel. Either way, I can offer a cable.
QLogic and LSI Logic cards use a connector called "HSSDC". This connector sort of looks like a squashed RJ45.
Emulex cards use a DB9 connector which is basically the same connector as a standard serial port.
HSSDC-DB9 and DB9-DB9 cables are available in various lengths, from approximately
2ft to 20ft, for $10 to $20. Please ask for details.
Q: What Fibre Channel PCI cards work with these
T-Cards?
A: In theory, any Fibre Channel card with a copper interface should
work. I’ve personally used the following setups:
-
Direct attach eight drives to an Emulex LP6000 and then an Emulex LP7000.
-
A few drives attached to a Vixel R4000 switch, with LP6000 and LP7000 clients.
Note that this combination was tricky, because the R4000 only works with
Emulex cards.
-
A few drives basically direct attached to QLogic QLA2000/QLA2100 cards,
connected through a hub. Hubs are basically direct attach, because they
only repeat the Fibre Channel signal, and don’t provide any intelligence.
-
A few drives attached to a Brocade Silkworm 2400 with QLA2000/QLA2100 clients.
This works, but it doesn’t work as well as it should. In order to properly
do this, you appear to need to have “fabric aware” clients, such as the
QLA2200. The LSI Logic-based Fibre Channel cards that I offer also work well with the Brocade Silkworm 2400.
- More recently, I have acquired a 16-port QLogic SANbox.
This seems to be a good switch, comparable in features and performance to
the Brocade Silkworm 2400, but at about half the cost on eBay. I'd recommend
the SANbox to anyone looking for a low-cost, feature-rich Fibre Channel switch.
Q: What hard drives work with these T-Cards?
A: In theory, the preceding answer should cover this one. Fibre Channel
drives are all basically the same, so they should all work.
Q: I have a SCSI card, for example an Adaptec
AHA-2940UW. Will the T-Cards allow me to connect these drives, that turned
out to be Fibre Channel, to my PC?
A: Sorry, the only type of adapter that would allow you to connect
a FibreChannel disk drive to a 2940UW is a multi-thousand dollar bridging
product... FibreChannel and SCSI are completely different interfaces, even
though FC basically uses SCSI commands. But that topic is beyond the scope of this discussion.
What you'd need to do is pick up a FibreChannel PCI card with a copper
interface, such as the card I provide. Or yell at the seller for not clearly stating that the disk drive was
FibreChannel.
Q: Similar question as above, except I want
to connect these drives to the IDE interface on my motherboard.
A: Same basic answer as the SCSI question, except you’ll need a crash
course in high-end storage concepts. Does the phrase “think outside of
the box” mean anything to you?
Q: How much faster is Fibre Channel than SCSI
or IDE?
A: Would you care to try for double jeopardy?
Seriously, the question isn’t really valid. You have to look at the
specs for the drives themselves to see what’s faster, if you’re talking
about connecting disk drives to a machine with a Fibre Channel / SCSI /
IDE interface.
Yes, the theoretical throughput of 1Gb Fibre Channel is 100MB/sec.
That’s slower than what you can get with current SCSI (160MB/sec and 320MB/sec)
or even IDE (133MB/sec) interfaces.
However, that’s really not looking at the big picture. With Fibre Channel
and SCSI, you can attach many devices to a bus to aggregate throughput from
multiple drives. IDE is fine for use as the boot disk for a desktop machine,
but it's not very scalable, and it's not possible to physically share IDE
drives.
But I’d prefer not getting into a religious debate over the merits of
IDE vs. everything else. The reason being is each of these three mentioned
interfaces has its place. All of my "desktop" machines, and even my secondary
server, have IDE boot disks. My primary server, which is up 24x7, has a SCSI
boot disk. And all of the machines share data through Fibre Channel.
Why is IDE not suitable for applications beyond these couple listed
above? Because that’s all it’s designed for. It’s not designed for scalability
or performance. It’s designed to be low cost. Sometimes low cost also ends up being low quality, with premature failures.
Getting back to the original question, the answer, again, depends on
the drive. Take a 5-year-old 9GB Barracuda with a Fibre Channel interface
and compare it to a 160GB 7200RPM ATA-133 drive, and you’ll find that the
160GB drive to be faster.
By the same token, put a dozen of those 9GB Barracudas in a striped
RAID, and you’ll find that those old drives are now running rings around
the 160GB drive, and are ready to crunch on the biggest A/V files your processor can handle.
Disk drives don’t keep up with CPU’s as far as advancements are concerned,
but they do improve over time. And nothing can beat the scalability or
connectivity of Fibre Channel. Yes, Fibre Channel had the distinction of
being the least interoperable interface available, since its inception.
However, as long as you either stick with the same manufacturer for all
of your cards if you’re doing it on the cheap, or stick with more recent
hardware, if you can afford it, then you should be in good shape. But don’t
expect old Emulex LP6000 cards to talk directly to similar vintage QLogic
QLA2100 cards; it just won’t happen.
Q: I’m having problems getting things set up.
Help?
A: There are so many variables involved that I can’t possibly hope
to address them all here.
I’ll try to offer an overview of some of the things I have picked up
on.
First, here’s another useful link on troubleshooting Fibre Channel
cards in a PC:
If you’re having problems with Win9x/WinME/WinXP, please note that
typically only WinNT and Win2k are supported for most Fibre Channel cards.
Check the manufacturer for driver availability; you just might get lucky.
A tip from one of my customers… You need to make sure that if BIOS
is enabled on the Fibre Channel card, and if you want to boot from a SCSI
drive instead:
“The card was supposed to be brand new, but it's true it was acting
like it wanted to boot first (when BIOS enabled) and resulting in the "NTLDR
not found" (Windows NT leader - something needed on a boot drive). This,
even if in the BIOS, the Boot setting was however not enabled. By looking
at the startup sequence, I could see that the Fibre card was initiliazed
first, then the SCSI card, and the Unit ID where #80 for my fibre drive
and #81 for my SCSI drive (the one I want to boot on) and this is regardless
of the device ID you assign to the drives... So my idea was to have the
SCSI card initialized first and then the fibre, in order to reverse thoses
unit IDs. And.... it worked, now the SCSI drive gets #80 and Fibre #81
so when come the time to boot, Windows look at the SCSI one so my system
now startup correctly - and the Fibre drive seem always accessible.”
The answer in this case was to swap the SCSI and Fibre Channel cards
such that the SCSI card was closer to the CPU.
Lastly, a note about QLogic optic cards: “Molex” brand optic transceivers
are known to be unreliable. I have personally confirmed six bad cards,
all with Molex parts. Siemens/Infineon appear to be ok in general. So,
if you have a choice between Molex and Siemens, for example, go with the
Siemens.
Q: Can I boot from a Fibre Channel hard drive?
A: Yes, you can, but I don’t.
If you are setting up a SAN or will do so in the future, then it’s really
not a good idea. Not unless you have an expensive switch (i.e. a Brocade
Silkworm 2400) with "zoning" features to keep other systems from stomping
on your operating system disk.
But if you're only setting up a single PC with one or more drives directly
attached to the HBA, then there's no reason why you shouldn't boot from
a Fibre Channel hard drive. That's assuming there is a boot BIOS installed
in the HBA's firmware, and almost all of them do.
Q: I bought some T-Cards from another seller,
but that seller wants $40 for a cable. (Editorial note: apparently the
T-Cards are a loss-leader to get you to buy the cables at 4x their market
value.)
A: I can offer short DB9-DB9 cables for about $10 each, but I can't offer
a terminator. None of my T-Card designs require a terminator, so I don't
have any immediately available.
You'll need to
be careful about cable lengths, though. I don't know what types of loads those T-Cards place
on the line, so just try to keep the lengths as short as possible.
Make sure you don’t just buy DB9 serial cables, though, because they’re
the wrong type of cable.
If you want to make your own cables, here’s what you should be able
to do. DB9 connectors and Cat5 wire should be sufficient. Make sure that
pins 1 and 6 are one pair with 1 being the solid color, and 5 and 9 are
another pair, with 5 being the solid color, and make sure the cables are
as short as possible. Make sure all of your cables are wired identically,
to avoid confusion later. If you're looking to JBOD 10 drives together,
you'll need to keep the lengths down to like 8" or so, if you can help
it.
Q: What about Shielded Twisted Pair?
A: I don't know of any commercially available STP patch cables, so if you have a source, I'd like to know.
The theoretical advantage of STP is the impedance is closer to what Fibre
Channel is supposed to be, and also can provide better resistance to noise
in high-noise environments. I don't really know what it would buy you though,
in a typical home environment. I haven't noticed any problems with the standard
Cat5e UTP cabling that I use and sell.
The UTP RJ45 connectors can be substituted with STP RJ45 connectors for an additional $5 per T-Card.
Media Interface Adapters:
Here’s an overview of Media Interface Adapters