HARDWARE FLOW CONTROL FOR HIGH SPEED COMMUNICATIONS

This must have happened to all of us at some time or the other while using a modem:

You are transferring a file, the error count keeps building up, and finally the transfer crashes. Bad line ? Not necessarily. As a matter of fact, it could be too good a line that's causing the problem. Data could be coming in too fast to be handled by your computer !

What's faster - your 486 or your serial port ?

Ever sat through a file transfer that seemed to take forever for even a small 100 KB file ? And, to make it worse, after the transfer was over it took just two seconds to copy the same file from one disk to another ? Can anything be slower ?

And yet it is possible for a modem to run faster than your computer can handle it !

Modems have always been rated very low in terms of speed, and saying that a modem could "outrun" your computer would bring pitying smiles on the face of the listeners. "Outrun my 486 based, 33 MHz, Cache RAM based super desktop ?" they would say.

After all, a modem typically works at 2400 bps, or 240 bytes per second (in asynchronous communication, 10 bits are required to transfer a single 8 bit byte). How can something as slow as that be faster than even a 4.77 MHZ PC ? Ridiculous !

But think again, friends. There are darker sides to your PC that make it an ideal target for "Comm Overruns". You don't believe me ? Well, read on, and be prepared for some surprises.

The Dark Side of the Moon

The PC (whether 8088 based or 80486) communicates with your modem via its serial port. This includes internal "modems-on-a-card", which typically have serial port circuitry onboard, but suffer from the same serial port problems as an external modem nevertheless. And this serial port functions on the principle that it receives one character from the modem, then waits for the PC to pick up that character, thereby clearing the way for the next character. This includes internal "modems-on-a-card", which typically have serial port circuitry onboard, but suffer from the same serial port problems as an external modem nevertheless.

Should a character arrive from the modem when the earlier character has not yet been picked up by the PC, then we have an overrun.

"OK, but why should the PC not pick up the character waiting in the serial port's buffer ? After all, the PC is thousands of times faster than the fastest speed that the serial port will work at !" says the sceptic.

True, a PC is much faster than its serial port, but there is a catch - a PC is sometimes not free to service the serial port. And just like even a Ferrari Formula 1 racing car cannot win a race if it doesn't even participate in it, the character waiting at the serial port can get overrun because the PC isn't checking for that character.

"How come ?"

To explain that, I'll have to give you a bit of background on how communication software deals with the serial port.

May I interrupt you for a moment ?

Modern communications no longer works on the basis of "polling" (checking every now and then) the serial port for characters. Instead, it relies on interrupts from the serial port to inform it that a character is ready to be picked up. This is much faster than polling, and is the way 99% of modern communications programs work.

To understand this, picture yourself sitting in an office, with a phone by your side. Now the phone is a useful thing to have, but its utility would be severely curtailed if you had to pick up the receiver every now and then to see if there is a call coming in.

Naturally, the phone has a way to tell you that there is a call waiting to be answered. It rings a bell. So you stop whatever you are doing, pick up the phone and talk. If the phone doesn't ring, you know that there is no call, and can go about doing something else.

The PC's communication interrupt mechanism works in a similar fashion. Instead of constantly checking the serial port, the PC relies on the fact that when a complete character has been received and needs to be picked up, the serial port sends an interrupt to the CPU, effectively tapping it on the shoulder and saying "Hey ! Could I have your undivided attention for a moment ?" The CPU passes this signal to your communications program, which in turn picks up the character from the port.

Unfortunately there are instances when the PC "takes the phone off the hook" for a while by turning off all interrupts, thereby making it incapable of receiving an interrupt from the serial port. If this happens, the CPU never realises that there is a character waiting to be serviced, and as soon as the next character comes in, it kicks the original character out of the buffer, causing data loss.

Paradoxically, the most common of such instances is when the PC is accessing the disk ! When this happens, the PC's BIOS turns off all interrupts until the disk access is complete. And if this happens for long enough, we get overruns at the serial port !

How long is "long enough" ?

Assume you are receiving data at 2400 bps from the modem. This means 240 characters per second, or one character every 1/240 of a second. So, the first character is ready after 0.004 of a second, the next one is there after 2/240, or 0.008th of a second. And that, my friend, is your "long enough" time. If your PC hasn't picked up the first character by the time the second character is fully received, then an overrun takes place !

Now just take a typical disk access. Assume that you are receiving data and, after every 1024th character, are writing this data to disk. The instant you start writing to disk, the PC turns off all interrupts, and begins the write operation. Now, if your hard disk is slow, this can take as long as a second (longer if the target is a floppy disk). That means that your PC could be "out of action" long enough to lose upto 240 characters !

Haven't had enough yet ? Consider that most modems today are MNP or V.42 modems, and are capable of operation at 9,600 to 34,000 bits per second, the "time out" is shortened radically ! If you take as an example a modem delivering data at 9600 bps to your PC, then the "long enough" time is 2/960th of a second ! Want to really fall into depressions ? Try 34,000 bps on your calculator !

The Electronic Traffic Cop

OK, enough of doomsday. Now that you know WHY overruns happen, let's see what we can do about it.

Actually, the matter is simple. Just tell the modem to stop sending data for the duration of the disk write.

"Aha !" you say "send an XOFF (Ctrl-S) to stop the flow !"

Nice try, my friend, but no cigar.

If your modem (or the remote source of the incoming data) is set up to recognise XOFF as a "hang on a sec" signal, then a spurious XOFF (caused by line noise or even program error) can send your communications link into limbo. Besides, sending an XOFF takes too long, anyway. And if you are transferring binary data such an EXE file using something like YMODEM or XMODEM, an XOFF that is part of the data can be misinterpreted as an error.

The answer lies in the very definition of the serial port's signals. If you have a description of a serial port's signals in front of you, you'll see that two of the pins are marked as "RTS" and "CTS". On a standard 25-pin serial port, they are pins 4 and 5. Now there must be a reason for these signals to be there in the venerable CCITT's recommendation of RS232C. And there is. These signals are known as the "hardware handshake lines" or, more commonly, as the "hardware flow control" lines.

The RTS and CTS signals are interpreted as follows:

When the PC is ready to receive data, it raises its RTS (Request To Send) signal. This causes the CTS (Clear To Send) signal of the modem (or remote device) to be raised, indicating that it is safe to send data.

If the PC needs to halt the flow in order to do something like a disk write, then it lowers the RTS signal, which causes the remote CTS to drop, telling the sender to "hang on while I do something".

Since the RTS and CTS signals are not part of the data stream, they are transparent to the actual data flow, and do not interfere in any way, making them ideal for purposes of flow control.

Using Hardware Flow control

So how do you implement it when using your communications program (such as Procomm or Crosstalk) ?

Easy enough. Just enable this hardware flow control setting in the program's setup. And make sure that your modem is set up to recognise them, too. In Procomm this setting can be found in the Terminal Setup, in Crosstalk, it is in the main setup screen. While you are there, make sure that you disable software flow control (XON/XOFF) which has no place in high speed communications, anyway.

Secondly, make sure that your serial cable from the PC to the modem has all the nine critical wires connected correctly. Some people have been using three-wire cables for modem communications, which is not enough if we are talking about speeds in excess of 300 bps.

Your modem needs to be set too. On most modems, the AT&R0 setting causes the modem to let RTS and CTS act normally (there may be some jumpers involved). On most MNP modems, the AT\Q3 setting will enable the modem's hardware flow control (on some it is AT&E4&E7). Check your modem's manual for this.

Now try downloading (transferring to your computer) that king sized file that you have been wanting for so long, but always failed to get because the transfer failed. Voila ! It worked !

Fish 'n' Chips

There are some other things you could also do to ensure that your communication session doesn't cause you grief. One of the most common problems in PCs sold in India is that the serial port chip (known as the UART) on the I/O card is very often not meant for use with that particular machine.

Specifically, all BIOSs written for the 80286 and above expect to find a chip called the 16450 managing the serial port. But some "fishy" manufacturers, in order to save money, use the older 8250B chip. What most people do not know is that the 8250B has a documented bug in it that causes the chip to "lock up" at random when operating at speeds in excess of 1200 bps. Older XT BIOSs were written keeping this bug in mind. But the newer AT and 386 BIOSs were not. That is why you can virtually guarantee disaster if you are using such a chip on an AT class machine.

I use an public domain utility called UARTID (written by TurboPower) that tells me what chip is in use on a machine, but you can also check this out physically by sneaking a peep at the I/O card in your PC. This can be a problem if your I/O card uses one of those "all-in-one" ASIC (application specific integrated circuit) chips, which hide the true identity of the circuitry. In such a case, you WILL need the UARTID utility.

If you find an 8250B chip in your AT class machine, change it pronto to a 16450, and many of your problems will go away.

You could also use the newer 16550AF UART in place of the 16450. This chip has a special buffer built in that avoids port overruns. Many newer communications packages recognise and activate this chip, resulting in almost perfect data flow.

RAMbling on

Finally, a little tip for those concious of the clock when they are online. Instead of downloading onto a hard disk, try downloading onto a small RAM disk. This RAM disk should be about 10% bigger than the largest file you plan to transfer.

With protocols such as YMODEM-G BATCH or ZMODEM, this can make a substantial difference in transfer time and reliability, since the time taken to write to a RAM disk is negligible compared to the time taken to a Hard or Floppy Disk. Because of this, the time that the PC "takes the phone off the hook" is reduced considerably.

Winding down

Oh oh - did someone ask "what are YMODEM-G BATCH and ZMODEM?" I thought you knew about those !

You didn't ?

Well, they are two of the most popular file transfer protocols in use today, and they, along with other protocols, feature in next month's column, where I will be talking about different aspects of file transfers using modems, including which to use when and how to achieve the fastest possible transfer times.

See you next month !