~Backgroud to transferring data via modems.` ~==========================================` The following is based on a text file captured by Raymond Wood from a call to Jim Davis' Retreat (713 497-2306) in Houston, Texas. But it has been edited and updated, and Amiga-ised by AJW on 29-Jan-89. It is somewhat disjointed, but contains a significant amount of background information. Terminology and concepts defined: --------------------------------- ~Bits & Baud :` ... In the vernacular of the communications industry, there are a few concepts that need to be understood before understanding 'HOW' is accomplished. For example, the word BAUD. This essentially means 'bits per second'. A 'BIT' being the most basic element or bit of information a computer can handle - just a yes/no, on/off, or 0/1 quantity. In fact, baud means something a little different than bits per second, but for openers, let's say that's what it means. ~Speed :` So baud is a measure of the speed at which two machines communicate data. Obviously we would like to go as fast as possible, but there are two limiting factors - the speed at which the machine can actually feed data to its communications port with the Amiga the usual practical limit is 19200 baud. The other is the quality of the interconnection between machines - telephone lines can have a lot of noise (from cross- talk, rattling mechanical exchanges, electric fences, bad connections etc) this can be overcome by the application of complex technology, the real limit being cost. To communicate both machines must firstly agree on what speed to communicate at, usually you, the caller, are the one with the limited resources, so the other end automatically matches your speed. The most often used speed, today, is 1200 baud/bits-per-second. ~Bits & Bytes :` Most users have no idea how many bits are involved in a file or message transfer. Let's look at another standard word: BYTE. There are 8 bits of information contained in a byte. That is, a byte is merely a set of 8 bits. Within a set of 8 bits there are 256 permutations available. From all bits being zeros to all being ones. Each letter in the alphabet and each digit and each other special character is represented by a predetermined set pattern of those 8 bits. A capital 'J' has a different pattern than a lower case 'j', for example. One system of representing letters etc is called ASCII (American Standard Code for Information Interchange - ie a standard computer 'alphabet' to let computers talk to one another). It consists of 52 letters (upper and lower case), 10 digits, 32 punctuation characters, 1 space character, 33 control codes (newline, newpage, delete etc). Thus, we have another 128 that may be used for 'special purposes'. What, for example? I'll get to that. ~Data + Telephone => Modem :` The sending of bits (on or off, high or low, in other words binary information) is, by definition, a binary process. That is, the computers need only recognize one of two states. The telephone, on the other hand, carries information that is other than binary. It can faithfully represent different tones, pitch, and volume. This is called analog rather than binary. The almost sole purpose of a modem is to translate binary signals into analog and vice versa. When you are going to send a set of bits across a telephone you will have to convert those binary 'states' into some form of sound (which is, after all, what the telphone is designed to best carry). MOdulating a signal from binary to analog is the 'MO' in MOdem. DEModulating an analog signal back into binary is the reverse and is the 'DEM" in moDEM. The simplest modems modulate the ones and zeros of the data into two different tones. ~Data + Telephone => Errors :` If we want the transmission to be highly reliable then we must do more than simply send the binary information (modulated). We have all heard 'noise' on a telephone line and without doing more than demodulating into bits, the receiver will no doubt have a virtually impossible time of being able to tell what sounds are bits or just plain noise. In some applications, we don't really care all that much. Examples include the transmission of plain text files. Recall that all that was necessary to send any letter, many special symbols and any digit was a capability that required no more than 128 different combinations of bits. 7 bits are sufficient to represent 128 permutations of those bits. That is, if a byte were only 7 bits long then it could contain as many as 128 different sets of bits being on or off). However, a byte is 8 bits long by definition. So, when transmitting ASCII we can use the first 7 of those bits to represent data and the 8th bit to represent some form of insurance or integrity check that the first 7 were received as they were sent. This is called using 'PARITY'. You can establish a convention between the sender and the receiver that every byte WILL have an even number of bits set to 1. You do so by setting the 8th bit to a 1 if an odd number of those 7 bits are 1s, or you leave the 8th bit as a 0 if there were an even number of bits set already. (An alternative convention is to use the eigth bit to make up an odd number of bits set to 1). If the receiving end ever gets a byte that has 'odd' parity then it knows that it received the byte in error (some bit or bits were either added or lost). That is all there is to parity checking in an ASCII transmission. Not at all very good, but sufficient for most text. In fact most systems don't bother to use the parity bit at all, and just set it to 0 (most common) or 1 all the time. ~Non text data :` Program files or data files or even text files that have been compressed (ARChived) in some way use all 8 bits in every byte to represent information. So, we have lost the ability to use parity as an integrity check vehicle. Instead, in every protocol other than ASCII, we add either one or two full bytes to the end of a 'block' of bytes. The block is a fixed length (usually 128 bytes). The purpose of those one or two bytes is to contain what is called a CheckSum character or Cyclic Redundance Check (CRC) word. Like parity, the CRC is constructed at the sending end to create a pattern of bits that demonstrates that the preceeding entire block of bytes has been received with integrity. The Receiving end dynamically creates its own CRC from the information received and compares it to the byte or bytes received at the end of a block. If it doesn't match then the block must be rebroadcast (requested by sending to the sender a signal that says: "Negative AcKnowledge" - NAK. If it was ok then it sends an ACK - meaning "ACKnowledge", and the next block is sent. ~Data transfer rate :` Now, let's go back to the idea of baud. At 1200 baud, the modems are able to send and receive 1,200 bits per second. How many bits per byte? Yes, 8, but not on a telephone line if you are using modems! Instead, we bracket a byte by sending what is called a start bit before the 8 bits of data and ending with what we call a stop bit (sometimes two stop bits - at 300 baud). So, every byte requires 10 bits, not 8. Thus, at 1200 baud your maximum possible data transfer rate is 120 characters (bytes) per second! OK. Now we know what we have to send and how many bits are required and that there is something called a response from the receiver called either an ACK or NAK. So why don't we get 120 bytes per second transfers using 1200 baud modems? Well, we already saw that for every 128 bytes of data, in most protocols, we send an additional one or two bytes of CRC. We DO NOT count the CRC byte(s) as data! Yet it takes time to transmit. Also, it takes time for most protocols to 'turnaround' and react to the ACK or NAK. For example, assuming all is well, the sender has a few hundred blocks to upload to the receiver. After the first block is sent he, by convention, must wait for the receiver to analyse the CRC and decide if it is going to respond with the ACK or a NAK. Then it takes a moment to send that to the sender who, in turn, has to receive it, verify that it got here properly (was not just noise) and decide whether to send the next block or to resend to last one that was improperly received by the receiver. That takes time, on a long distance call, or via a packet switching network, or to a time-shared system it may take seconds before the reply comes back. We can easily spend half our time waiting for those acknowledgements. All time used as described above is called 'overhead'. Overhead does not include the transmission of DATA, only control bits and time. Thus, it is impossible to get to an effective DATA transmission rate of even 118 characters per second let alone 120 (CRC, etc). That is the way XMODEM transfers data, it's an old simple system but easy enough to implement on almost all machines. ~Improved transfer rate :` But, we know that the telephone is capable of carrying sound in both directions simultaneously. So, why should the sender have to wait for the receivers ACK or NAK? (This mode of operation is often called 1/2 duplex, by the way). The only reason is that it is simpler to check that the last block was received before sending the next, otherwise you have to work out how far to back up if the other end reports an error. Newer protocols do not wait. They assume that a transmission will be successful and will result in getting an ACK. So they go immediately to the task of sending the next block. Always listening, of course, for that ACK or NAK. When an ACK is received all is well and we have gained performance. If it gets a NAK, the software has to decide which block or blocks have to be rebroadcast. Because we are sending flat out, it may not be the last block we sent that was in error, it could be a couple earlier, depending on how long it takes for a block to reach the other end. So the ACK or NAK is sent together with a byte called the packet number (0 to 255), and possibly more information. If an ACK is received the recipient knows which of a series of blocks (packets) it is referring to. Similarly it would know with an NAK. Yep, more bits and more overhead! ~Transfer Protocols :` Almost nobody uses ASCII transfers any more. Why should they when they are so poorly controlled and when you realize that ONLY un-compressed raw text can be sent with it? Still, a great many first time communications users try to send programs of other files this way. And, while the transmissions will appear to work, the resulting files will be garbage, of course. Only 7 of the 8 bits are being transmitted in each byte! Many communications programs will allow you to use ASCII even when they should know that the result will be unsatisfactory. All programs (including BASIC), pictures. digitised sound, archives (even if they're archives of text files) and many data files use all 8 bits of every byte, and aren't ASCII (text) files. They can not therefore, be transmitted successfully via an ASCII or text file send command. Some communications programs will not let you even try, but most will. What are the protocols that currently exist in wide spread usage across the country? The most frequently seen is called XMODEM. This protocol is quite reliable (about 96%) and uses blocks of 128 bytes plus one CheckSum character at the end of every block. It is because it uses only one CheckSum character that the reliability is only 96%. Another is called XMODEM/CRC. This is exactly the same as XMODEM but it uses two CRC characters. The result is that the effective performance is reduced insignificantly (1/130th), but the reliability is increased to about 99.6%. In any case where you have a choice between the two you would, of course, opt for XMODEM/CRC. Then there is what is called XMODEM-1k. This protocol differs from the earlier ones in that it sends 8 - 128 byte blocks together as a 'super block' before it sends the two byte CRC word. As a result it is the fastest protocol that I have ever seen for micro computers that use 'dumb' modems (ie, non self correcting ones). The advantage over XMODEM is that it only has to wait eighth as often for an ACK reply and therefore is faster. The disadvantage is that on noisy lines, it has to retransmitt eight times as much each time there is an error. Then, and this is particulary true in environments where one of the computers being involved is either a mini or a mainframe, there is a protocol which is called Kermit. I believe it uses 128 byte blocks and other overhead such as a 'header block - block zero' that provides control information. It is also very reliable (99.6% I believe) but it is SLOW!!! It was designed to work on systems that were only capable of communicating 7 bit data, which is fine for ASCII, but with other types of data you must translate those 8 bit bytes of data into 7 bits thus slowing the system down. There is now an 8 bit version of Kermit. An improved system is called WXMODEM which stands for 'Windowing XMODEM'. This uses 128 byte blocks but it does not wait between blocks for a response. It is always listening for those ACKs, and NAKs of course. Extremely high performance is the result, relative to XMODEM or the other 1/2 duplex protocols. WXMODEM tries to stay 4 blocks ahead of the receiver at all times while the receiver tries to get 'caught up'. The difference between the block being sent and the most currently received ACK or NAK is called the window (a number between 1 and 4). Then there are two more odd protocols that have become relatively visible of late. These are called ZMODEM and YMODEM. YMODEM sends information about the size, and date of creation of the file, and allows you to specify a set of file names (a 'batch' of them). A number of programs are mistakenly called YMODEM when they infact only support a subset of YMODEMS features - XMODEM-1k is sometimes mistakenly called YMODEM. ZMODEM was designed for use in a pacnet type environment. Like WXMODEM, ZMODEM does not wait for ACKs and NAKs. Unlike WXMODEM, ZMODEM doesn't have a fixed 'window' size, ie it doesn't worry how far ahead of the reciever it has got. ZMODEM can drop back to YMODEM protocol if the other end cannot handle it, and will also drop back to 128 byte blocks if the line is too noisy. ~Communicating file name :` What must a user know to do a file transfer? What protocol is available on BOTH ends of the transmission, What the name of the file is at the sending end, What the file is to be called at the receiving end. That is, if the receiving end of a transmission already has a file with the same name as the file to be transmitted, you will need to call the new file something else, also the two systems involved may have different file naming conventions - the file being sent may have a name containing 'illegal' characters or may be too long for the receiving system. Thus, most communications programs allow the specification of the file name on your end and then the name on the other end. Protocols that allow 'batch' file transmision often assume the filename is the same at both ends, or make up a new name if the filename is invalid. Well it seems to stop a bit short but hopefully that'll at least get you started!