from plesiochronous to synchronous

Up until the 1960s, telephone calls were carried as analog signals. There was a great deal of frequency division multiplexing onto open wire, and sometimes coaxial cables, but in the post-war period buried multipair telephone cables began to dominate. While the network was surprisingly functional—long distance calls from coast to coast were still quite intelligible—there were fundamental limitations to analog carriers. The quality of the call would degrade with each step, a problem that was much more acute for the sensitive digital connections that were already the apparent future of the telephone system. These were digital signals modulated over the top of telephone lines: the telephone system itself still handled them as analog calls, just ones that required special care for quality.

During the Second World War, it became apparent that many of the long distance network's challenges could be greatly simplified by using digital signaling as the basis of the entire long distance system. In 1943, Bell Labs contributed to the design of an early encrypted voice communications system called SIGSALY. Unlike the "voice scramblers" and other analog techniques of the early phase of the war, SIGSALY was recognizably similar to modern encryption. The voice data was quantized into numerical values, so that digital arithmetic could be used to encode it. In the quest to build a cryptosystem for voice, Bell Labs incidentally built the first applied system for PCM digital voice transmission. After the war, it was an obvious next step to refine this technology and apply it to regular service.

This didn't necessarily mean direct digital service, at least not at first: most subscriber loops remain analog to the present day. For the first years of the digital network, even the exchanges themselves switched purely analog signals. Between offices, though, encoding the analog channels into digital ones solved problems from audio quality, to microwave path overloading, to flexibility in traffic routing. Digitization became a core part of AT&T's concept of a carrier, just like frequency division multiplexing had in the 1920s. That's the background to the 1962 introduction of T-carrier, the first digital carrier in the commercial telephone system.

The Digital Hierarchy

Back in the article on T-carrier (which you should read if you are at all interested in this topic!), I explained that T1, T2, T3, etc. are hierarchically larger-bandwidth connections that carry the DS1, DS2, DS3, etc. digital signals respectively. Let's quickly review this hierarchical concept and, just to keep things interesting, add the further confusion of the D1 channel bank. A D1 channel bank is the physical piece of equipment that digitizes 24 voice channels and multiplexes them into a DS1 signal, which will be carried over T1 carrier.

Similarly, four DS1 signals can be combined into a DS2 signal and carried over T2. Seven DS2 signals can be combined into a single DS3 signal and carried over T3. And so on, at least as far as they were able to get with the technology of the time. The DS2 level maps generally to the D2 channel bank, but that relationship can be confusing as the subsequently developed channel banks added features as well as capacity. The D2 channel bank could multiplex 96 channels onto a DS2 signal, but it also increased the overall quality of the audio channel to meet the technical requirements for general mixed toll service (the D1 had originally only been approved for conventional voice channels, not other applications like data-grade leased lines).

In order to make that improvement, the D2 had to reduce the amount of quantization noise introduced into the signal. The D1 channel bank had used PCM audio with seven bits per sample, and calculations showed that eight bits per sample would be required for a toll-grade connection . In a way, that looked easy: DS1 already used 8-bit units. But the eighth bit, not used as part of the PCM sample, was reserved for signaling purposes. No bother: the D1 channel bank hadn't really used the full signaling bandwidth anyway, so it could comfortably be reduced to meet a compromise position. The D2 channel bank used 8 bits for most PCM samples, but every six frames the last bit of each sample was reserved for signaling use rather than audio. This created a slight increase in quantization noise, and a significant decrease in signaling rate, but the resulting numbers for both fell within acceptable margins. A perfect compromise, but one that feels technically inelegant: in a sense, the D2 channel bank used 7.833 bits per sample. Ouch.

This must have given the Bell System's engineers similar discomfort, but the aesthetic offense was worth it for backwards-compatibility. By the time the D2 channel bank was fielded in 1970, there were a huge number of D1 channel banks and T1 connections in service. Backwards-compatibility to existing T1 lines was a firm requirement. The D2 multiplexed 96 channels, four times a D1, and could carry those channels over four T1 links—in fact, initially, this was only the option. T2 and subsequent carriers were not fielded until after the D2. This backwards compatibility provided an upgrade path, as well as a way to subdivide the 96-channel capacity.

You might realize that it is also a completely alternate explanation of the "hierarchy."

You might wonder why the multipliers at the levels of the hierarchy are so inconsistent—24, 4, 7, 6. Reading the thing about signaling bits in every six samples above might have stirred deep memories of a concept called "robbed-bit signaling," a general concept of stealing occasional bits for other purposes that was widely used with T1. These are all smells, indications that a system was not architected, but grown. The "digital hierarchy" of T1, T2, T3, etc. is not some elegantly-designed system, at least not nearly as much as T4-era Bell Laboratories Record articles like to make it sound. It's mostly a result of happenstance, as the history of the Dn channel banks shows: each new channel bank, the D3, the D4, was designed to carry many more channels than those before. They were also required to be backwards-compatible with the use of the older channel banks and carriers. This process of exponential improvement with backwards compatibility led to the digital hierarchy as a natural consequence, not as a designed outcome.

Plesiochronicity

This emergent concept of layers of successively larger muxes explains the digital hierarchy, but we now need to consider the problem of timing and synchronization.

In any system based on digital sampling, you have to be aware at the point at which those samples are taken in order to combine multiple signals. Consider the scenario where several simultaneous phone calls are being combined into a single digital mux for transmission. When the phone calls reach a D1 (or D2, D3, etc) channel bank, it samples all of the inputs at an appropriate time to build its DS1 (or higher) output stream. Nothing about this case is particularly complicated, except that it does require that all of the calls involved in the digital stream are sampled under the control of the same clock.

But what about cases where lines need to be digitized in different locations? Say you have two smaller offices in a city, that need to deliver inter-office traffic to a tandem in the next city over. An obvious approach would be to digitize the calls in each of their local offices, send them as digital signals to one of the offices (perhaps the city's main downtown exchange), and then combine all of those digital signals together to be sent to the tandem switch. Doing this with channel banks wasn't very desirable: you would actually need to convert the calls back to analog, and then back to digital, in order to relay them from the first digital leg to the second. The first digital telephone leads were like this: completely point-to-point, without the possibility of switching or routing in the digital realm. That wasn't so big of a limitation early on when all of the switches were analog and digital leads just made the lines between them denser, but it prevented the construction of the fully digital network that Bell Labs envisioned.

What was clearly needed was some sort of mux that could take four DS1s, already in that form, and combine them into a DS2. Repeat this for the higher levels as well, and you have the basis of a system that can combine and separate traffic as needed to meet the level of demand between various offices. But what about the sample timing?

If you have four DS1 streams from four different places, at least back in the 1970s, they are not going to be synchronized. The four D1 channel banks would each be operating on their own sample clocks, and because of the imperfection of the physical world some of these clocks would run slightly faster or slower than others. When our four DS1 streams arrived at our notional multiplexer, the four streams would be not only out of phase, they would actually run at slightly different speeds.

We can imagine a lot of modern solutions to this problem, probably by buffering relatively large blocks of samples in memory and then transmitting out of that buffer. In the '70s, things weren't so simple: first, memory was expensive and slow compared to the bitrates involved, and second, the latency involved in such extensive buffering was not considered acceptable in a network that had previously been all-analog and thus had no more latency than that implied by the speed of light.

The solution to this problem, the Plesiochronous Digital Hierarchy or PDH, is not easy to understand from our contemporary perspective. Many textbooks on networking and communications make some nod to PDH, but it's often short and cursory, and leaves you with a nagging feeling that the whole thing doesn't make much sense. PDH is hard to relate to for two reasons: first, it was developed to solve a set of problems that (due to broader technical advancements) we no longer have. Second, even within those constraints, it is a solution that has been obsoleted by newer approaches. PDH is not something that anyone would design today, and in retrospect, it's probably best to think of it as a hack. PDH was an interim measure, if you will, that bridged a gap between the development of digital channel banks and the later development of a good approach to synchronizing a digital network.

It would have been more convenient if those two systems had been designed at the same time, but in practice, they weren't: digital long distance lines were in common use a solid decade before AT&T had a reasonable architecture for a digital network. You can already see that they were thinking through these requirements in the late 1960s, leading to the concept of the PDH as a hierarchical network architecture, but most of the actual technology to provide that network was built later, in the '70s and '80s. For a while, there was just, you know, something that worked. A duct tape fix. Conceptual bailing wire.

So what even was the PDH? It is helpful to start by defining the word "plesiochronous," which every textbook has to do since the term was invented by Bell Labs to describe their design. I will spare the roots, but plesiochronous means "almost synchronized" or "approximately synchronized." And that's what the PDH was: it was a synchronous digital system with some slop. There was a little wiggle room.

A DS1 signal is 1.544 Mbps, so if you multiplex four of them, you might expect 6.176 Mbps. You would be wrong: a DS2 is actually 6.312 Mbps. The overhead here comes from the plesio part of plesiochronous. To understand how that works, we should look at the design of the multiplexer from the beginning. AT&T practice would call a multiplexer from the DS1 level to the DS2 level an M12, for, well, multiplexer from 1 to 2. In practice M13s seem to have been more common equipment, which were often implemented internally as seven M12s feeding one M23.

In any case, the M12 takes in four different DS1 streams and then bit-interleaves them, such that every 4th bit of the output is the next bit from one of the inputs. This is simple to reason about and has the advantage that virtually no buffering ever happens. But what if one of the inputs is running slightly fast, and another slightly slow? The first thing to know is that the output stream is intentionally slightly faster than all of the inputs. A typical PDH system allowed a 50 PPM plus or minus variation in the rate of the inputs, which means that the output had to be at least 50 PPM fast relative to the sum of the inputs. That way, the multiplexer will never "fall behind" because an input is providing bits faster than the output can interleave them.

The other thing to know is that the multiplexer must perform a function known as either stuffing or justification. It is said that the term justification was introduced within AT&T because it sounded more technical, while stuffing was more popular with the actual engineers. Certain parts of each outgoing frame are defined (by standard) as stuffable, and they're sometimes called gaps. These are sections where the multiplexer is allowed to either include real data, or to include some nonsense data called stuffing bits (or, if you want to sound more technical, justification bits). By choosing to either include real data or stuffing in the gaps, the multiplexer can actually adjust the "real" output data rate up or down. That way, it converges onto the correct average (the sum of the inputs) despite the "timing noise" of the inputs varying slightly up and down in real speed.

If that didn't make sense to you, here's a different way to think about the same thing: we can imagine the DS2 output frame as consisting of four "slots" into which the four DS1 input frames are inserted (this isn't quite intuitive because they are interleaved bit-by-bit, but the math works out the same). Under the PDH design, these slots are all slightly wider than the input frames, so that there is "free space" for the input frames to be shifted a bit in either direction or stretched out. That allows the multiplexer to "catch up" on a fast input or to "draw out" the frames from a slow input. Thinking about it in terms of a wide slot that you can place the input into as desired also gives more value to the term "justification:" you can shift bits around, or add extra space in between them, just like you would to justify text.

The actual implementation of this idea is pretty complicated, because the demultiplexer on the other end needs to know which bits were stuffed so that they can be discarded. That's why PDH specified gaps at fixed positions: the control signaling included a set of flags that indicated whether the fixed gaps were used for data or stuffing. When the multiplexer decided whether or not to use a stuffable bit (based on whether or not the input was getting ahead of the output mux), it also had to store that decision for the end of the frame so that the flag could be set correctly.

PDH wasn't an elegant solution, but it was a solution, and multiplexers were designed up to the DS4 level based on the PDH approach. We all owe PDH a huge thanks for providing the clunky stopgap that allowed the construction of a nationwide digital telephone network. That said, well, it had quite a few downsides.

First, PDH was never all that well standardized. Researching PDH is tricky because there aren't a lot of documents that provide conclusive detail about the actual implementation in the US, mostly just summaries, conceptual sketches, and examples from Europe. As far as I can tell , Western Electric never really offered a standalone PDH multiplexer. Larger integrated systems from Western Electric (like, say, a microwave carrier equipment bay) might include a PDH multiplexer, but it's not clear to me if WE even made them—the advertisements and pricing information I can find for PDH multiplexers are from other vendors like Farinon and NEC.

In North American practice, it doesn't seem like PDH was ever robustly standardized, and there were implementation differences between equipment from different manufacturers. That's probably one of the reasons for the scarcity of information on PDH equipment: you wouldn't usually buy a single PDH multiplexer, but instead would specify a specific model as part of a larger system. That meant that the PDH network was far less flexible than AT&T had hoped.

The problem was even bigger when you looked overseas: AT&T tended to field the best thing that Bell Labs and Western Electric could come up with, and then promulgate that as a standard later on. The European telcos tended not to adopt new American developments until they were mature, giving them the opportunity to make standards before any equipment was in place. Thus the European versions of US telephone carriers are often a bit better, and PDH is no exception. In fact, the US, Europe, and Japan used three different hierarchies for PDH signaling. The European standard put more channels into a stream (the "E1" at about 2 Mbps), and used a consistent multiplier of 4 for each level. European PDH practices even got a complete published standard from CCITT. The Japanese system used the same DS1 standard as the US but with different multipliers (4, 5, 3, 4 instead of 4, 7, 6). PDH could thus not be used for international calling without completely remuxing everything.

Another shortcoming of PDH had to do with that remuxing. The decision to bit-interleave the input frames together, stuff some bits, and then put flags on the bit stuffing into the interleaved stream, meant that a PDH multiplexer effectively "scrambled" the input streams. If you had, say, a DS3 stream and wanted to extract a single voice channel from it, you had a lot of work in front of you. It simply wasn't feasible with the electronics of the era to analyze the stream to determine the interleaving and stuffing bits at each level. Instead, you would have to put the DS3 through demuxers all the way back down to the bottom of the hierarchy, pull out the channel you wanted, and then remux it all the way back up. Pulling a single channel out of a DS3 could require two 11.5 foot tall equipment bays, and even further quantization noise due to the nature of the robbed bit scheme.

Two things happened that pushed PDH to its limit: first, fiber optics. AT&T was famously a late adopter of fiber, and 1970s documents on the digital hierarchy talked about conveying it over T-carrier, microwave, and coaxial. We have talked about coaxial before, but L-carrier was an analog system. Digital coaxial did emerge as the T3, T4, and T5 carriers, but these were relatively short range and mostly only saw use on shorter inter-office links.

Things were changing, though: by the time DS3 equipment was appearing in telephone exchanges, competitors were already evaluating fiber optics. AT&T dipped their toe in the water in 1976, with an experimental DS3 fiber optic link in Atlanta. The next year, both AT&T and competitor GTE had DS3 fiber lines in service (in Chicago and the LA area, respectively). Outside manufacturers started offering T1-over-fiber equipment around 1980, and in 1983 AT&T had a multimode fiber system stretching from New York to Washington. Sparing the (fascinating) story of the business and competitive factors, the writing was on the wall: fiber was the future of long lines. AT&T engineers realized that the PDH design did not scale well to the enormous capacities that fiber could reach, and even worse, that complex fiber networks would require exactly the capability that PDH lacked: extracting single channels from the mux. Besides, despite the systems in use, there really wasn't any standard for conveying PDH signals over fiber, which only meant more compatibility issues.

Something else was coming over the horizon in 1983: divestiture. The complete reorganization of AT&T in the mid-1980s, breaking the Baby Bells up into independent companies, made the interoperability problems with PDH much more severe. Bell operating companies (BOCs) would need to easily exchange traffic with both AT&T Long Lines and, eventually, with other BOCs and competitive phone companies and long distance carriers. That was very difficult when that traffic was mostly conveyed in loosely-standardized black-box muxes. Many aspects of US telephony were loosely standardized prior to the Bell System breakup because AT&T ran the entire network and didn't really need to cooperate with others. After 1984, the US became a lot more like Europe: for technology to be widely adopted, it would need to have a well-defined, published standard.

That meant a standards body—the part of AT&T that had formerly published the practices used by all of the BOCs, after divestiture, became an independent company called Bellcore. Bellcore had an immediate goal of developing a fully standardized set of digital interfaces to be used between telephone companies. At the same time, a group of carriers and equipment manufacturers organized an association to start an ANSI standards process for fiber-optic equipment. They had a goal of an industry standard for high-speed digital fiber optics, so that terminal equipment from different manufacturers could be used together.

That effort formed the ANSI T1X1 working group, titled Digital Hierarchy and Synchronization. In 1985, Bellcore proposed their draft standard to the T1X1 committee: they called it SONET.

Synchronicity

First, I will briefly explain some context for my global readers. One of the shortcomings of PDH was its lack of international standardization, and SONET would not make the same mistake. Shortly after SONET was proposed to ANSI, it was also proposed to CCITT. For historic reasons, and despite the fact that they are largely identical, SONET (Synchronous Optical Network) is a North American standard while the European standard is called SDH, for Synchronous Digital Hierarchy. I will stick to the term SONET because it is catchier, but the name SDH better conveys that SONET is essentially the same idea as PDH but it's actually synchronous. Beyond that, it is also more feature-rich and much better standardized.

SONET starts out higher up in the digital hierarchy. The "lowest" SONET level is Synchronous Transport Signal 1, or STS-1, which is typically transported by Optical Carrier 1, or OC-1. OC-1 operates at 51.84 Mbps, sufficient to carry a PDH DS3 channel with signaling overhead. By this point in telecommunications history, channel banks had improved and 50 Mbps wasn't what it once was, so DS3 was a good choice for the basic unit of a high-capacity transport system. This also provided critical backwards compatibility: DS3 was the most widely deployed PDH signal, so by accommodating a DS3 within an STS-1, SONET provided an easy route to tunneling legacy PDH links through a SONET replacement.

I put a lot of time into writing this, and I hope that you enjoy reading it. If you can spare a few dollars, consider supporting me on ko-fi. You'll receive an occasional extra, subscribers-only post, and defray the costs of providing artisanal, hand-built world wide web directly from Albuquerque, New Mexico.

By this time, the Digital Hierarchy principle from PDH was well established, so SONET follows the exact same pattern—but, fortunately, the naming convention improved. OC-3 is three times the bandwidth of OC-1, OC-12 is twelve times the bandwidth of OC-1, and so on, with OC-24, OC-48, OC-192, and even OC-768 in common use, the latter achieving over 38 Gbps of payload.

SONET is complex compared to what you might expect, or at least different. A SONET frame does not consist of a header and a body. Instead, like with PDH, all of the data is interleaved together in a way that feels messy. In an STS-1 signal, for example, a frame consists of three bytes of protocol overhead and 87 bytes of payload, making up a 90 byte pattern that is repeated nine times to make up an 810 byte frame. Higher levels use larger frames but keep the same structure, effectively mixing the header into each frame. Like PDH's bitwise interleaving, this is done to manage latency: interleaving the header with the payload avoids the need to buffer an entire frame of data at any point. Because of this design decision, SONET can provide a latency that is less than the frame time.

This description also oversimplifies by discussing only one part of SONET's protocol overhead. There are actually several layers of overhead, all interleaved together, that serve different purposes. The "lowest layer" overhead is recreated at each regenerator along the route (devices that decode and then reencode the optical signaling to compensate for path loss), while higher layers are only processed by terminal equipment and are used for purposes like connection quality monitoring.

These types of overhead are important because, unlike PDH, SONET provides a specification for the entire physical infrastructure. SONET is virtually always used with one of a few standard architectures, each of which are designed to provide redundancy. SONET uses unidirectional signaling on the fiber, so any SONET link will have two fibers normally active, one for each direction. Where things vary are the approach to redundancy: in a linear route with automatic protection, there are two "spare" fibers. When terminal equipment detects (via monitoring protocol overhead signaling) that the connection quality is no longer good on the primary pair of fibers, it will switch automatically to the secondary pair.

Particularly common with SONET though, and particularly distinctive, are rings. A unidirectional ring uses two fibers, one for each direction, but each of those fibers goes in a circle and returns to the origin point. At the origin point, each frame is transmitted both directions into the ring. The terminals elsewhere on the ring compare the two copies that they received, one from each direction, and use the better of the two. This provides protection against damage to one of the fibers: terminals that are "downstream" from a broken fiber should still be able to receive the frames that went the other way around.

A downside to this protection method is that it is inefficient: sending every frame twice means that two fibers are still providing a theoretical maximum bandwidth equal to the capacity of a single fiber. Bidirectional rings use a more complicated monitoring scheme to detect when frames are lost, so that they do not need to preemptively send them twice. A broken fiber will be detected and, depending on the ring, either sent the other way around or sent on a second redundant fiber allocated to the same direction.

SONET's ring architecture is one of its most interesting properties. Unidirectional rings are often used as the "last mile" component of a SONET network. Consider, for example, a large business in the early days of the internet. If you wanted 100 Mbps internet connectivity, one of your best options was to purchase SONET service from the telephone company. They would take an existing ring, starting and ending with a telephone exchange, and figure out how to add your premises somewhere along it. The ring shape allowed redundancy with a relatively efficient use of fiber length.

Bidirectional rings, with their greater complexity but greater efficiency, were usually used between telephone exchanges where there was adequate room for more complex terminal equipment.

The ring topology also illustrates the biggest advantage of SONET over PDH. Unlike PDH, SONET hierarchy levels keep overhead and frame data in the same position, with consistent offsets to the framing data necessary to find an individual channel. That means that it is practical to inspect a high-rate SONET link, find a specific channel, and pull out those bits. You can even find where a channel is located and then overwrite just those bits, inserting data into the stream. This pattern is called "add-drop multiplexing," because the multiplexer usually adds some channels and drops others.

Let's work a concrete example. Many years ago, I was an intern at a General Electric plant in Virginia. Although the facility's main internet connectivity had since been upgraded, SONET service was still active for redundancy and for a legacy TDM PABX. In an equipment room, the telco-owned add-drop multiplexer sat in a rack with the fiber enclosures where the building was actually spliced into the ring. The multiplexer was the customer premises equipment, and our Cisco router was connected by OC fiber to the multiplexer's "low speed" side while its "high speed" side had two connections, one each way, to our "neighbors" on the ring. The telco had left a diagram taped to the rack that made this material: one pair of fibers went to the National Radio Astronomy Observatory, the other went to the National Geospatial Intelligence Agency. There must have been some interesting traffic on that ring!

The multiplexer found the time slots in the SONET ring that were assigned to us, "dropped" them and sent those bits out on the low-speed side, and then received bits on the low-speed side that were "added" back into the same time slots. Each of the organizations along the ring did the same thing with their own channels.

The fact that the SONET ring had to "pass through" each of its users was a material downside, since a failure of an add-drop multiplexer at one customer could cause service disruptions for others. That's why the multiplexer was telco property, and further why SONET rings were always deployed with some type of automatic protection and redundancy. Ultimately, though, a telephone company technician still showed up at the security desk from time to time, needing to perform maintenance or repairs on the SONET equipment.

The ability to perform add-drop multiplexing at any point on a SONET link also enabled digital cross-connects. This, finally, was the original dream of the nationwide digital network: by pairing two add-drop multiplexers together, you can switch channels from one SONET line onto another, completely digitally and with almost no latency. Paired with a management protocol called Transaction language 1 (TL1) for remote reconfiguration, digital cross connects make the long distance network vastly more flexible. Big chunks of long distance traffic could be instantly rerouted, and consolidated onto major routes or spread across more minor ones.

Now, you might wonder, what happened to the whole synchronization problem? The most obvious difference between PDH and SONET is that PDH is plesiochronous while SONET is synchronous. SONET requires that all of the terminals, at least within a given interconnected network, operate in synchronization. That's exactly the requirement that PDH rejected. What changed is the state of the art in timing and synchronization.

A few things came together to obsolete the need for plesiochronous technology. One was the introduction of the Bell System Reference Frequency: to coordinate both analog and digital systems that were timing-sensitive, AT&T introduced a nationwide time synchronization system. By the 1980s, the 2048 kHz Bell System Reference Frequency was distributed to nearly all telephone facilities over the long distance infrastructure. Various time code systems followed. During the same period, GPS was coming online, and GPS receivers became a relatively inexpensive source of precise time.

Most terminals on a SONET ring got their timing from the ring: they just followed the signal and added or dropped channels at the appropriate time. A "master" terminal, typically at the telephone exchange, had to maintain that beat. In the early days, it would likely derive its clock either from an atomic clock in the office or via a reference signal coming from another office. In the later days of SONET, the time source was more likely to be a "BITS" or Building Integrated Timing Supply. BITS are basically time servers for telephone offices, typically using a GPS receiver as their discipline source and transmitting a few different types of TDM time codes to whatever devices needed them.

Despite all of these measures, it's interesting that SONET still cheats. There is a bit of wiggle room. Part of the SONET protocol overhead is a pointer that indicates the offset within the frame that the payload frame actually starts. SONET devices can thus start the payload a little bit late in a SONET frame, providing an opportunity to fix timing errors. This capability means that SONET is still an option even in plesiochronous environments, which eased the fairly rapid transition from PDH to SONET.

SONET in Use

SONET is fast enough to survive into the modern era, although it is now decidedly legacy. 38 Gbps is nothing to sneeze at, and still compares favorably with a lot of common long distance connections. Where SONET shows its age is in the actual network payload. What do you actually put on a SONET ring?

Well, of course, you can fit a whole lot of 64 Kbps telephone channels onto any of the SONET tiers. You can do that using a "raw" T1 signal or via ISDN, which uses the same basic fiber signals but better standardizes telephone signaling (e.g. off hook status) and interoperability. But even an OC-1 has 672 channels, more phone calls than just about anyone needs to make. In the 1980s, the driver for commercial SONET service was data.

SONET itself is incredibly protocol neutral—one of the reasons for its unusual framing structure is that it is agnostic to the size, rate, and timing of the frames of the next layer protocol. SONET basically treats all connections as opaque serial streams, so conceptually, you can put just about anything over it.

Because SONET evolved during the same period as ISDN, the two are closely related. During the late '80s, as SONET was being deployed, the "Broadband ISDN" initiative created Asynchronous Transfer Mode (ATM). ATM is a virtual circuit switching protocol that multiplexes a large number of small, fixed-size packets (called cells) over high-speed connections like SONET. ATM was, at one time, a serious contender for the dominant protocol in wide area networks. That didn't come to be, though, and IP took over—but not before ATM was well-enough established to be a tempting way to carry IP. IP over ATM was fairly common, and in fact, remained the norm for DSL until fairly recently.

There were other options as well: Packet over SONET was a standard for SONET-based PPP connections, which could carry just about anything but were often used for IP. A newer offering is Ethernet over SONET, which does exactly what it sounds like.

The internet provided a compelling use-case for SONET, but it also became a challenge. SONET was designed within the telephone industry, which emphasized handling fixed-speed channels with very high reliability and very low latency. The IP-based internet embraced the opposite: opportunistically moving variable speed connections over relatively high-latency and low-reliability channels. That meant that internet service over SONET became awkward.

For one, it tended to be slow. The reason was bandwidth allocation: getting time on a SONET ring means getting channels assigned, and you will pay for those channels 24/7 regardless of whether or not you use them. A technology called virtual concatenation was developed to allow SONET's lower-level building blocks (like STS-1) to be combined in arbitrary ways to produce just about any amount of bandwidth you could want, but every Mbps over SONET was an Mbps that couldn't be used by any other customer. You paid for it. Later on, variable-bandwidth allocation systems were developed, but they were complex and missed the opportunity to make much of a difference.

SONET's fixed-bandwidth design made it excellent for real-time media, the wheelhouse of the telephone industry, but ironically real-time media was missing from the information revolution. Internet connections were generally too slow and unreliable to carry it. By the time real-time media returned, IP connections had gotten so fast that they could make up for the lack of traffic engineering with raw speed. Slow and reliable lost out to fast and unreliable almost every time.

SONET also had all the typical warts of a telephone industry network architecture. By the '90s, IP-based network equipment was both cheaper and easier to use than ATM and SONET devices. Upstart internet service providers were much more competitively motivated, driving down prices on non-SONET services.

SONET has been superseded as a standard by OTN, Optical Transport Network. OTN is an ITU standard for a digital hierarchy that starts at 2.5 Gbps and goes up, well, as high as you want. 800 Gbps "OTUC8" is now in service. Besides being faster, OTN is also designed to accommodate variable packet connections like Ethernet much more efficiently. And, of course, even OTN is losing the fight against schemes that carry Ethernet frames directly over fiber.

1. Is this the reason that 8-bit bytes became so universal in computing? I genuinely do not know, this would be a good topic for future research. A long time ago, on a different blog, I spent some time trying to establish the lineage of the 8-bit word length that is now so universal. Without devoting a career of research to the topic, I think the best I can say is that there seem to have been several parallel paths that converged on 7 and 8 bits as good choices for communications purposes. Digital telephony equipment was certainly one of them.
2. And I would very much appreciate information from anyone who knows otherwise.