Content Delivery Networks Fundamentals, Design, and Evolution

The definitive guide to developing robust content delivery networks

This book examines the real-world engineering challenges of developing robust content delivery networks (CDNs) and provides the tools required to overcome those challenges and to ensure high-quality content delivery that fully satisfies operators’ and consumers' commercial objectives. It is informed by the author’s two decades of experience building and delivering large, mission-critical live video, webcasts, and radio streaming, online and over private IP networks.

Following an overview of the field, the book cuts to the chase with in-depth discussions—laced with good-natured humor—of a wide range of design considerations for different network topologies. It begins with a description of the author's own requirement filtration processes. From there it moves on to initial sketches, through considerations of stakeholder roles and responsibilities, to the complex challenges of managing change in established teams. Agile versus waterfall considerations within large blue chip companies, security, commercial models, and value chain alignment are explored in detail. Featured throughout the book are numerous "what if" scenarios that help provide a clear picture of the wide spectrum of practical contexts for which readers may be tasked with building and implementing a CDN. In addition, the book:

• Discusses delivery of live, catch-up, scheduled on-demand, TVOD and SVOD
• Offers insights into the decisions that can to be made when architecting a content distribution system over IP-based networks
• Covers CDN topologies, including Edge-Caching, Streaming-Splitting, Pure-Play, Operator, Satellite, and Hybrid
• Examines computer hosting and orchestration for dedicated appliances and virtualization
• Includes real-world cases covering everything from IETF, regulatory considerations, and policy formation, to coding, hardware vendors, and network operators
• Considers the future of CDN technologies and the market forces driving its evolution

Written by a back-room engineer for back-room engineers, Content Delivery Networks gets readers up to speed on the real-world challenges they can face as well as tried-and-true strategies for addressing those challenges in order to ensure the delivery of the high-quality content delivery networks that clients demand and users expect. 

Dom Robinson is Co-Founder and Director of Innovations at id3as, London, UK. With more than two decades in the field, Robinson is one of the pioneers in online video, streaming media, and content delivery networks. His company, id3as, has been at the forefront of cloud delivery of live video at-scale, and is responsible for the software that powers a wide range of high profile online publishers, including NASDAQ, Perform, Intelsat, Arqiva and numerous others. He writes for StreamingMedia.com, is a Visiting Lecturer at Sussex University, Chairs a number of CDN and related Conferences, and frequently is quoted by industry magazines as a CDN and technical networks expert.

Frontispiece xiii

Topics Include xiii

About the Book xiv

Synposis xiv

Unique Perspective xv

Market Need xvi

Audience xvii

1 Welcome 1

1.1 A Few Words of Introduction 1

1.2 The “Why” of this Book 2

1.3 Relevant Milestones of the Personal Voyage 3

2 Context and Orientation 9

2.1 History of Streaming 10

2.1.1 Foundations – What does “Streaming” Really Mean? 12

2.1.2 Streaming 13

2.1.3 Related Network Models 16

2.1.4 Physical Network Considerations 16

2.1.5 Internet Layer Considerations 17

2.1.6 Transport Layer Considerations 17

2.1.7 Applications – Transport Protocols 18

2.1.8 Protocol Evolution 19

2.1.9 Format Evolution 25

2.2 Industry Evolution 25

2.2.1 “Stack Creep” 26

2.2.2 Real World – Blue Chips and Video Delivery Networks 26

2.3 Consumer Adoption 29

2.3.1 The Audience 29

2.3.2 Traditional Ratings Companies and Audience Measurement 32

2.3.3 Streaming Media and Measurement 34

2.3.4 Predictions of Others 37

2.3.5 The Pending Collapse of the Value of Broadcasting to Advertisers 41

2.3.6 “Device Effect” and Formats 41

2.3.7 Video Formats (in Particular, Multicast and UDP) and Network Architecture 43

2.3.8 Discovery, Curation, and Social Media 45

2.4 Encode > Serve > Play 54

2.4.1 The Basic Building Blocks 54

2.4.2 The Acacia Patent 55

2.4.3 Akamai vs. Limelight 57

2.4.4 Standards, Standards, Standards, … 58

2.4.5 D�]Book Connected TV Standards from the Digital Television Group 60

2.4.6 The CoDec Concerns 61

2.5 What is a CDN: A Simple Model 63

2.5.1 Setting the Scene for CDNs 63

2.5.2 CDNs as Money Savers 66

2.5.3 Request Routing 67

2.5.4 CDN Brokerage 69

2.5.5 SaaS Models within the CDN Ecosystems 70

2.6 Cloud Inside – New Generation 75

2.7 The Three Generations of CDN 76

2.8 Software Definition 82

2.8.1 Multicore CPU and Functional Programming 86

2.8.2 Functional Programming and Containers 86

2.9 “Service Velocity” and the Operator 87

3 Workflows 89

3.1 Live Event Focus 92

3.1.1 Approaches to Webcasting 93

3.1.2 Think Before You Start – Your Client Probably Hasn’t! 94

3.1.3 Budgets 95

3.1.4 Objectives – Quality vs. Reliability 97

3.1.5 Production Principles 98

3.2 Backhaul/Contribution and Acquisition 102

3.2.1 Broadcast 104

3.2.2 Wire 104

3.2.3 Wireless 107

3.2.4 Satellite 108

3.2.5 3g/4G CellMux 109

3.2.6 Reliable UDP and HTTP/UDP Solutions 111

3.2.7 Throughput vs. Goodput 112

3.3 Cloud Saas 113

3.3.1 In Workflow “Treatment” (Transcode/Transmux, etc.) 114

3.3.2 DVR Workflows 117

3.3.3 Catch�]up Workflows 119

3.3.4 VOD Workflows 121

4 Publishing 125

4.1 Publishers, OVPs, CDNs, and MCNs 126

4.2 Small Objects, Large Objects, or Continuous Streams 129

4.2.1 Compression 132

4.2.2 The “Quality Question” … 134

4.2.3 Latency 136

4.2.4 Application, Site, Web, and Games Acceleration 137

4.3 Desktop and Device Delivery Applications 138

4.3.1 Standalone Media Players and Applications 138

4.3.2 Video Tags in HTML5 141

4.3.3 WebRTC – Beyond HTML5 142

4.4 Request Routing (The Dark Art of the CDN) 142

4.5 Logging Analytics and the Devil in the Detail 143

5 Service Velocity 145

6 Charging for IP�]Delivered Content 151

6.1 Lessons from the Music Industry 151

6.2 Success Cases 153

6.2.1 YouTube 154

6.2.2 Netflix 155

6.2.3 On the Horizon 156

6.3 Failure Cases 158

6.3.1 Scour.net 158

6.3.2 mp3.com 159

6.3.3 Napster 160

6.3.4 Broadcast.com 160

6.3.5 The “Yacht Projects” 162

6.4 General Commentary on Commercial Models 163

6.4.1 Cable TV 164

6.4.2 IPTV 165

6.4.3 OTT Pureplay + Operator CDN 166

6.4.4 Fog Distribution 167

6.4.5 Variation from Live Linear to VOD, and Everything in Between 168

6.4.6 DRM 169

6.4.7 Watermarking 171

7 Competition and the Regulatory Environment 175

7.1 ISOC, ITU, and WSIS 176

7.2 Policy – Net Neutrality 179

7.3 Value Chain Alignment with QoS and SLA Propositions 181

7.4 Layer�]

2 Workaround? 181

8 Cultural Change 183

8.1 Traditional Broadcasters 183

8.2 The Millenial Subscriber 185

8.3 ISP and Content Providers 186

8.4 Telco and Telecoms 188

8.5 Content Providers 188

9 Preparing for Change in Your Design 191

9.1 Preface and Philosophy 191

9.2 Models, Diagrams, and Schematics 193

9.3 How to do a Good Diagram? 193

9.4 Scenario Planning 194

9.5 Risk, Responsibility, and Reassurance 196

9.6 Optimization and Upsell 196

9.7 Value Creation/Agility 197

9.8 Expectation Management 197

10 Multicast – the Sleeping Giant 199

10.1 Multicast Recap 199

10.1.1 Basics 199

10.1.2 Routing Protocols 200

10.1.3 Flood, Prune, Storms, and a Bad Taste 201

10.1.4 Commercial Outcome 201

10.2 What Happens Now? 202

10.3 To Singularity and Beyond 204

11 Deep�]Dives (Case Studies) 207

11.1 Hitting the TV Screen – IPTV/Hybrid TV and OTT 207

11.1.1 The Taxonomy of OTT Video 210

11.1.2 Arqiva Connect and Freeview Plus 214

11.2 Creating Nasdaq’s Cloud�]Based Virtual Workflow 217

11.2.1 The Genesis of a Virtual Workflow 218

11.2.2 The Technology Behind the Workflow 219

11.2.3 Why Amazon EC2? 220

11.2.4 What Sort of Scaling Issues did You Face? 222

11.2.5 How about SLA? 222

11.2.6 What about Signal Acquisition? 222

11.2.7 What about OS Choices and Stacks? 223

11.2.8 How Is the System Controlled? 223

11.2.9 How Does it Report? 224

12 Wrap Up 225

Index 229

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.