White Smoke Over-The-Top

Internet behavior during special events is always of interest. Last week we had our eyes on the Sistine Chapel chimney and its manifestation across the Internet. When would the black smoke turn white?

Naturally many viewers across the globe were watching live video from the Vatican simultaneously. The smart nodes in our DiViCloud network could identify these simultaneous similar sessions.

We found out that high peaks were demonstrated in international traffic flowing to Central American and specifically to Costa Rica and El Salvador. The graphs below, taken from a specific ISP in Costa Rica, show the clear live traffic spike, comprising over 10% of the total international traffic.

Picture1

Our DiViLive value-added-service unifies multiple live-video sessions into a compacted format. By using DiViLive, ISPs can load x10 live sessions on the same bandwidth.

Are all HTTP objects created equal?

You already know that we like to take close look at our traffic to improve our large and growing network. We also share our findings in the global data flow report. This time we analyzed HTTP objects flowing through our network.

DiViNetworksHTTPObjectsTrafficBySourceThe graph shows the percentage of overall HTTP objects flowing downstream in our network. Reminder – our network delivers international data capacity from 15 major Internet exchange points to over 100 ISPs in regions where capacity is expensive.

It’s interesting to compare the HTTP objects mix to the capacity volume mix, as we presented in September 2012.

The winner of the HTTP objects contest – Facebook, reaches 4-6th when it comes to volumes. CDNs such as Limelight and Edgecast, contributing significant volumes, apparently do that with much larger objects, hence do not appear on the HTTP objects map.
Dropbox, on the other hand, is an HTTP pump but at relatively low throughput volumes. My guess – HTTP graph provides early hints to the rise of new content pumps. As Dropbox free storage quota grows, we will see more and more traffic from their AS.
Want another hint? I was very surprised to hear today that Amazon’s traffic is balanced. Yes, their incoming traffic is as big as their outgoing. How come? Perhaps it has to do with Dropbox stored on Amazon’s S3.

Is there a case for Internet Exchange Points in Internet edges?

We have recently witnessed proliferation of Internet Exchange Points (IXP) in remote locations, away from the main traffic hubs. More and more IXPs are established in S.E.Asia, Africa, Middle East; some by NPOs (e.g. ISOC, NSRC), some by well established Exchanges (e.g. DE-CIX), and some by local for-profit ventures.

Local IXP serves as the meeting point between local networks, instead of exchanging traffic abroad, as viewed in the following image (source: Analysys Mason).

Traffic flow without (left) and with (right) IXP

Traffic flow without (left) and with (right) IXP. Source: Analysis Mason, 2012

The benefits in having a local IXP are:

  • Saving international bandwidth cost
  • Improving QoE by reducing latency for local content
  • Encouraging hosting of local content
  • Pulling in international CDNs
  • Enabling local online services (e.g. governmental)

The commercial value of setting up an IXP in emerging markets is quite elusive. Critical local traffic mass is required to justify IXP establishment. Yet international capacity prices, poor QoE and lack of local content inhibit accumulation of such mass.

At DiViNetworks we continuously analyze which IXPs to join and connect to our DiViCloud network. Looking at the global data flow, we calculate the benefits our customers and will enjoy if we join an IXP. We have already directly joined  HKIX (Hong Kong), Equinix-Singapore, Equinix-SydneyMSK-IX (Moscow) and PTT Metro (Sao Paulo). We participate in all major IXPs through our upstream providers and plan to join additional IXPs soon.

Packet Loss – International and Access – Asia, LATAM, Africa

Handling over 100Gbps for over 50 ISPs worldwide, combined with smart devices on international and domestic PoPs, the DiViCloud Network holds a unique view on packet flow in international networks.

We recently are taking a closer look at packet loss on international and domestic links. Surprisingly, high packet loss occurs also during off-peak hours, without any congestion on the line – see results below. Such packet loss results in compromised QoE.

Packet loss – International and Access – DiViCloud’s Hong Kong PoP – Serving S.E.Asia

Packet loss – International and Access – DiViCloud’s NYC PoP – Serving LATAM

Packet loss – International and Access – DiViCloud’s Amsterdam PoP – Serving Africa

Summary of the above findings is presented below.

Packet Loss on International Links as Measured by DiViCloud

Holding both sides of the link, DiViNetworks can almost eliminate such packet losses, significantly improving end-users QoE.

An interesting case occurred this week, when a carrier providing transport from Singapore to one of our customers, suffered fiber fault. The following figures present the packet-loss rate prior to the fault, and during the fault.

Packet loss – International and Access – DiViCloud’s Amsterdam PoP – Serving Africa

Packet loss – International and Access – DiViCloud’s Singapore PoP – Serving S.E.Asia – During Fiber Fault

Additional observations are offered in our Global Data Flow Report.

Packet loss in International vs. Access networks in LATAM and Africa

As previously discussed, our DiViCloud global network of smart PoPs provides us with tons of insight about what’s going on in the big-Internet. We have shared some with you in our Global Data Flow Report.

This time we are taking a look at packet loss. High packet loss significantly compromises end-users’ experience and deteriorates link utilization. The reason being that packet loss signals to the TCP peers that the link is close to full, in which case, being a polite protocol, TCP reduces transmission rate.

The graphs below present packet loss in the Access network vs. packet loss on the International link. The measurements are taken on the DiViCloud PoP in London, serving Africa, and the DiViCloud PoP in NYC, serving LATAM.

Retransmissions in Access vs. International networks, as measured by DiViCloud

Clearly, such packet loss rates contribute to poor QoE. We’ll keep you posted about novel ways to eliminate this yellow-blue stuff, for the favor of the ISPs, content providers and CDNs alike.

How did we get access to data flows of 100 ISPs worldwide?

Our Global Data Flow Report became very popular, with hundreds of downloads and shares. We received many questions about the report and its observations. The #1 question had to do with the way we gained access to the data.

Our DiViCloud network serves ISPs worldwide with virtual capacity. DiViCloud comprises of PoPs (points of presence) located at major Internet junctions.

Close to 100 ISPs are served by the DiViCloud network. 100% of the international downstream traffic to the served ISPs passes through these PoPs. As traffic passes through the PoPs, the software in the PoPs collects traffic statistics. These statistics serve as the raw data for the current report.

The information gathered includes: Originating AS (autonomous system), Destination AS, Traffic volume and Bit-stream repetition. We analyzed this raw information to obtain the analyses in the report, such as the one below.

Please share you observations or questions about the Global Data Flow Report.

CDN Contribution to Int’l Traffic in Different Regions

Our Global Data Flow Report Is Now Available

As our earlier blog posts here and here, we discussed global data flow and how we would publish our findings in DiViNetworks’ Global Data Flow Report. DiViNetworks holds a unique point of view – global PoPs positioned at strategic Internet junctions, combined with intelligent software overseeing masses of international traffic. This position enables us to study global data flow characteristics and trends. We would like to share these observations with you.

Below is an example of one of the graphs illustrated in the report to show originating AS source in different regions.

Originating AS (autonomous system) source in different regions

If you would like to gain access to the complete Global Data Flow Report – September 2012 please click here.

Slowdown in global Internet growth – trend or hiccup?

According to Telegeography’s Alan Mauldin, Global Internet capacity reaches 77Tbps, yet the growth is in decline due to a proliferation of CDNs and Cache.

The observation is indeed true, but it should be noted that the entry of a CDN or local Caching has merely a one time impact on growth ratios.

The analysis below simulates a single network operator, with some basic growth assumptions. Once the CDN and/or Caching solution is introduced into the network, a singular deceleration of international traffic is witnessed. In the following year demand reverts back to the organic pace – which is the product of broadband penetration and broadband speed.

The deceleration observed by Telegeography is a result of gradual introduction of CDN and/or Caching solutions . As such, systems will cover the market growth and return to its organic pace.

Want to learn more about Global Traffic? Follow our LinkedIn for or you can go here and fill out a form to be notified when the Global Data Flow report is available.

New DiViCloud PoP in Sydney, Australia

We’re happy to announce the DiViCloud network expansion with our newest DiViCloud PoP in Sydney, Australia. We now operate 12 PoPs worldwide.

The DiViCloud PoPs Map

DiViCloud PoPs are located close to content sources, rather than close to eyeballs.The opposite is true for a CDN which is placed close to the content consumers. By selecting such locations, DiViCloud can apply the technology on almost all the traffic transferred to the ISPs and thus generate more virtual capacity.

How do we know where to place PoPs? We continuously analyze traffic sources and routes. As we see more traffic originating in a new location we conduct an economic analysis for capturing this traffic. We need to conclude if the price gap can be bridged with this part of the traffic.

Since we have started serving virtual capacity to the pacific, we learned that more and more traffic originates in Australia. The main driver being CDNs, who established and scaled up their Australian PoPs. Wholesale capacity economics in Australia and the pacific is challenging jigsaw puzzle; and we have solved it.

Using our new PoP we can now improve our service to Australia, New Zealand and Pacific ISP customers.

Adding S.E.Asia and Carrier Traffic to our Global Data Flow Report

We recently released a first glimpse to our Global Data Flow report. Located at major Internet junctions, the DiViCloud network oversees masses of traffic in different locations, providing us with insights about traffic patterns and sources. Below is an updated graph, which includes information about traffic in S.E.Asia, as well as traffic originating from major local and global carriers.

CDN and Carrier contribution to ISP’s traffic, as measured by DiViCloud network in: LATAM, Africa, Europe, S.E.Asia and CIS.

Follow our LinkedIn or you can go here and fill out a form to be notified when the report is available.