How did we get access to data flows of 100 ISPs worldwide?

Our Global Data Flow Report became very popular, with hundreds of downloads and shares. We received many questions about the report and its observations. The #1 question had to do with the way we gained access to the data.

Our DiViCloud network serves ISPs worldwide with virtual capacity. DiViCloud comprises of PoPs (points of presence) located at major Internet junctions.

Close to 100 ISPs are served by the DiViCloud network. 100% of the international downstream traffic to the served ISPs passes through these PoPs. As traffic passes through the PoPs, the software in the PoPs collects traffic statistics. These statistics serve as the raw data for the current report.

The information gathered includes: Originating AS (autonomous system), Destination AS, Traffic volume and Bit-stream repetition. We analyzed this raw information to obtain the analyses in the report, such as the one below.

Please share you observations or questions about the Global Data Flow Report.

CDN Contribution to Int’l Traffic in Different Regions

How much redundant traffic is left over by CDNs and Caching?

Fascinating to note that even in the presence of local CDNs and local Caching, international traffic still includes very high redundant traffic, passing over and over again on those expensive links.

International traffic repetition-rate by AS in ISPs with and without local caching

To gain access to the complete Global Data Flow Report – September 2012 please click here.

Related Infographic:

How repetitive is the Internet? (Infographic)

Our Global Data Flow Report Is Now Available

As our earlier blog posts here and here, we discussed global data flow and how we would publish our findings in DiViNetworks’ Global Data Flow Report. DiViNetworks holds a unique point of view – global PoPs positioned at strategic Internet junctions, combined with intelligent software overseeing masses of international traffic. This position enables us to study global data flow characteristics and trends. We would like to share these observations with you.

Below is an example of one of the graphs illustrated in the report to show originating AS source in different regions.

Originating AS (autonomous system) source in different regions

If you would like to gain access to the complete Global Data Flow Report – September 2012 please click here.

Adding S.E.Asia and Carrier Traffic to our Global Data Flow Report

We recently released a first glimpse to our Global Data Flow report. Located at major Internet junctions, the DiViCloud network oversees masses of traffic in different locations, providing us with insights about traffic patterns and sources. Below is an updated graph, which includes information about traffic in S.E.Asia, as well as traffic originating from major local and global carriers.

CDN and Carrier contribution to ISP’s traffic, as measured by DiViCloud network in: LATAM, Africa, Europe, S.E.Asia and CIS.

Follow our LinkedIn or you can go here and fill out a form to be notified when the report is available.

Global CDN traffic contribution in LATAM, CIS, Africa and Europe

We recently announced our Global Data Flow report, analyzing the flow of mass data in our growing DiViCloud network. We are busy finding our way in the masses of information accumulated in our PoPs, so in the meanwhile I wanted to give you a glimpse.

Global CDNs (content distribution networks) play a primary role in traffic flowing into ISP networks, comprising around 40% of the overall traffic volume. The figure below depicts the CDN mixture in various territories.

CDN Traffic Contribution DiViCloud

CDN contribution to ISP’s traffic as measured in the DiViCloud network

The Global Data Flow report will include, inter alia, the contribution of global CDNs  by territory, network, time-of-day and other factors. We will also analyze other sources of content, their logical and geographical flow, as well as overlaps and differences between ISPs in different territories.

Follow our LinkedIn or you can go here and fill out a form to be notified when the report is available.

Infographic – To be or not to be Unique?

We did a little bit of research to find out if our web surfing habits are really unique like we would all like to believe, or, are we all viewing the same content?

Click on the infographic (you may want to zoom in), and you’ll find out  that 50% of the data traffic has already passed the network – even in a short window of 6 hours. Isn’t that a waste of expensive bandwidth?

In Yair‘s post “we are all individuals,” it was explained that with as little as 150GB of storage at the network edges, most of this redundant data can be saved without any deterioration in service.