[IPv6crawler-wg] Ideas for mapping / display of results

Sun Apr 11 00:34:47 BST 2010

Hi there,

with the great progress made by Sameh and his team to get the crawler to
work and collect data, one of the important parts of the IPv6 crawler
project involves the display of information that the crawler has
collected. I know that Sameh is putting together some automatically
generated pie charts of IPv4/IPv6 penetration criteria per
region/gTLD/ccTLD or per other type of selection (the selection of
domain names being our choice, for example, only UK academic
institutions), but we now need to look further.
Therefore, I think that since we are now seeing output data from the
crawler we need to:

1. define & freeze the output format. If the current format is what we
are all happy with, then let it be.
2. start defining the type of analysis we would like to perform on this
data. This might involve having to choose what type of
output/deliverables we would like, bearing in mind the output we have
proposed in the specifications sheet which was part of the ISOC project
funding proposal:

"For example, records based on Geo-localisation will enable geographic
colouring of
international maps showing concentration of IPv6 nodes. Geo-IP databases
exist, both on a
commercial basis (For example, MaxMind -- http://www.maxmind.com ) or
Open Source (For
example, hostip.info -- http://www.hostip.info ). Open Source Databases
often allow
downloading of the complete database (and updates) and this would be
used within the tracing
software to locate IPv6 enabled nodes, through their IPv4 dual stack.

Mapping of results on World maps is then possible, either by formatting
data in a format
readable by Google World, or through direct mapping such as that shown
below.
On this map (example from the hostip.info site), small red dots
represent access to a Web
page from specific regions of the world. Dual Stack IPv4/IPv6 nodes
could be mapped. Later,
when IPv6 only nodes start appearing, they could be plotted in another
colour (as long as Geo
Tracking source is established for IPv6).
Other possible visualisation could be a mapping of the IPv6 world; rate
of IPv6 growth in
particular regions, a node-centric mapping of the virtual Internet space
with distances defined
as end to end delays, etc.
A study of further data visualisation is outside the scope of this
project, given the in kind
funding secured from current project partners."

The GEO database referred to above is hostip.info and the HostIP
complete database can be downloaded from a link on:
http://www.hostip.info/dl/index.html

So here are some suggestions/ideas for analysis/display of data:

A conventional way of showing data is through tables. I know we already
have some in the first prototype but the examples following seem
appropriate as well.

* Some slightly nicer looking set of tables are possible here.
https://fit.nokia.com/lars/meter/ipv6.html
It is worth noting that at the moment, the data which our crawler is
collecting, is probably able to go further than the Nokia project, since
they are only testing a few countries, and we are unsure whether they
are testing for real IPv6 connectivity or just going by the IP addressing.
That said, they are also testing for DNSSEC, DKIM, SIP, XMPP and SPF:
https://fit.nokia.com/lars/meter/data/2010-04-08-results.txt

Sameh: with the current modularity of our crawler, will it be possible
to add those features with ease? I am hoping that this is so - and I am
hoping that therefore, we also keep a similar modularity for the
generation of our analytical result displays, so that we don't need to
re-program everything from scratch if we wish to extend our data sets in
the future. The reason for this is that this might allow us to ask for
further funding from ISOC to extend the work.

But there are also different formats of tables possible.

* For example, look at the table of results for inter-network connectivity:
http://www.internethealthreport.com/Main.aspx?Period=RH24
In this type of format, we could have various metrics being updates at
each run.

And then, we've got various possibility for maps, and I would really
like us to decide on mapping because I really believe that a picture is
worth a thousand words - and it would really help visualise things on a
macro level.

* For example, maps of IPv6 penetration could be created using dots on a
map, or wider coloured regions of the same type as:
http://www.wirelessmapping.com/WISP%20National%20Map.png

or  something using Google maps, like:
http://www.wirelessmapping.com/Google%20Maps3.htm

* Possible historical charts for IPv4 latency vs. IPv6 latency:
http://inetcore.com/project/metrics/46rtt.html.en
(or for the spread of IPv6)

The CyberGeography Web site has many many examples and ideas for
Internet data visualisation.
Pages of particular importance are:

- geographic display
http://personalpages.manchester.ac.uk/staff/m.dodge/cybergeography//atlas/geographic.html
We have several ideas there, including the dots on a map, but also the
stacks on a map, or the colour densities.

- Also some interesting maps on:
http://personalpages.manchester.ac.uk/staff/m.dodge/cybergeography//atlas/census.html
(the map of France with the bubbles, for example.
Also, the Internet Cartogram, where the size of countries is related to
the No. of Internet users)

* There's also an extensive set of further data visualisation techniques on:
http://delicious.com/quarlas/datavisualization

* Also check to some map visualisations by Akamai:
http://www.akamai.com/html/technology/dataviz1.html

These are just a few more ideas that I am suggesting for further study.
Of course, there's the link to the 50 great examples of data
visualisation on:
http://www.webdesignerdepot.com/2009/06/50-great-examples-of-data-visualization/

* but I would particularly like to look at data mining displays on:
http://datamining.typepad.com/gallery/blog-map-gallery.html

* Or look at the type of data display (globe with stacked dots) on:
http://www.twingly.com/screensaver

* Of course, I love the globe on:
http://www.geocodearth.com/
I have emailed the owner of the site to find out what were the terms of
use of the globe and he replied to me with:
"This project use Papervision for the 3d sphere.
Papervision is an OpenSource project so you can build you own sphere for
free.

All other component such as drag, rotation, spin are part of Geocodearth
and proprietary."

So I searched for Papervision 3D ( http://www.papervision3d.com ) and
found a link to a demo:
http://www.psyked.co.uk/adobe/flash/geocoding-google-maps-locations-to-a-papervision3d-globe.htm

with the demo itself by Adobe on:
http://www.adobe.com/newsletters/edge/june2008/articles/article2/download/GeoGlobe.html

and the article by Adobe on how to do it:
http://www.adobe.com/newsletters/edge/june2008/articles/article2/index.html

There is another tutorial for a similar project explaining how to do it
(with a different bitmap) on:
http://www.insideria.com/2009/02/creating-an-interactive-globe.html

Lots more examples & how-to discussions by doing a Google search on:
papervision3d globe

Okay -that's all for now. I'd like some feedback and suggestions from
everyone here, please. It's very welcome at this stage.

Warmest regards,

Olivier

-- 
Olivier MJ Crépin-Leblond, PhD
http://www.gih.com/ocl.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100411/a87cc86a/attachment.html>