From ocl at gih.com  Mon Sep  6 17:16:14 2010
From: ocl at gih.com (Olivier MJ Crepin-Leblond)
Date: Mon, 06 Sep 2010 18:16:14 +0200
Subject: [IPv6crawler-wg] Fwd: Re: IPv6 Crawling project info
Message-ID: <4C8513CE.3010006@gih.com>

 Hello everyone,

some feedback from Christian de Larrinaga, ex chair of ISOC England, and
also a veteran in UK Internet. Please read the attached message as well,
with suggestions from him.
In a separate message, Christian asked if he could forward details of
the Web site to Tim Chown (on the UK's experts on IPv6 - I understand he
wrote some of the IPv6 features), Keith Mitchell (another UK hot shot),
and Peter Kirstein (the father of the Internet in UK). I gave him the
green light, in view of the fact that we want the ball to get rolling.
Kind regards,

Olivier

-------- Message original --------
Sujet: 	Re: IPv6 Crawling project info
Date : 	Thu, 2 Sep 2010 09:25:35 +0100
De : 	Christian de Larrinaga <cdel at firsthand.net>
Pour : 	Olivier MJ Crepin-Leblond <ocl at gih.com>
Copie ? : 	Tricia Drakes <tricia.drakes at parvil.demon.co.uk>, Dominic
Pinto <dominic.pinto at ieee.org>, Desiree Miloshevic
<dmiloshevic at afilias.info>



Olivier 

This is great progress! 

Minor suggestion .. Abstract end of first para... suggest deletion of 
"The detailed project application is available elsewhere." 

delete this bit ... and will not be repeated here since it would distract from the Team?s achievements."
if the project application is online give the URI. 

This is some data I can see on the peers so far.
http://bgp.he.net/AS31493

BGP Peers Observed (all): 4
BGP Peers Observed (v4): 4
BGP Peers Observed (v6): 1
IPs Originated (v4): 8,192
AS Paths Observed (v4): 120
AS Paths Observed (v6): 54

Average AS Path Length (all): 3.632
Average AS Path Length (v4): 3.508
Average AS Path Length (v6): 3.907


ASN	Name
AS8468	ENTANET International Ltd
AS9191	NEWNET plc Autonomous System Peers
AS3356	Level 3 Communications, LLC
AS8426	ClaraNET



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100906/79761b53/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chart.png
Type: image/png
Size: 8338 bytes
Desc: not available
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100906/79761b53/attachment-0001.png>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Portion de message jointe
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100906/79761b53/attachment-0001.pl>

From ocl at gih.com  Mon Sep  6 17:20:51 2010
From: ocl at gih.com (Olivier MJ Crepin-Leblond)
Date: Mon, 06 Sep 2010 18:20:51 +0200
Subject: [IPv6crawler-wg] Fwd: Re: SV: SV: SV: Tracking usage of IPv6
Message-ID: <4C8514E3.9090509@gih.com>

 More feedback: Patrik F?ltstr?m is one of those really influential
people out there...
http://www.intgovforum.org/cms/index.php/component/chronocontact/?chronoformname=2010PanelistBioView&wspid=117

I will see him in Vilnius next week and give him a deeper demo of our
Web site.

Kind regards,

Olivier

-------- Message original --------
Sujet: 	Re: SV: SV: SV: Tracking usage of IPv6
Date : 	Wed, 1 Sep 2010 18:42:31 +0200
De : 	Patrik F?ltstr?m <paf at cisco.com>
Pour : 	Olivier MJ Crepin-Leblond <ocl at gih.com>
Copie ? : 	Torbj?rn Ekl?v <torbjorn.eklov at interlan.se>



Thanks!

Great to see people actually doing things!

   paf

On 1 sep 2010, at 18.41, Olivier MJ Crepin-Leblond wrote:

> 
> And in my haste, I forgot to mention the Web site's address:
> http://www.ipv6matrix.org/
> 
> :-(
> 
> Le 01/09/2010 18:35, Olivier MJ Crepin-Leblond a ?crit :
>> Hello Torbj?rn,
>> Hello Patrik,
>> 
>> no luck for the t-shirt photo, although I'll have a picture of myself
>> taken wearing the t-shirt on the Beach in Cannes.
>> 
>> I'm coming back to you (better late than never) because I got a green
>> light from ISOC to start publicising the Web site resulting from our
>> IPv6 Crawler project.
>> I am now facing the task of populating the FAQ page with explanations of
>> how it all works, and what it all means, the problem being that I have
>> spent so much time on the software that I find everything to be obvious
>> - and I'm told it's not, and indeed, it isn't.
>> I'm also getting the programming team to work out the bugs - and there
>> are a dozen which we're aware of, mostly on the front end.
>> Please find my report to ISOC for explanations at this stage. Look
>> forward to your feedback & questions!
>> 
>> Kindest regards,
>> 
>> Olivier
>> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100906/3a5a068e/attachment.html>

From ocl at gih.com  Wed Sep  8 17:16:52 2010
From: ocl at gih.com (Olivier MJ Crepin-Leblond)
Date: Wed, 08 Sep 2010 18:16:52 +0200
Subject: [IPv6crawler-wg] IPv6 Crawling project info
In-Reply-To: <490F5AE5-24B5-4EA2-9A23-8E4A61F9D4A6@firsthand.net>
References: <4C7E8306.8060707@gih.com>
	<490F5AE5-24B5-4EA2-9A23-8E4A61F9D4A6@firsthand.net>
Message-ID: <4C87B6F4.80605@gih.com>

 Hello Christian,

thank you very much for your kind reply. I really value your feedback
and look forward to receiving feedback (and Frequently Asked Questions
to populate the online FAQ) from others on the distro above.

Le 02/09/2010 10:25, Christian de Larrinaga a ?crit :
>
>
>  
> 2020media are not members of LINX nor LONAP. So you are having to organise direct peers I suppose? As the installation is in Telehouse might it not be worth contacting John Souter to see if they will sponsor a port through LINX and give LONAP a nudge? If 2020 don't need to peer directly perhaps the project / ISOC England could.  I'd be happy to negotiate this on your behalf. 

I'll need to discuss this with 2020Media. I am unsure as to what peering
directly would entice and will await feedback from them before deciding
on this.

> In 3.3.1
> It is good to see data will come through for www. dns. mx. and even popping the downstream servers. It would also be interesting to see use of sip. and possibly naptr as these are critical for voice services which are increasingly dependent on IP. 

The trouble we're hitting is how to test for it without generating too
much unusual traffic which would trigger firewalls. Also - it appears
that use of naptr & sip is not so widespread when compared to the other
services.

> In 3.3.4.1
> In testing for IPv6 connectivity, and with no agreed standards or methodology, we have found that the results we have gathered are somehow subjective and that more accurate trends are more likely to appear as more results are gathered over time.
>
> Yes this is not unexpected. It might be useful to add a process to get this to happen (help standardise methodologies). It is worth talking to NANOG and UKNOF both to let people know about the project but also to get feedback and input. 

Would you recommend that we speak to UKNOF before validating our
results? I know that the chaps at 2020Media are regular attendees at
their meetings.

> In 3.5.3 Mistakes made
> I am very impressed you have managed to find a way through the maze. It was clear after the domain spring clean project the chapter worked on in 2000 that DNS suffers entropy. 

All credits go to Dr. Al Ansary and his team for this... and we're
learning about more potential mistakes every day. I've now come across
DNS records which contain commas! I didn't even know BIND could work
these out...

> In 3.5.4 Future work
> In its current status, the project has laid the foundation for wide-scale crawling and we are sitting on top of a mine of valuable data. The project time and resources were sufficient to do primary analysis of this valuable data. The logical next step is to start a sequel project to curate, filter and analyze this data.
>
> Might it be helpful to have an API so that other projects can interface to the data directly?

The API exists, it just needs to be explained, which is what I'll try
and do in the second part of the report. Effectively, the Web site and
its click-on filters generate an automated query with the back end
running on port 4444 of the machine, and which responds to SQL queries.
So you can format SQL queries in an HTML line and get results accordingly.

> I've taken a brief glance over the website at 212.124.204.162 and www.ipv6matrix.com and it is a great start. 
> I think this is brilliant Olivier. Well done to you and the partners you enthused to get involved. This has clearly been a lot of work. 
>

Thank you. ISOC's feedback so far points to the same direction, although
they have asked me to populate the FAQ part of the Web site, which is
what I am doing every time I can think of a question to put in there.
Any other suggestions?

Kind regards,

Olivier

-- 
Olivier MJ Cr?pin-Leblond, PhD
http://www.gih.com/ocl.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100908/2b4dc81c/attachment.html>

From cdel at firsthand.net  Thu Sep  9 11:11:39 2010
From: cdel at firsthand.net (Christian de Larrinaga)
Date: Thu, 9 Sep 2010 11:11:39 +0100
Subject: [IPv6crawler-wg] IPv6 Crawling project info
In-Reply-To: <4C87B6F4.80605@gih.com>
References: <4C7E8306.8060707@gih.com>
	<490F5AE5-24B5-4EA2-9A23-8E4A61F9D4A6@firsthand.net>
	<4C87B6F4.80605@gih.com>
Message-ID: <67F7436B-3AF9-409C-B954-7FA0C3304053@firsthand.net>

Peering: I didn't go into depth but when I looked t didn't see 2020 on the peering matrices.  2020 may either be peering indirectly (i.e., through their backbone provider) or allocated physical ports and negotiating on a case by case basis. Either way this is likely to cause you additional suspicions from downstream networks who wonder about all the TCP/UDP port popping you do.  If you peer at an exchange point as an AS with BGP then you are both only using one port (physical) but also set up direct peers with the various ISP's and networks and can explain your good conduct directly. 

SRV and NAPTR (SIP and E2E etc): I think the point about these is to see how support for Voice and similar over IPv6 is progressing. It is early days as so few SIP vendors (for instance) support v6 across their voice switch fabric today.  BUT voice services over v6 is a natural (or so one (I) would think!). Anyway I'd be interested!

UKNOF: You should deliver a presentation at the next UKNOF (rather than just 2020). They can support you doing this online at a pinch. I'll wire you in with Keith. 

DNS: commas! You have to smile. Another catcha for IDN's?

API: great! This should save you work in the end! AND something to mention in your presentation to tech. communities NANOG, UKNOF etc. 


Christian



On 8 Sep 2010, at 17:16, Olivier MJ Crepin-Leblond wrote:

> Hello Christian,
> 
> thank you very much for your kind reply. I really value your feedback and look forward to receiving feedback (and Frequently Asked Questions to populate the online FAQ) from others on the distro above.
> 
> Le 02/09/2010 10:25, Christian de Larrinaga a ?crit :
>> 
>> 
>>  
>> 2020media are not members of LINX nor LONAP. So you are having to organise direct peers I suppose? As the installation is in Telehouse might it not be worth contacting John Souter to see if they will sponsor a port through LINX and give LONAP a nudge? If 2020 don't need to peer directly perhaps the project / ISOC England could.  I'd be happy to negotiate this on your behalf. 
> 
> I'll need to discuss this with 2020Media. I am unsure as to what peering directly would entice and will await feedback from them before deciding on this.
> 
>> In 3.3.1
>> It is good to see data will come through for www. dns. mx. and even popping the downstream servers. It would also be interesting to see use of sip. and possibly naptr as these are critical for voice services which are increasingly dependent on IP. 
> 
> The trouble we're hitting is how to test for it without generating too much unusual traffic which would trigger firewalls. Also - it appears that use of naptr & sip is not so widespread when compared to the other services.
> 
>> In 3.3.4.1
>> In testing for IPv6 connectivity, and with no agreed standards or methodology, we have found that the results we have gathered are somehow subjective and that more accurate trends are more likely to appear as more results are gathered over time.
>> 
>> Yes this is not unexpected. It might be useful to add a process to get this to happen (help standardise methodologies). It is worth talking to NANOG and UKNOF both to let people know about the project but also to get feedback and input. 
> 
> Would you recommend that we speak to UKNOF before validating our results? I know that the chaps at 2020Media are regular attendees at their meetings.
> 
>> In 3.5.3 Mistakes made
>> I am very impressed you have managed to find a way through the maze. It was clear after the domain spring clean project the chapter worked on in 2000 that DNS suffers entropy. 
> 
> All credits go to Dr. Al Ansary and his team for this... and we're learning about more potential mistakes every day. I've now come across DNS records which contain commas! I didn't even know BIND could work these out...
> 
>> In 3.5.4 Future work
>> In its current status, the project has laid the foundation for wide-scale crawling and we are sitting on top of a mine of valuable data. The project time and resources were sufficient to do primary analysis of this valuable data. The logical next step is to start a sequel project to curate, filter and analyze this data.
>> 
>> Might it be helpful to have an API so that other projects can interface to the data directly?
> 
> The API exists, it just needs to be explained, which is what I'll try and do in the second part of the report. Effectively, the Web site and its click-on filters generate an automated query with the back end running on port 4444 of the machine, and which responds to SQL queries. So you can format SQL queries in an HTML line and get results accordingly.
> 
>> I've taken a brief glance over the website at 212.124.204.162 and www.ipv6matrix.com and it is a great start. 
>> I think this is brilliant Olivier. Well done to you and the partners you enthused to get involved. This has clearly been a lot of work. 
>> 
> 
> Thank you. ISOC's feedback so far points to the same direction, although they have asked me to populate the FAQ part of the Web site, which is what I am doing every time I can think of a question to put in there. Any other suggestions?
> 
> Kind regards,
> 
> Olivier
> 
> -- 
> Olivier MJ Cr?pin-Leblond, PhD
> http://www.gih.com/ocl.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://salsa.gih.co.uk/pipermail/ipv6crawler-wg/attachments/20100909/639ad834/attachment-0001.html>

From colinj at mx5.org.uk  Sat Sep 18 16:05:51 2010
From: colinj at mx5.org.uk (Colin Johnston)
Date: Sat, 18 Sep 2010 16:05:51 +0100
Subject: [IPv6crawler-wg] Response to port scanning security alert
In-Reply-To: <4C56DC38.2040004@gih.com>
References: <4C546A57.5020605@gih.com>
	<6387F274EB8F4C74900DA5F094135480@EricPC>
	<4C55F0DE.40205@gih.com> <4C56DC38.2040004@gih.com>
Message-ID: <50DAB972-40AD-4646-862E-22957869E76D@mx5.org.uk>

same scanning happened last week as well

Why ?

Colin

On 2 Aug 2010, at 15:54, Olivier MJ Crepin-Leblond wrote:

> Eric,
> 
> just a quick follow-up:
> 
> On 01/08/2010 23:10, Olivier MJ Crepin-Leblond wrote :
>> 
>> 
>>> Two suggestions ..
>>> 
>>> 1.  We suggest that you put text at the index page of the web
>>> server(s) http://212.124.204.162/ and http://turtle.ipv6matrix.org/
>>> explaining about your project. - perhaps some of your text in this
>>> email might help.  It is usual for search engine robots to leave a
>>> trail in the scanned server log file giving the url of the robot and
>>> if the server owner accesses that url it says something like "We are a
>>> robot collecting ..." This is reassuring to the scanned server owner.
>>> 
>> Very good idea indeed. I was initially thinking of shutting the Web
>> server down, but now that you mention this, I'll put something together
>> this week, with a link to the www.ipv6matrix.org results.
>> 
> 
> Done.
> 
>>> 2. Somehow our firewall detects that your IP address is called
>>> turtle.ipv6matrix.org  We wonder if the wording might be changed.
>>> networkscan.ipv6matrix.org   would be more meaningful.  We get many
>>> scanning attacks from all over the web and often the apparent source
>>> IP address has many PCs hidden in a LAN behind it.
>>> 
>> We've got a CNAME as crawler.ipv6matrix.org, but I think you're right,
>> this would probably be helpful to anyone enquiring, to swap the name &
>> cname round. Yesterday, I added a DNS TXT field to turtle.ipv6matrix.org
>> which points to the Web server. I'll ask my Team what they think of the
>> idea of swapping the names around and act accordingly.
>> 
> 
> Done.
> 
> Warm regards,
> 
> Olivier
> 
> -- 
> Olivier MJ Cr?pin-Leblond, PhD
> http://www.gih.com/ocl.html
> 



From ocl at gih.com  Sun Sep 19 16:54:33 2010
From: ocl at gih.com (Olivier MJ Crepin-Leblond)
Date: Sun, 19 Sep 2010 17:54:33 +0200
Subject: [IPv6crawler-wg] Response to port scanning security alert
In-Reply-To: <50DAB972-40AD-4646-862E-22957869E76D@mx5.org.uk>
References: <4C546A57.5020605@gih.com>
	<6387F274EB8F4C74900DA5F094135480@EricPC>
	<4C55F0DE.40205@gih.com> <4C56DC38.2040004@gih.com>
	<50DAB972-40AD-4646-862E-22957869E76D@mx5.org.uk>
Message-ID: <4C963239.5070100@gih.com>

 Hello Colin,

indeed, the UK zone was crawled last week.
This will be done monthly and results recorded, to identify trends in
the spread of IPv6 over time.
As I mentioned in my previous message:

> The tests will take place on a monthly basis, and you therefore might be
> receiving recurring security alerts in the future, unless you can
> parameter your firewall to ignore alerts from our crawler which runs on
> from 212.124.204.162.
> Alternatively, I would be happy to take your domain name out of the
> crawler's site testing list. If so, please email me the list of domain
> names which you are in charge of and I'll make sure they are removed
> from the testing list.


You can see the results on the Web Site's data archives.
http://www.ipv6matrix.org/

Kind regards,

Olivier

Le 18/09/2010 17:05, Colin Johnston a ?crit :
> same scanning happened last week as well
>
> Why ?
>
> Colin
>
> On 2 Aug 2010, at 15:54, Olivier MJ Crepin-Leblond wrote:
>
>> Eric,
>>
>> just a quick follow-up:
>>
>> On 01/08/2010 23:10, Olivier MJ Crepin-Leblond wrote :
>>>
>>>> Two suggestions ..
>>>>
>>>> 1.  We suggest that you put text at the index page of the web
>>>> server(s) http://212.124.204.162/ and http://turtle.ipv6matrix.org/
>>>> explaining about your project. - perhaps some of your text in this
>>>> email might help.  It is usual for search engine robots to leave a
>>>> trail in the scanned server log file giving the url of the robot and
>>>> if the server owner accesses that url it says something like "We are a
>>>> robot collecting ..." This is reassuring to the scanned server owner.
>>>>
>>> Very good idea indeed. I was initially thinking of shutting the Web
>>> server down, but now that you mention this, I'll put something together
>>> this week, with a link to the www.ipv6matrix.org results.
>>>
>> Done.
>>
>>>> 2. Somehow our firewall detects that your IP address is called
>>>> turtle.ipv6matrix.org  We wonder if the wording might be changed.
>>>> networkscan.ipv6matrix.org   would be more meaningful.  We get many
>>>> scanning attacks from all over the web and often the apparent source
>>>> IP address has many PCs hidden in a LAN behind it.
>>>>
>>> We've got a CNAME as crawler.ipv6matrix.org, but I think you're right,
>>> this would probably be helpful to anyone enquiring, to swap the name &
>>> cname round. Yesterday, I added a DNS TXT field to turtle.ipv6matrix.org
>>> which points to the Web server. I'll ask my Team what they think of the
>>> idea of swapping the names around and act accordingly.
>>>
>> Done.
>>
>> Warm regards,
>>
>> Olivier
>>
>> -- 
>> Olivier MJ Cr?pin-Leblond, PhD
>> http://www.gih.com/ocl.html
>>
>

-- 
Olivier MJ Cr?pin-Leblond, PhD
http://www.gih.com/ocl.html