These are unedited transcripts and may contain errors.
IPv6 Working GroupThursday, 17th of October, 2013, at 9 a.m.:
MARCO HOGEWONING: It's close to 9 o'clock or it is 9 o'clock. Welcome, this is the second IPv6 Working Group, while I am being interrupted by a bell.
Yesterday, we had our first session, which was great, a lot of discussion, a lot of questions, very nice presentations. And for the next one, for those of you who were here yesterday know that we had to push a talk back to this morning, Yannis again, thanks for volunteering and getting up so early.
In the meantime, if you are around we still have this IPv6 only network, if you have any trouble, please tell us at the end of the session, that is where we are going to discuss the feedback, there is the password and please don't harass our technical team if anything goes wrong because they have got nothing to do with it.
This was the agenda as it was planned and as it is published on the website, we have got Benedikt, that is actually a presentation I asked for specifically because we keep running into people that ask us like how can we do this. After a chat with Benedikt in Berlin this summer, he said: "I think I know how to do it". So he is going to present his solution.
Jen Linkova looking a bit on IPv6 source addresses, we have got Tassos talking about his project ?? me and my partner in crime Andrew Yourtchenko, who is going to talk about her IPv6?only experiment.
Housekeeping, please switch off your cell phones or at least turn them to silent. Well, the doors are in the back back in case of emergency. Anything else, we are monitoring this and webcasting, if you do have a question, please come to the microphone and please state your name and affiliation so that people back home know who they are talking with. For the people back home, we have remote participation available, Rumy here is happy to relay your comments, so if you are on the channel please speak up and she can repeat it to the room.
And once again, thanks again for RIPE NCC for providing our scribe, Fatima, and all the text support, of course.
So here the IPv6 only, this is what we are going to do, apart from Yannis who is going to talk first, if you can squeeze it in your ten minutes that we have ?? we came late yesterday, we try to get you up, and thanks for the other speakers for leaving the room to do this.
Yannis: Hello, good morning. I work for a company called OTE in Greece, and I am here to present you a case study about an IPv6 addressing plan for an Internet service provider. Actually, this is our addressing plan. Before we get into the addressing plan, I would just like to take one minute and remind everyone that OTE has dual stack all of its network and all of its services are available, all of its connectivity services are available through dual stack, which is we are waiting for people to come in. So that is ?? that is going to presentation.
First of all, I have to say that this IPv6 addressing matter has taken quite a bit of our time and effort and I think this is the third or fourth try into laying out a proper addressing plan, hopefully this is the last one as well. So, what are the qualities of a good IPv6 or a good addressing plan in general?
It has to be future proof and scaleable and also it has to be consistent and it has to help operations and engineering people to lay out some good security policies and it has to be able to produce space that is easily aggregatable and most of it all, it has to be simple. I know that doesn't like it, but it has to be simple. The rules have to be simple.
So, some ground rules for our case:
We got a /29 from RIPE and we were quite happy about it. We decided and we agreed upon giving our customers at least one/56. Well, except maybe mobile subscribers, but even there we are not really sure, because by the time we gave out the dual stack service to our mobile subscribers maybe we will be giving out /56 to them also. So, and finally, no prefixes larger than 64. Again, there might be, and actually there are some special cases and we will talk about them in a little while.
So what we wanted to do, basically, was to incooperate, include some important aspects into the addressing plan and those were location and service, we needed ?? we needed our plan to be location and service?aware and also we wanted to include the traffic type, and when I say traffic type, it could either be trusted or non?trusted, or it could be customer versus infrastructure space. Location could be anything, I mean, the traditional point of presence or a city or a terminating device name, like a BRS name, it's really up to the individual to decide, and what services, well basic services offered like Internet or TV or voice or whatever comes up next.
So let's get into it.
As you can see, I hope you can see, I didn't really test this on a big screen, so I hope you can see the bits here. You can see the bits from 33 up to, all the way up to 64 and the yellow bits are the location bits. And the green bits are bits that can be assigned. X bits we don't care about for now. We have allocated 8 bits for location, which leaves us with 256 locations. For us, that sounds more than enough; we have about, roughly, 50 points of presence and our midterm planning shows about 80 so we say, OK, 80 let's multiply that by 2, what is the next power of 2, 256. We think that this is enough. But even if it isn't enough, what we could do, we can utilise it, a little trick that we will see a little bit later.
Also, we see that we are left with about 65K /56 per location, 16 bits; which, again, for us, is pretty decent. I mean, it's more than enough, and again if it's not enough we can use more than one location, more than one location per POP, so we can use, for example, for Athens we can use Athens, 1, 2, 3, reserve more than one location, and again, we can utilise another trick to increase that 65K number.
Our 56 subnet now looks like this. We can see our location digits, we can see our aggregatable digits, and this is actually one of our prefixes. So, for example, if we have assigned the binary 10110 for Athens, which is 22 nhex, we can see how that translates into the /40, because that is what ?? that is the prefix that is resulting from giving out 8 bits for location, /40.
So let's move on to the next aspect, which is service we can see the same bits, but you can see that we have allocated bits from 56 to 64, for service, and that, for some people, might seem a little odd. Anyway, again, we have 8 bits for our service, 256 services is definitely more than enough, for now and forever, I suppose, so what we could do is, we could reserve more than one subnet, more than one 64 subnet for each service, for example we could reserve the first 10 or 20 for Internet, the next 10 for TV and the next 24 for voice, for example. Or whatever suits the individual needs.
Well, of course, some cuss misation will be needed in the CPE side in order for the CPE to be able to announce the right prefixes to the right LAN ports, etc..
So our 64 subnet now looks like this, we can see the location digits, the assignable digits and as you can see the assignable digits did not change and that is the main reason we are probably sticking with this, because we still have 65K subnets per location. And you can see the service digits as well, so if we assign 01 to the Internet access, this is the subnet here is a customer subnet for Internet access.
And finally, well last but not least, the traffic type; as I said before, we wanted to introduce the traffic type so we could differentiate between trusted and non?trusted traffic or customer and non?customer traffic so we have used the bits 32 through 32, that divided our prefix into /32s, which is quite nice. We can use one of them for infrastructure, is more than enough, and we can use many of those for customers. So we can differentiate as I said between trusted and non?trusted, for example, we can use the last /32 for infrastructure and all of the rest /30,/31 and /32 for non?trusted or customers. I mean, people could also reserve a couple of those for future use. I guess so. This is what our subnet looks like. We have our traffic type digit, location, service, everything, everything is in there.
So, now I would like to mention a few sort of special addressing categories.
Our customers WANs and LANs, point?to?point addresses and VLANs. For user ?? single IPv6 address from a single /64 or 56, because I think some vendor had an issue with that. From each device. So we are locating a /64, for example, per P NG and from that the user gets a single IPv6 address for the CP 1. If needed of course.
For the LANs we said the minimum is /56, of course customers, our business customers can get 48s and even multiple 48s with some justification provided. And that is the way we chose to sort of allocate or reserve the space in each POP, in each location, so we start from the top and we allocating/44s to our residential customers and then start from the bottom, going upwards, allocating/44s for our business customers. There is no real strict boundary here, so it depends on the location.
Point?to?point links, we are using /64s for each link but we reserving a /64 but we are using a/127 per link. So what we could use here, I think I mentioned before and if I didn't, I will mention it now, from the 256 locations, we can reserve some and name them special for users where we want to address something that is not really location?aware. So, for example, point?to?point links fit into that category, and we could use one special location and then use a couple of /48s from that location. For example, we could reserve location DE and then use def and ded which are easy to remember for our point?to?point links, two /48s. If we need another one we can use a third /48 from the same location.
As far as loop back addresses are concerned, single IPv6 address for each address ?? for each loop back address, many loop backs can be figured for each device of course, and again we can use another special location, for example, FF, or 00, that is even simpler, and that way it's easier to differentiate ?? I mean, with us splitting our space into /32s it's easy to differentiate between trusted and non?trusted loopbacks. Sorry, for example, if we want to assign a trusted loop back we assign it from the /32 from a trusted space. If we want to assign a non?trusted loop back assign it from one of the /32s that is reserved for customers.
For VLANs and mostly for our infrastructure VLANs but people can implement that in their customer VLANs as well. We thought that it might be a good idea to incorporate the VLAN idea, use the simplest possible way to do that, which is take the decimal VLAN idea and just put it right into the IPv664 subnet. This is an example. That is about it.
I just wanted to say that this is definitely not original work; I mean, there is some very good pointers, like I am sure ARIN has a good documentation, and I remember Sander Steffann wrote a really nice enterprise guide a few years back and I am sure there is others available, but we thought that this might be a bit useful for other service providers like ourselves. And we also thought that we might be getting some useful feedback from all of you. So thank you very much for your attention. I will be happy to take any questions.
MARCO HOGEWONING: Yes, thank you. Any questions for Yannis?
SHANE KERR: ISC. I don't have questions, but I think it's really interesting in concept, so I really appreciate you sharing it with people, and one of the things we noticed was, this isn't similar to the proposal by Peter Lothberg, but it is, right?
YANNIS: I was afraid to mention that.
SHANE KERR: So you have different goals and different use of all the bits but the basic idea of encoding information other than just raw topology and aggregation into the addresses, is quite interesting and as we discussed yesterday we don't really have policy tools to kind of evaluate this, so what I think we are going to do, we are going to talk about it on the list.
YANNIS: I was going to propose that as well.
SHANE KERR: Excellent. Because we have to do things in our normal way but hopefully we will get good consensus that this kind of experimentation is really the way forward, we have all these bits and a new type of addressing, let's see if we can make it work a little better. Anyway, very cool.
MARCO HOGEWONING: Follow up on the list. In the meantime, incoming message on behalf of the Programme Committee is that their nomination processes closes at 3 o'clock, so if you want ?? tomorrow at 3 o'clock. That was the nomination or the election? Elections.
AUDIENCE SPEAKER: Nominations close at 3:00 today.
MARCO HOGEWONING: So if you want to be on the PC, please make yourself known. I see Ruediger standing there.
RUEDIGER VOLK: I had intended to stay out to comment but after Shane's comment I feel I have to throw a little bit salt here. I am guilty of ?? have been talking about using a few address bits in v6 for semantic purposes, since RIPE meet the last Stockholm meeting which is pretty long ago.
MARCO HOGEWONING: That was RIPE 50, yes.
RUEDIGER VOLK: And a few people have taken that somewhere. The experience is when in IETF this kind of address usage is presented. There is big attack by all the senior architects that have been around all the time and I usually step up to the mike and say, well, OK, calm down a bit, this makes sense. But one has to be really careful not to burn up address space and throw in stuff that uses bits for something that is administrative convenience, that is a way, if one does that, uncontrolled, we will need IPv 7 fairly soon and that will have 2K bits of address, and the other thing is, if you actually define very careful operational and fore forwarding semantics to the bits, you probably do not burn up too much. What I saw in this proposal was going, in my opinion, far too much in the direction of administrative convenience.
MARCO HOGEWONING: Thank you. And as Shane suggested, we are going to take this up on the mailing list so I am sure we can work on that over there.
Thank you, Yannis. And thank you for your flexibility.
(Applause)
Which leaves me with our next speaker, Benedikt, talking about how to get redundancy without running BGP.
BENEDIKT STOCKEBRAND: I am going to talk about a couple of things that turned up at the or before the IETF meeting in Berlin, which I for once attend again and I ran into Marco and after a couple of beers he talked me into giving this sort of presentation. This isn't actually new stuff; it's basically what the IETF originally thought why we don't need PI address space for IPv6 any more. So you probably know this is not going to work in the general case but I guess there are a number of people, customers, small companies, whatever, who actually could benefit from this sort of set?up.
The I have been doing IPv6 training for about ten?and?a?half years now and there was a period a couple of years ago when I was doing a lot of public trainings and most of the participants were these one person IT contractors sort of thing, the people who, in the morning they replace your toner cartridges in your printer and by lunch the packet filters in some fritz box or whatever, and in the afternoon they do network design at a small scale. These people are generally not the sort of people you want to let run BGP.
My idea is or my understanding from these people is that they have an increasing number of customers, smaller ?? small business customers who need, really need network connectivity, and reliable network connectivity and who are using all sorts of more or less weird fall back scenarios which are increasingly painful and we can't possibly have every lawyer or ?? tax advisor, chartered accountant sort of people because they really, really need network connectivity at certain times and these small organisations usually are just a handful of people, we can't have all them do BGP, and even if they could, the co?routers couldn't handle it in the zone.
This is what this is really about. What do we want? We have the Internet, we have an end site which needs redundant connectivity of some sort. The easy way that a lot of people use and a lot of people are not entirely happy about is to have some sort of redundant links to a single ISP who is doing the BGP stuff: Sun link can be fixed whatever, DSL, cable modem, whatever, one might actually be something like 3G dial?up, fall back sort of thing, it might actually be a permanent connection to the ISP, whatever. It's a ?? there are different sorts of customers and different sorts of needs and different sorts of bandwidths involved, so this is an option, but it's not quite the best for everyone. There is another reason why people don't like this:
If you are the customer and you are stuck with an ISP who did a good ?? a very, very good job two years ago, doesn't mean through mergers and acquisitions or whatever the service has gone seriously bad, and then you are pretty stuck with your one ISP, who is not delivering proper quality any more. So this approach can actually be used in the transition period from one ISP to another. So if you try to be a good ISP and get customers from your competitors, this might also be helpful.
So what do we want? We have one ISP, reconnected to, we have a second ISP connected to and we get Internet connectivity for both of them. This is the IPv4 way to do it. We have an ASA for ISPA, a second AS for the second ISP, and the customer run a third AS. Lots of extra routes in the default free zone and we need somebody to run this sort of set?up at the customer site.
I have had at least two customers with this sort of set?up where somebody actually set up the BGP, the only one who knew about BGP actually and then he got a better job and the customer was left basically, OK, don't touch this box, we don't know what it does, but we know what happens if you turn it off. So, this is the sort of dead?end you really want to avoid, and having three or four?year?old BGP set?up that nobody has touched since and nobody understands is not really a way to make your network connectivity more reliable in the long run.
So, do we really want the end customers to run BGP? Probably not. We should actually, yes, leave it to the experts, people like that. Are you all asleep? This is basically what happens if you go to a RIPE meeting in some place where they drink some funny duck beer and the morning after the RIPE meeting in Dublin, I actually walked past this place and what was that? So yeah. Anybody else seen that? You missed something.
So, so, what can we do? The easy way to do it, and this may actually serve the job in a number of cases with IPv6, we have our two ISPs with different ASes and we use PA address space for the end site, deploy both prefixes in parallel, so every machine has two prefixes or every interface has two prefixes and two addresses at least, plus unique local but link local plus whatever else they use, and we get redundant connectivity of sorts. Or is it? Well, actually, not, because the key about redundancy is what happens if one side fails. I mean, in aviation everything is redundant, you know, so that is why aircraft typically have two wings. Now, what happens if one ISP gives us a problem? We have three different cases to consider. The first one is the ISPA is down and the client from outside wants to connect to our server at the end site. This can be a local web shop, it can be a VPN gateway which is probably quite common with many of these kinds of customers, can be whatever. Now, when this client tries to connect, what will happen? It will do a DNS lookup, hopefully both addresses are entered in the DNS and then it will try the first address it decides usable, in RFC 3484, 3484 bis or whatever, some sunny vegetable vendors have actually implemented.
So what happens? We try to connect, of course, through the broken ISP, because otherwise it will work, that would be a theoretical case if it worked right away. We hopefully get destination unreachable, try the next one, next address we get from the DNS and then connection would run through the working ISP. So we get the SYN packet through, traffic going back is a bit troublesome. We have to do some sort of failover mechanism, deciding which ISP to use for an upstream so we have to do policy routing here which is a pain but OK, otherwise we could convince ISPs to actually route source addresses from their competitor through ?? lots and lots of negotiation involved.
So, second thing we can do ?? we should consider, we have a client at the end site and one connector serve. How does that work? The client obviously uses the yellow address because otherwise things would work right away and that never happens, in reality. So, we use ?? we have to make sure that the client will use the proper address for the first up on packet. In reasonably small set?ups for reasonable definitions of reasonable, what we have to do is make sure as soon as our connection to the ISPA or whatever fails, that we deprecate the yellow address prefix, not trivial if there are internal routers involved but in a small scale environment with very limited number of subnets, that is ?? that can be pretty straightforward, really depends on the sort of set?up and CPE you actually use. Now, so we have to convince the client, please don't use the yellow prefix any longer. And then dynamic routing should take care that we send our packets out through the non?working up link.
Last problem is existing connections, doesn't matter which side serve our client, once the connection is established it works the same. When that happens of course, we are using the yellow prefix for the peer at the end site, so our connection breaks down. We have a couple of options here, we can decide, OK, who cares about it, just press "reload" and things are fine again. That is basically what Gert said yesterday. In a number of cases that works, it can be a bit of a pain if you do remote back?ups and your backup stream breaks down. There are a couple of ways to deal with the situation.
The immediate one is to make the second still functional ISP temporarily announce the yellow prefix from that end site through the BGP. I have got a degree in theoretical computer science so I am allowed to come up with this idea. It doesn't really make much sense. The alternative way, could set up redundant links between ISPs, that won't help us if the yellow ISP breaks down completely or gets depeered or whatever, but that happens fairly reasonably. Normally, it's just somebody who is reasonably comfortable working with an excavator, digs out some funny tree roots in the middle of nowhere and they are all shiny at the end and oops, that sort of thing. What we can do is set up redundant links to both ISPs. In some cases, that is feasible if 3G is enough bandwidth for the sort of purpose, but can be expensive, we need more bandwidth.
There is one last way, and that is basically what the IETF came up with some, I don't know, ten years ago, when they said we don't need to do PI space any more, we can replace these additional fall back links with tunnels running through the opposite ISP. When you try to set that up it is sort of scary at first. If you approach it like this with these steps I have shown, it's not as bad as it looks. What you want to do ?? now, this is the simple most case, I have a single tunnel end point at the end site, it's probably not a good idea. What you want is to have two different ones because that allows you, as an ISP, to manage the tunnel end point at the customer site and they just know OK, we have this box and it has a phone number on it, if anything goes wrong that is where we call and they will fix it as opposed to running BGP by yourself. So, this is the last option you have. The reason why I showed this is because I think from what I have seen in my trainings, that there is increasing number of people who sort of need this kind of functionality. It doesn't have to be done that way, if you have better ideas, fine, just make sure you send them to the v6 Ops mailing list or ?? so other people can benefit from it as well. The problem is, it is solvable, you can make money out of it, you can offer it as a product, so next time I put on a suit and talk to management about this rather than you. And I hope you got an idea that, yes, there is something we can do, we can't really do with IPv4, like this.
In case you have questions, want me to hire ?? expand on this for a bit of time, that is where you can reach me. Questions?
AUDIENCE SPEAKER: We are doing something that solves this problem and it's much simpler than any of the solutions. Basically, what we are doing is that we are getting a PI block, provider independent, for the end user, but we are not running BGP to the end user, we are just originating the prefix for them and so the ISP B. So, they don't have to learn BGP but they have stable addressing, and if the link our ?? downstream interface to the customer goes down the route disappears from their IGP and we immediately stop announcing it to the world and so does ISP B, so if they want to force the traffic, the customer can just shut the interface and it will automatically just converge. And we are doing this in v4 as well, and have been doing it in v6 for several years now, and it's simple and it actually works.
BENEDIKT STOCKEBRAND: Yes, but if you do that for 500,000 customers, that boils down to 500,000 extra routes in the fault?free zone. That is where it gets painful. Other people are going to pay the bill for this because they have to pay for the extra memory.
AUDIENCE SPEAKER: This is what we have with IPv4 PI as well and that is what this community has decided, it's what we are going to allow.
BENEDIKT STOCKEBRAND: Correct. But for I don't know how long people have been talking about, OK, the default free zone, the routing tables, are getting bigger and bigger and they are getting bigger faster than we can buy extra memory or bigger routers and with an increased amount in highly reliable network connectivity, I suppose that is a dead end, so yes, for now, you are right, but I don't expect this to work in the long run.
AUDIENCE SPEAKER: I would like to point out that it shows the IPv6 PI graphs at every address policy meeting we are doing and there is no explosion of IPv6 PI.
BENEDIKT STOCKEBRAND: Because too few people actually using it but once people use it as much as IPv4 we are probably going to see that.
MARCO HOGEWONING: I am going just ?? you clearly don't agree. And we are a bit bound for time. So let's ?? please continue this, if you want to take this to the mailing list, then fight about the number.
MIKAEL ABRAHAMSSON: So, the small way here, I like you said smart, I have been doing that for a couple of years myself with two up links, it works really well. We need something automatic to make this work and can people can offer this, it's basically an IPv6 tunnel broker thingee. You could have it as a third party as well, doesn't have to be your upstream ISP, could be anyone. Then I would would also like to see end hosts doing the intelligent stuff like multipath TCP or SHIM6, whatever, those kind of mechanisms, it would have been good if you brought that up as well because I think, long?term, that is the way we need to go in order for this to work without cludges.
BENEDIKT STOCKEBRAND: Maybe. Maybe not. We will see about SHIM6, basically it's a probably with locator identifier.
MIKAEL ABRAHAMSSON: That kind of mechanism where the end host has multiple addresses for ISPs and it can then choose, that is the only way this is going to scale long?term.
GERT DORING: Well, we have discussed this but for the sake of the audience, to bring another point of view; it would work and it's good to save global routing slots so there is definitely some benefit to it. I think it's overdoing it because what you gain from it and that is worth a goal is session surviability if one of the ISPs goes down. I have been convinced that most sessions for standard end users, not the folks in this room, are so short if one of the ISPs goes down they just click reload in their web browser and that is it, problem solved. So, I think the market for something as complex as that is pretty small. But at the same time, we need to be working on teaching applications to handle multiple sources and failing of one source address better because that is right now pretty dark matter at the IETF. I tried this source address, it timed out, no fall over to another source address. This is not specified at all.
MARCO HOGEWONING: By the way when I asked Benedikt the question, I was more in behind small or medium businesses than actually end users being fully redundant, but obviously if you fit your house with sensors, it might become beneficial during this at a Homenet.
BENEDIKT STOCKEBRAND: Yes, we should actually mention when we talk about it, we ?? it should be afterwards, not before. We sort ?? we are talking different things there at that time. Your concern was that in some countries where there is some sort of national firewall sort of thing, maintaining a session might actually be helpful. That sounds, well, politically incorrect or correct or whatever, but depending what country you are coming from. It actually helps to maintain session goes with the same provider rather than going this way and that way and you can't filter any more so even that might be a business case, not so much for the national firewalls but maybe because you have customers who want you to run the firewalls for them which is another kind of business that is actually important to some people.
MARCO HOGEWONING: Thank you, Benedikt. Thank you for stepping up and we would love to see you in a suit next time.
(Applause)
So next up Jen Linkova from Google, who did some research in source addresses. Are you cool with 15 minutes.
JEN LINKOVA: I will try to. I can do it, it will be 25 slides per second. Easily. Address selections, we are monitoring, trying to find out have v6 connection doesn't work, at one point I was curious where those packets might be dropped. I was thinking maybe those packets successfully travel through the Internet and dropped, what I did, twice, because I couldn't trust myself, I did measurement, I logged all packets from invalid ?? v6 IPs, coming from big Internet to Google network so this is not about Google, I just use my network at a kind of tool to look at what is going on globally.
And I collected data for a couple of days so you cannot digitally compare the numbers I am talking about because it was not exactly the same number of hours and minutes I run the measurements but we can compare percentage.
So, two years ago I collected about one million packets, 32K of unique IPs and about ten times more this year.
So, I look at what kind of source addresses I come into my network, it's actually boring so most of them are just 6to4 Marshals ?? plenty of ULA, this picture is useless because you cannot see much in the most interesting part of it.
So let's zoom in. Yeah, people using the documentation, using examples from documentation and just copy and paste those and in their configuration. Presently, I did not see much from 2001 call DB80 which could I see in some documentation examples. So I am going to briefly look at each type of bug and source observed.
Traffic profile, in most cases it's end users trying to reach Google TCP, and now I can see some UDP traffic related to DNS, so sometimes probably a martian cannot resolve, and I still see in some country, I see ICMP packets, and users keep ?? google.com. Unfortunately, from the wrong source address. And I also see some number of packets related to infrastructure, I mean, I do see decreased number of time exceeded packets which I think is good. I don't have exact explanations but I think it might be related to the fact that most of tunneling is going away and packets are not dieing quietly in the network. I do see packet too big messages coming from the wrong ?? which is pretty bad because it means that source could not adjust MTU accordingly so it means some v6 users do suffer from MTU issues because could not receive those packet too big messages and the destination unreachable.
And I have seen one very interesting case of ICMP blocks blocked and I am going to talk about it a little bit later.
Most interesting case: Link local packets coming to my packet from Internet, which is most surprising, things can be broken in many ways.
So I looked and the good thing is number of packets decreasing over time. And by the way, I just ?? I realised recently in 2011 I did my measurement before v6 launch. It means that I did see less traffic than I see now. So if my numbers showing that something increasing it might be just the result of v6 launch but if I see decrease and some types of traffic it means that we actually see less and less of other traffic.
So, looks like vendors are fixing their software bugs and people upgrading their routers. Because two years ago it was about 24 different vendors, now just 18 and I wasn't able to define just one vendor this time. And most of those addresses are based on Mac 48 which gives me ability to identify the vendor.
Good. Good news.
But another good news, yes, plenty of traffic is from end user machines but there is traffic from network vendors and I am pretty sure I saw some people from those vendors in the room. But for not TCP traffic two years ago packets too big, time exceeded, now I see only two routers in the whole Internet and those two routers for the last two years constantly keep sending to our front end neighbour discovery reject. Which is very interesting because that is a special case when router really could send you packet from link local source to the global destination. It's permitted. But that destination must be on link for that router. But what I am observing, their two routers somewhere in Europe, by the way, from two different routers. Still sending to our Google front ends reject. Now after two years I am curious enough and I am going actually to capture those packets and look and find those boxes. Because I really want to know why they are doing that.
So, what ?? again, neighbour discovery reject is not a big problem because I would expect regional packets to still come through hopefully. But it's actually disappointing that looks like most of the Internet routers just ignoring scope address architecture and forward packets leaving the zone of the source address, basically crossing the boundary of the scope. Normally, such packets should be dropped but I am pretty sure none of the platforms I am aware of does that.
Another type of quite popular incorrect source address are ULAs.
I looked at them and I found quite a lot of people do understand that they shouldn't be using FC block and they should be using FD. There are some people who don't know the difference and apparently yes IPv6 is very hard, we can count up to 16, not everyone knows that FC 00 it's not the same as FC and FC 01. I am pretty sure you all know that but just in case please, pass the message. So presently, most of those packets are coming from ?? most of the addresses I am seeing privacy extensions, so very few of them actually based on Mac 48 so I think it's ?? and most of the traffic is TCP so it might be end user machines. So, I was looking at how random ?? how unique are those addresses, because I know people are lazy. And it's actually not obvious to me how to automatically detect randomness, so I decided to be lazy as well and I will just looking at the pretty simple cases when only numbers only letters are used and if prefix contains three or less different characters. And yes, I did some spot?check of the result, it looks pretty reasonable, people do like choosing FC 00, FD 00 and all that stuff. And the good thing is, people now using more random prefixes and I definitely see decrease in the number of non?ram come to prefixes in the packets I am receiving.
Surprisingly, ten years ago, site local addresses were deprecated. Probably not everyone still aware of it. Because I see more or less constant rate of traffic from site local prefixes and in this case it's not TCP mostly; I can see plenty of ICMP traffic like time exceeded, destination unreachable, so my explanation is those addresses are somewhere on infrastructure links. If you know someone who is in site local, please tell them it's the time to renumber.
Long, long time ago in a galaxy far away there was a backbone called 6 bone, which was used here for I should say testing IPv6, and I was very surprised to see packets from 6 bone, which was decommissioned a while ago, come into my network. When I looked closely I found most of those are not from 6 bone prefixes but from /32 prefix which was used by Teredo until, I don't remember , 2006 or 8 when Microsoft changed the prefix using for Teredo. So, now, I can more or less ? number of ?? with v6 connectivity in the world. And surprisingly, all those machines are just trying to pingle google.com. I think it's a kind of stone age monitoring system, which will not work. And the server ?? actually, 8 unique v6 addresses coming from 6 bone address block. And it's actually machines trying to open and brows google.com. Again, I don't know where are those people and why nobody told them that this address space, it's now should not be used.
OK. Another interesting case v6 mapped, those addresses should not be on the wire. It's just for API to denote v4 address and I still see constant amount of it. CP traffic from v6 and even worse, IPv4 compatable which deprecated for seven years and I am actually again seeing even more traffic from those address space.
But in this case it's mostly destination unreachable, looks like, again, someone is using it for routers.
Probably some people should devise again address policy and how they allocate IP address for their machines. And actually, I think I am doing even faster than I expected, it's almost the last slide, I saw some packets from loop back which is understandable, if machine doesn't have any other v6 address we can use the only one available, surprisingly enough I see some packets coming from unspecified address so some devices want to reach google.com over v6 and they don't have any other address. What I can't explain is I am seeing some traffic where subnet is all zeros but interface ID is 64 non?random bits, and it's not based on a Mac 48 so I cannot identify them, so I am not aware of any cases when host could choose some address. If anyone sees anything like that, please let me know.
What I did not see, I might expect almost any kind of brokenness but I did not see any multicast, good. Very, very little traffic from legitimate but non?allocated blogs but people do like prefixes like all As or Bs or something like that.
And to summarise it, yes, we heard yesterday that we are all doing great job in deploying v6 but there are some broken things, and at least some kind of brokenness is slowly going away so eventually we will be there and devices will be able to properly choose source v6 address and I still have no explanation for some kind of traffic I am seeing so I appreciate any ideas and comments, and yes, indeed, people who write the code or design the code probably not pay enough attention to things like scope address architecture and we have just seen illustration on how bad the situation with BCP 38 because all those packets should be blocked pretty close to its source. And that is actually ?? that is it for me. I am on time? Yes.
MARCO HOGEWONING: So let me just queue and ask the first question. On the site local what do you think, is this old because it's really deprecate add long time or old installations or old books
JEN LINKOVA: Some people probably choose them they are not aware of deprecated and look for private addresses for v6 and did not pay enough tension to differences between U L and site local. It's my explanation. Because again, going back to my site local slides, I am seeing plenty of ICMP packets, not just machines, probably some people were using them for infrastructure links. I don't know. Maybe there are some other explanations.
MARCO HOGEWONING: Might be a point for people doing IPv6 training to spend more time on this.
BENEDIKT STOCKEBRAND: Now, this is a bit of a wide shot but there is possibly some explanation for some of the really weird stuff. I have had a couple of customers or trainings for software developers on IPv6 and I remember at least two cases where people took me aside and said, OK, don't tell the boss, we have added IPv6 support as far as we could but we never got a chance to test it so there is a good chance there is some sort of software out there that was tested with IPv4 only, the developers sort of tried to implement IPv6 in it but obviously screwed up because they could never test it. It might be interesting to see if these completely weird stuff changes or if it stays relatively constant.
JEN LINKOVA: You mean this one, yes?
BENEDIKT STOCKEBRAND: For example, or the all As and all that sort of stuff.
JEN LINKOVA: For random stuff, I can check the numbers yeah, because I have all my data on my laptop, I can compare two years and see the difference, yes. I can look into it, thank you.
BENEDIKT STOCKEBRAND: That might be kind of interesting because I guess there are more people who tried to get IPv6 up and running when they didn't know what they were doing and the software is still throughout in the wild for IPv4 only.
MARCO HOGEWONING: Thank you.
Following up on your comment, would it be worth to cooperate with, for instance, the MAT Working Group and see if we can expand this type of measurement into something that is long running, I think that was Benedikt's suggestion?
JEN LINKOVA: I am actually going to have a discussion with our security group because I basically did firewall filtering and just collected all the thing. It will be interesting to extend and see how it changes over time.
MARCO HOGEWONING: Maybe also see different pick?up points apart from google.com, see if we can grab it together. We would love to see more and as Benedikt says, see how it changes over time. Well, thank you, Jen, unless there are any last comments.
(Applause)
Yes, please do filter BGP 38 also applies to IPv6.
Next up, Tassos, and Forthnet, explaining a bit on their IPv6 project and remember they switch IPv6 on their website overnight this week. Thank you for that.
TASSOS CHATZITHOMAOGLOU: I want to thank Lee Howard for bringing that to our attention. He gave us a challenge. But it was much easier than we thought, just a single slide on how we did it. This is high level design ?? so we had IPv6 already in our routers, we have done that two years ago so it was not something we had to do yesterday. We had already IPv6 enabled firewalls because we replaced our end of life firewalls this summer so we had that already ready. And we had load passes, upgraded to support IPv6 so we also had that. We had some services running over IPv6 so we had send our mail over IPv6 and platform to respond over IPv6 because we are going to IPv6 only network using DS?Lite mostly so almost everything ready and the only things we had to do were those four, trying to find some IPv6 prefixes and configure those and update the filters and then we had IPv6 on our corporate site.
So now, something about the presentation. Some statistics we collect from our network. This is how some ?? RIPE counts for something ?? APNIC, probably use the same data. W6L gives us little lower ?? this is our own metrics so the purple line shows the subscribers, percentage that have actual IPv6 addresses but the red lines shows the subscribers that are actually using IPv6, and we have almost ?? we had around 10 percent of subscribers using IPv6. This amount of subscribers increased as time passed by so last week we reached around 23 of subscribers. But we missed something, so actually only 8% of them were able to actually do it. We missed a setting on one of our CPEs, so we found that and we are running upgrade so soon we are going to do something like that, hopefully next year we are going to reach and have the same, both lines following the same pattern.
Some statistics about IPv6 traffic versus IPv4 traffic. We took two measurements, the first one was in June when we first enabled IPv6 so this was the percentage of users using percentage of IPv6 traffic versus 4 traffic. This was the download of traffic ?? falling like similar patterns. We see that there is a percentage of users around 10%, between 10 and 15% using more than IPv6 than 4 which is good. We took some measurements the previous week, so this is the up load now, so is got bit better, the line moved to the left and up. This is and ?? this is download so it's also better and in general we have around 20% of our subscribers using more IPv6 than IPv4 and around 15 of our subscribers using more IPv6 upload than v4.
This is the bandwidth users, this is the bandwidth ?? we have almost 2% of traffic of our total Internet traffic being delivered over 2% and the major increase became ?? was in June where we started enabling, then we had a decline because of the summer months and then we start increasing again and we will keep that increasing and since we expect to also fix the issues to reach around 6% during next year.
Some statistics about CPE, we have five in our network, all support dual stack and we had an average of 6 firmwares in order to make dual stack fully compatable with our own needs ?? fully support DS?Lite, fixing every bug found and we currently have only one CPE with PCP, until now we have five firmwares and keep excepting new ones so it will take a while.
Most common problems found on the CPEs, we have many cases with prefix was not working or not enabled by default. DNS info being passed to clients was not something ?? we had CPE passing link local addresses ?? who has CPEs were passing the global one address over the CPE as DNS server to the land lines and who were passing the DNS servers so we tried to standardise that and we are still working on that.
We had some cases with MTU because a lot of vendors preferred to be the safe side so they loaned and they are not using the best possible so we tried to somehow persuade them to increase that.
So DS?Lite was the decision we took because we had, we were running out of IPv4 addresses sometime in the past. So before going into that step we took another step, we moved from this decentralised to centralised in order to gain some IPv4 addresses. This is a summary of the process we did in order to move over. So we actually took only the what we call overflow BRAAS desizes that have some ?? some v4 addresses are not being used so we abandoned all of them on TCP server, managed to aggregate a lot of IP 4 address and gave us an extension. This was just ?? a measure in order to gain some months. We had to decide what was the next step and as you may already know, we took that decision some years ago, we are already running DS?Lite but that is comparison we did with cost of the solutions we checked at the time, the table on the left shows some parameters we took into calculations. We considered that we have served most of them so we have ?? so we focused mostly on the cost and bandwidth and characteristics of the service guys that use ?? we put in some calculations and we found that there is no way you are going to buy v4 addresses because we increase our subscribers, you are going to pay more and if the price increases then we are going to have much bigger issues. The red line shows the cost of buying IPv4 addresses. On the left you can see the actual cost, so it goes very high, and the horizontal line shows the months. We had to choose between two solutions:
Not use IPv6 and stay on v4 so we had to do ?? to the solution that use IPv6, overload some traffic IPv6, at the same time do something with v4 traffic. And we did a comparison with those solutions and we found that after one year, the cost of investing over 4 solution gets much higher because at the same time we have IPv6 traffic, so while you use the same device, the same actual router to do the CGN you have less traffic passing through in case of DS?Lite so as the IPv6 traffic increases, your IPv4 traffic needs get lower and this is the ?? after one ?? one year?and?a?half and you start having all the advantages.
DS?Lite was the neck following Forthnet chose to move, this decision was taken almost two years ago when we start testing some versions. DS?Lite is a combination of two technologies, actually IPv4 and IPv6 tunneling and ?? translation. We saw that it was, technologies that being used were already all ?? the only thing we had to do is combine them. There is one and we still believe it's the easiest one in terms of provisioning which you just need single address to configure and everything is ready and the most important you use IPv6 because we wanted to go to IPv6.
This is something to consider, we don't know know for how many years use technology but as long as we have idea in the Internet, we will use something and we have a pilot running since 1st of April and soon will make it production, we have solved our issues so there is nothing left there.
DS?Lite, just a brief description of technology, I am going to skip most of the things. We actually have our IP host, we have our CPE which is able to do DS?Lite, called v4 ?? the network which transparent so it doesn't matter and we have our AFTR which is centre point where all tunnels are terminated and IPv4 Internet, what is actually happening when the IPv4 host wants to connect an IPv4 server with someone on the Internet it has to do some things. First, we have software, then we get the packet sent, nothing different from the current implementation we use, we have in our IPv4, the only thing that CPE does, it's an IPv6 ?? it's actually tunnels the packet so nothing more that. The AFTR removes IPv6 header so only IPv4 is in and it gets to the translation. This translation is the same that should have been done by the CPE if it had centralise the router. The packet arrives on the server, nothing strange here, server response and on the opposite way the AFTR makes the translation and encapsulates the packet and it arrives at the host. Very simple solution, it works fine, no issues there.
What we had as a problem is that we need ?? we had to solve the logging problem because we need to keep track of every session logged, so this is the log parameters we already use before tabling DNS LITE but having dual stack. So we had our user logging parameter, we have our IPv4 addresses and networks, then if we add the two parameters one of ?? which is used in dual stack. The one that characterise the subscriber is the one with IPv6 prefix is the one we use in correlation, between this and this. This has the new logging parameters that are being produced by the AFTR. The ones that we actually use, someone else who will probably have to do another solution with CGN might have more parameters. So we actually had to load the subscriber and identifier which was actually the software address which is made of the one IPv6 prefix so we have ?? then, we had to ?? ex personal POP ?? the ICMP identifier and protocol identifier. We did some evaluations and we found two protocols were able to do that. IP FIX and Syslog. IP FIX it's quite efficient, you have to the ability to make a lot of changes on the customer fields being logged, so it's nice. But it has disadvantage that you need the collector to, a special collector to get the data and need to translate them from something text. Syslog has already that already so you get reachability by, but that means you get reduced performance especially on the router size and much larger volume. There is a solution that, it's a solution that could combine both of those the IP FIX, you get the significant log processor that and IP FIX to Syslog, so you have the best of both worlds.
This is some of the logs we get when we have DS?Lite sessions. On the left side you see the original message and you see what we need to log, the original value and the final values, the actual value we need to log.
So we take all the surges we made some calculations and we found subscribers in our network almost 27,000 connections per day, well a bit torrent user makes the same connections. This list to us to assist ?? has 32,000 subscribers, we need around one terra bite and a half of storage per month in order to support all this scenarios. This assumes no compression and no other stuff. So it's not very big but ?? for our network size it's considered very big. So we had to move another solution. So what we did, we moved to the so?called bulk allocation, so instead of logging every connection you actually log connections per port rates so on the left you can see the connection that logs a range of 1084 ports so one single connection, one single log for 1,000 connections. We did the same measurements, the actual number of bytes is this time a little bit longer because we instead of logging ports we logged a range of ports. But the main difference is here because we now count connections and we now count them in side blocks. So instead of having thousand connections we only have 60 block allocations per day and the bit torrent user actually makes 8 per day so it's the same, not much difference. We did the same calculations and ended up with only three?and?a?half gigabytes per month. No compression, no optimisation, PCP entries as you can see there is a entry there, a single entry that is almost static. You don't have these things changing, so it doesn't make any difference. We set it to go with this approach.
Next steps regarding this DS?Lite stuff, we also taking another solution that we probably ?? use the logs. We are thinking of moving to lightweight as soon as we have a running implementation and there might be some interest for other solutions like map but unlikely from the current answer we have got from the vendors now, we don't see any interest from vendors so we have to wait a little more.
PCP, I have to make it a little bit faster. So PCP port control protocol allows us to solve the problem with DS?Lite, we cannot have incoming connections, it has ?? you can a client proxy, main objective is to for applications to work. Just to get a very brief idea of how PCP works. So the client sends a PCP map request to the PCP server which actually is AFTR also and a specific port for a specific period of time, this time for run?out. If the AFTR has the same port it returns the port. So the CPE knows incoming connections and the client from the other side can initiate using DS?Lite and passing through the hole that is created and reach the PCP. The same happens when you have UPnP also, originate the from the client side, triggers the connection, the message gets relayed to PCP but this time there is not the actual port available so PCP returns another port, that is not an issue and the client on the other side connects to the Newport and on AFTR.
We had to create a custom page on our so this is the place we create in order to identify. And some basic stats. We created a web server on the entry so the PCP table include that entry, the same port was asked and given, so that is easier case. So we get the connection from the outside and web server is answering. Then we create another entry for another port, this time we got random port, didn't go to the one we wanted so have to use that port in our connection but still everything works, also connection to second web serve. This was with PCP, we haven't seen anyone implementing the RFC PCP, they have implemented drafts although the RFC is still out, there might be some issues there. No ?? we haven't found any known apps supporting PCP so we don't have functionality for PC ?? some CPE ?? some vendors' CPEs said they would be using prefer failure which we don't like and we still have issues with lifetimes and time outs but this is going to be solved. There is a lot of stuff to be done, nevertheless it is we have work implementation and we are happy with that. And the final slide, this is a time plan, since we started back in 2008, so we are now at 2013, next step is put DS?Lite in PCP protection, hopefully we will have 30% of subscribers use IPv6 next year and don't know what we will be doing in 2015. Thank you, if you have any questions.
(Applause)
CHAIR: We have time for one question and that already came in off Jabber.
AUDIENCE SPEAKER: Have you experienced any issues with path MTU discovery on IPv4 when you run that over DS?Lite?
TASSOS CHATZITHOMAOGLOU: We had a lot of issues until we solved the issue with IPv4 ?? initially when we started testing that was one year ago, that was resolved and then every had their own MTU so we had a lot of issues there, we fix that had.
AUDIENCE SPEAKER: So TCP.
CHAIR: Final question.
AUDIENCE SPEAKER: Hi, Rumy from RIPE NCC. I am relaying a combined question from Ole and Mark Townsley from Cisco. Their question ?? I am trying to speak fast and combine several things in one ?? is they didn't really understand the point of why DS?Lite was cheaper than NAT 444 both having centralised CGN with similar amount of state and they were wondering if you calculated the cost of moving to net state to the CPE as in running A plus B?
TASSOS CHATZITHOMAOGLOU: For the second one we haven't done that until now but we are evaluating that. For the first one, as IPv6 traffic increases, we have lower traffic from the CGN so instead of having the host subscriber traffic passing through the CGN in 444 solution we only pass the actual IPv4 traffic so when in the future we reach 50% of IPv6 traffic the CGN load will be only 50% because half of traffic will be outside the CGN. So, as time passes by it becomes cheaper.
AUDIENCE SPEAKER: But it's the same with dual stack?
TASSOS CHATZITHOMAOGLOU: Yes. But we don't have any cost on the dual stack side.
CHAIR: Thank you very much. We very much appreciate your presentation, of course. It's getting time, we are getting close to the end.
Our final topic is feedback from the people who felt lucky and know better, which are also the people who tried out the IPv6 only network and Marco is going to talk about that and guide a discussion.
MARCO HOGEWONING: I am going to try to make this lightning talk to not take too much of your coffee break. IPv6 only network, for a reference of you who were in ENOG in Moscow or St. Petersburg earlier this year, you saw this slide. We figured out when the first IPv6 network was available at RIPE meeting, it was RIPE 37 in September 2000. 13 years in the making, time for the next step, we thought. So we ran an IPv6?only network, we won't give you IPv4.
It was not supported by the RIPE NCC technical team, it was all voluntary effort and it was all best effort.
Acknowledgements. Thanks to Cisco for their gear and manpower and making this work and it came in the form of Andrew Yourtchenko who did a lot in piecing this together and getting it up and running, while the RIPE operations team didn't work it, yes, they did, without them it would be completely impossible so thanks to Marco and all his colleagues, men know, everybody who helped out in getting this to run.
How we did it: We had a sort of wholesale model. RIPE NCC provided us with SSID that was running on the normal frequency map you have got and delivered all the traffic to us where we pumped it into a Cisco AS RI K, which we borrowed and had an uplink to the RIPE ?? Juniper in between. The uplink to the Juniper was dual?stacked, the site towards the wireless was entirely IPv6?only and since we were running net 64, we also needed the DNS server, so we bought a really fancy server in form of old MAT book running some virtual servers and that was all dual?stacked. So it was really very simple, and very straightforward set up, all wires no VLANs or whatever complicating things.
Well, so we switched it on on Sunday, this was the first message my Mac gave me, it's designed to do that, I already ?? it was yellow so it wasn't really happy but slowly we make stuff work. Some of the statistics: Well, traffic not that much, a few megabits, we saw some one big spike, I hope we broke your limit and not us but we saw traffic spike of 24 megabits. Clients, it's a big small but this is a graph we took on Tuesday afternoon and we saw about 47, 30, 40 clients on?line all the time, five and five gigahertz. Well I know the average here is about 700 devices so I think about 5% of you have tried this.
Incidents: Well, there was one failure, Monday afternoon, just after we announced it in the plenary. That was due to a faulty patch cable falling out of our laptop serve and without DNS and everything, shut down. So we had to rush up ?? Andrew had to rush upstairs to replug that.
And other than that, we tried to take recommendations from Brian and imprecise network monitoring. So a lot of stuff happened to Twitter and that was using NAT 6?4. We had this hash tag and sons of comments there and people explaining what they were doing so thanks for that.
You might want also to try v6 monitoring so we do Facebook, they do v6, unless you have a iPad it really says you don't have an Internet connection, thank you, bye bye. So Facebook supports v6 on their website but not on their apps. We had the mailing list.
Well, unfortunately, unless you have an iPad because sorry we can't get your mail over a v6 only network. We did get a lot of feedback and of course we had over ways of reading our e?mail.
So some of the observations that we saw and people relayed to us either in person or via mailing list and one of the first somebody said, oh, I am trying to re?set my Cisco password and it gave the wonderful failure called IP 101, I love the creativity on the name there. Since we have a Cisco employee here I will leave him to explain what happened.
ANDREW YOURTCHENO: So this actually is not specific to IPv6 only, and this is actually the reason why you should enable IPv6 on your website now rather than one year later. Because that was something that slipped past the testing as we were dual stacking bunch of back end servers a month or so ago and because the percentage of users is realtively low, because we discovered it after all the paperwork has been done and everything has been lit up, so we decided to keep IPv6 on the server in spite of this, and you can blame me partially for this slip because I was one of the testers so sorry for that. The good news is that we are working on this and hopefully tomorrow maybe I will more news but I don't have right now. So yes, if you have the need to change the password, please use the legacy IP for that. Sorry for that.
So, DNS issues, so still a lot of people use the operating systems that relatively use them, that don't support getting DNS, either by stateless DH VP v4 or RD NA which R I K supports as well, you had to do the manual configuration which is not a funny thing to do especially with IPv6 addresses. Speed test. That was very interesting thing that I did on Sunday, and apparently IPv4 over IPv6 is still faster than IPv6. The good news that it was isolated to a specific serve which by the way is Netherlands so I think there is still some work to do there.
Arahav manager, it's a bit small font, basically all this small blue is unknown operating system. So, the detection mechanisms in the management systems still need to catch up a little bit. Then interest thing is the total number of clients when ?? as measured as a sum of 2.4 and 5 gig clients is bigger than the number of clients as measured by the operating system detection. Well, normally that should be the same number, right? So you have clients by operating system and clients by radio, that should add up so it doesn't, so some of the clients end up somewhere in limbo. So it probably was the people who switched off IPv4 completely and then the state mission doesn't know yet how to deal with it.
How many people use only IPv6 during this conference? So that is about 30ish addresses that we saw in the conference. I was one of them as well. Thank you.
And with that, the question, so how was it, should we repeat that, would you try this in your own network?
MARCO HOGEWONING: Any questions or comments?
PATRIK FALSTROM: I was running the wireless network at a social media conference in Sweden that people that thought the Internet was Facebook and Twitter. And I decided to run an IPv6?only network there with about 500 people and they absolutely do not know how to touch the network settings, don't know the terminal window but. It was still very interesting and they were very thankful and they asked me to rerun that test of IPv6 only. Of course they then understood that the Internet was actually only half of twister and Facebook because Twitter didn't have any IPv6 very good experience, it was good experience for me to run it in a network where people did not know anything about IPv6 or networking so I would say repeat this, very good. Thank you.
MARCO HOGEWONING: Thank you.
AUDIENCE SPEAKER: Jen. Thank you very much. It was very interesting experience, we definitely should do it again. And I cannot say I was using only IPv6 network because I switched to v4 twice, first to change my Cisco password and my VPN but I was using it most of the time and it's good we can test all applications besides Facebook and Twitter.
MARCO HOGEWONING: I cheated because our corporate VPN while working over IPv6 only became quite unstable so I switched back to the regular production network to keep accessing my e?mail.
GERT DORING: Just a minor correction, it's not the first time we did this, we did this in Berlin with good success and the feedback was let's do it again so let's keep it, let's have it as a regular service on every damn RIPE meeting.
MARCO HOGEWONING: We will relay this message also to the meeting organisation, to see what they can do. As a reminder I was there in Berlin and we ended up without any network where we switched.
GERT DORING: Actually, it worked much better than Berlin this time so it was appreciated. So you did a very good job. But let's say a learning curve, Berlin was first experiment, now we know how to do it.
AUDIENCE SPEAKER: Just one remark about the Net 6?4 prefix, you use the documentation prefix, why?
ANDREW YOURTCHENO: Blame my laziness. So I used the well?known prefix in some cases where the addresses were RFC 1918 and as you know the standard prohibits the use of the well?known prefix so in that instance I quickly hacked in to ?? it was just a bad modern memory ?? so for various reasons I do promise ??
AUDIENCE SPEAKER: It's about time ?? recommendation get sent to Google.
ANDREW YOURTCHENCO: That doesn't get sent.
MARCO HOGEWONING: One final question: Who here discovered something that didn't work and went back and fixed it? We have one. OK. Cool. So that was part of the purpose of this experiment was find out what doesn't work and fix it. And I will leave it to you David.
DAVID KESSENS: Thank you very much for being here. This brings us to the end of the agenda, very important again the PC asked us again to say there will be the nominations process will close by 3 p.m. today, so please if you want to nominate yourself or somebody else do that by that time and the elections will actually take place on Friday. This is the end. We will see you back in Warsaw. In the meantime, please spend lots of time on IPv6 and we ?? IPv6 only network.
MARCO HOGEWONING: And we are very broad scope, everything to do with IPv6 is welcome on the mailing list. It doesn't have to involve policy.
(Applause)
