Hello!
On 04/12/2012 05:17 PM, Gioacchino Mazzurco wrote:
You need a routing protocol between the gateways to do that, Batman creates
Wouldn't just need something like an announce mechanism so each node would add an L3 route to the gateway announcing the said prefix?
Since from layer 3, the batman mesh looks just like a big LAN, any node could simply add a route saying "prefix A via gateway B dev bat0" where B is a mesh node known to batman via its L2 identifier, no?
Jernej
Hi!
On Thu, Apr 12, 2012 at 5:37 PM, Jernej Kos k@jst.sm wrote:
Wouldn't just need something like an announce mechanism so each node would add an L3 route to the gateway announcing the said prefix?
One way could be arp proxy. So that border node would pretend to hold all IPs of subnet behind it and reply with its own MAC address for all those IPs. Once packets gets there is then routed behind. I have been playing with this once and it does work.
https://en.wikipedia.org/wiki/Proxy_ARP
Since from layer 3, the batman mesh looks just like a big LAN, any node could simply add a route saying "prefix A via gateway B dev bat0" where B is a mesh node known to batman via its L2 identifier, no?
So we would just need some other L3 routing protocol on top of batman-adv, without any need of knowing the wireless infrastructure, only to configure dynamically those announcements on all nodes, no? So that when gateway B appears, that all nodes add its prefix A to their L3 routing tables?
Mitar
Hi!
On Thu, Apr 12, 2012 at 5:37 PM, Jernej Kos k@jst.sm wrote:
Since from layer 3, the batman mesh looks just like a big LAN, any node could simply add a route saying "prefix A via gateway B dev bat0" where B is a mesh node known to batman via its L2 identifier, no?
So I have almost convinced Kostko that we could switch to Batman, but he does not want to run multiple routing daemons on nodes. So could Batman support announcing also L3 prefixes? For example, that in the node configuration you could define that that node (let say it has IP 10.10.10.10) should announce also some L3 subnet. Like 192.168.42.0/24. And when that node would be reachable from other node, that other node would run:
ip route add prefix 192.168.42.0/24 via gateway 10.10.10.10 dev bat0
Doable? Because to run additional routing daemon just to rediscover which nodes are around and announce things ... when Batman already knows all this ...
Mitar
You do not need a routing daemon on all nodes but just on the gateways, each gateway will talk to others and saying i am announcing that subnet with that cost, so when the gateway receive a packet for that subnet it forward the packet to the right router and tell the client "next time you need that subnet use the other router" with an icmp message
On 04/12/12 23:08, Mitar wrote:
Hi!
On Thu, Apr 12, 2012 at 5:37 PM, Jernej Kos k@jst.sm wrote:
Since from layer 3, the batman mesh looks just like a big LAN, any node could simply add a route saying "prefix A via gateway B dev bat0" where B is a mesh node known to batman via its L2 identifier, no?
So I have almost convinced Kostko that we could switch to Batman, but he does not want to run multiple routing daemons on nodes. So could Batman support announcing also L3 prefixes? For example, that in the node configuration you could define that that node (let say it has IP 10.10.10.10) should announce also some L3 subnet. Like 192.168.42.0/24. And when that node would be reachable from other node, that other node would run:
ip route add prefix 192.168.42.0/24 via gateway 10.10.10.10 dev bat0
Doable? Because to run additional routing daemon just to rediscover which nodes are around and announce things ... when Batman already knows all this ...
Mitar
On Thu, Apr 12, 2012 at 11:08:48PM +0200, Mitar wrote:
Hi!
On Thu, Apr 12, 2012 at 5:37 PM, Jernej Kos k@jst.sm wrote:
Since from layer 3, the batman mesh looks just like a big LAN, any node could simply add a route saying "prefix A via gateway B dev bat0" where B is a mesh node known to batman via its L2 identifier, no?
So I have almost convinced Kostko that we could switch to Batman, but he does not want to run multiple routing daemons on nodes. So could Batman support announcing also L3 prefixes?
A few points here:
1) You need, in your head, a clear separation between L3 and L2. Your L3 gateways need routes. Your L2 only nodes don't need routes, except the default route, to the gateway out of the current L2 mesh.
Just think about plain old Ethernet networking, and you won't go too far wrong. In an Ethernet based network, you have maybe 2 nodes which are gateways, and run L3 routing protocols. All the rest of the devices are L2 only and have a default route pointing it one of the two gateways out of the Ethernet to the rest of the world.
2) BATMAN is not a daemon. Its a kernel module. Hence it cannot play around with routes, even if it wanted to. Also, adding/removing routes is policy, and the Linux/Unix view of the world is that user space is responsible for deciding policy.
For example, that in the node configuration you could define that that node (let say it has IP 10.10.10.10) should announce also some L3 subnet. Like 192.168.42.0/24. And when that node would be reachable from other node, that other node would run:
ip route add prefix 192.168.42.0/24 via gateway 10.10.10.10 dev bat0
Doable? Because to run additional routing daemon just to rediscover which nodes are around and announce things ... when Batman already knows all this ...
Nope. BATMAN has no idea. That is a L3 address. BATMAN is L2, it only knows L2 Addresses. All it knows is that MAC address 02:01:02:03:04:05 can be reached by sending packets towards the next hop 02:06:07:08:09:0A.
Here is a rather boring example of that BATMAN knows, from a real device:
root@tmn4:~# batctl o [B.A.T.M.A.N. adv 2012.0.0, MainIF/MAC: br0/00:80:48:6b:9d:36 (bat0)] Originator last-seen (#/255) Nexthop [outgoingIF]: Potential nexthops ... 00:80:48:6b:9d:35 0.084s (255) 00:80:48:6b:9d:35 [ wlan0]: 00:80:48:6b:9d:3a (166) 00:80:48:6b:9d:35 (255) 00:80:48:6b:9d:3a 0.352s (216) 00:80:48:6b:9d:35 [ wlan0]: 00:80:48:6b:9d:35 (216) 00:80:48:6b:9d:3a (181)
So: In order to reach 00:80:48:6b:9d:35, send packets to 00:80:48:6b:9d:35 In order to reach 00:80:48:6b:9d:3a, send packets to 00:80:48:6b:9d:35
This comes back to:
You need, in your head, a clear separation between L3 and L2.
Andrew
Hi!
On Fri, Apr 13, 2012 at 7:59 AM, Andrew Lunn andrew@lunn.ch wrote:
This comes back to:
You need, in your head, a clear separation between L3 and L2.
:-)
OK, maybe I am getting a bit of hold on this. :-)
OK, so if we would like to provide a way for all nodes to also announce their L3 subnets which are maybe behind them, we should run additional L3 routing protocol between them so that they can exchange information about those subnets and configure routes accordingly. Which routing protocol are you proposing for such task? If I understand it can be a really simple one, checking just when a node is accessible and setting its announcements on all other nodes. There is no need to think about link quality or anything else. If node is accessible, set routes for its announced subnets to it. OK, one need we would like is that it provides some routing filtering capabilities. So that we could for example forbid to announce anything from 192.168/16. How does such protocol determine node availability?
Mitar
On Fri, Apr 13, 2012 at 09:58:26AM +0200, Mitar wrote:
Hi!
On Fri, Apr 13, 2012 at 7:59 AM, Andrew Lunn andrew@lunn.ch wrote:
This comes back to:
?? ?? You need, in your head, a clear separation between L3 and L2.
:-)
OK, maybe I am getting a bit of hold on this. :-)
OK, so if we would like to provide a way for all nodes to also announce their L3 subnets which are maybe behind them, we should run additional L3 routing protocol between them so that they can exchange information about those subnets and configure routes accordingly. Which routing protocol are you proposing for such task?
How many subnets do you think you will have? Are they all using classful networks or classless network?
http://en.wikipedia.org/wiki/Classful_network http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
If you have a small network which is classful, you could use RIP.
http://en.wikipedia.org/wiki/Routing_Information_Protocol
Otherwise, i would probably use ospf. http://en.wikipedia.org/wiki/Ospf
OSPF is much more scale-able, and typically used for enterprise networks. If you are considering joining this network to some other network, you are probably going to end up using OSPF, or maybe BGP as the protocol between the networks. OSPF is designed for this sort of multi-organizational networks.
quagga is a well used suite of routing protocols. http://www.nongnu.org/quagga/
Andrew
Hi!
On Fri, Apr 13, 2012 at 10:15 AM, Andrew Lunn andrew@lunn.ch wrote:
How many subnets do you think you will have? Are they all using classful networks or classless network?
Probably not much. At most 10 % of nodes would announce their home subnets too. But we should support also classless networks.
quagga is a well used suite of routing protocols. http://www.nongnu.org/quagga/
If I understand correctly, this would allow us also easier peering with other networks as quagga supports also redistribution of routes and so on. So if we decide for OSPF, it will be easy also to setup BGP on border nodes within the single daemon, no?
How CPU and memory heavy it is to run it on consumer routers like TP-links and so on? Probably depends on number of routes and not on itself? But is there a big penalty of running it on all nodes?
Mitar
If I understand correctly, this would allow us also easier peering with other networks as quagga supports also redistribution of routes and so on. So if we decide for OSPF, it will be easy also to setup BGP on border nodes within the single daemon, no?
Yes, quagga has BGP, but depending on your organizational structure, it might be easier to use OSPF on your peering links. It depends on your peers. If its The Internet, then you probably have little choice, you need BGP. However, if your peer is another mesh network, you can decide for your self what routing protocol to use.
How CPU and memory heavy it is to run it on consumer routers like TP-links and so on? Probably depends on number of routes and not on itself? But is there a big penalty of running it on all nodes?
I've used quagga on Gateworks Cambria devices, with a Xscale CPU, 667MHz, 128Mbytes RAM. These tend to have more RAM than typical consumer devices, but i've no idea how much is actually consumed.
Andrew
Hi!
On Fri, Apr 13, 2012 at 11:24 AM, Andrew Lunn andrew@lunn.ch wrote:
Yes, quagga has BGP, but depending on your organizational structure, it might be easier to use OSPF on your peering links. It depends on your peers. If its The Internet, then you probably have little choice, you need BGP. However, if your peer is another mesh network, you can decide for your self what routing protocol to use.
OK, but quagga can run multiple protocols at the same time? Like OSPF into one direction and BGP into another? (Just making sure if I understand correctly.)
I've used quagga on Gateworks Cambria devices, with a Xscale CPU, 667MHz, 128Mbytes RAM. These tend to have more RAM than typical consumer devices, but i've no idea how much is actually consumed.
We would like to run that on Foneras. Does anybody have experience with this? So we are running OLSR and it works good. If we switch to quagga and OSPF and same network, will we have more or less CPU usage?
Mitar
On 04/13/2012 05:41 AM, Mitar wrote:
Hi!
On Fri, Apr 13, 2012 at 10:15 AM, Andrew Lunn andrew@lunn.ch wrote:
How many subnets do you think you will have? Are they all using classful networks or classless network?
Probably not much. At most 10 % of nodes would announce their home subnets too. But we should support also classless networks.
I'm curious as to why those nodes wouldn't use the same IP address space as the net if they will be announcing their routes to the mesh anyway.
One thing to be aware of is that when a home network (on a different subnet) is connected to the batman mesh you get some side-effects, like for example an in-house router responding to DHCP requests... It happens here in QuintanaLibre where one user has a router with 2 IPs, one for the house network and another for the mesh and DHCP server enabled using the home net address space, which sometimes "leak" to a client in the mesh.
I understand the batman-adv mesh works almost as one big switch, so it makes sense that this router will answer for requests... but at first I could not understand what was happening.
One other suggestion, for VPN is to take a look at Tinc; it has a routing (or router?) mode which might be useful in your case.
quagga is a well used suite of routing protocols. http://www.nongnu.org/quagga/
If I understand correctly, this would allow us also easier peering with other networks as quagga supports also redistribution of routes and so on. So if we decide for OSPF, it will be easy also to setup BGP on border nodes within the single daemon, no?
How CPU and memory heavy it is to run it on consumer routers like TP-links and so on? Probably depends on number of routes and not on itself? But is there a big penalty of running it on all nodes?
Mitar
Hi!
2012/4/14 Nicolás Echániz nicoechaniz@codigosur.org:
I'm curious as to why those nodes wouldn't use the same IP address space as the net if they will be announcing their routes to the mesh anyway.
Probably they will use the same IP address space, but we want to route those subnets not bridge them. This is the main difference.
One thing to be aware of is that when a home network (on a different subnet) is connected to the batman mesh you get some side-effects, like for example an in-house router responding to DHCP requests...
If we do routing between home network and mesh network, no.
Also, our routers will not run DHCP itself. (Probably DHCP will be run inside home networks. But as I said, we would like to only route there, not bridge.)
One other suggestion, for VPN is to take a look at Tinc; it has a routing (or router?) mode which might be useful in your case.
No, it is an user-space daemon. Context switches between user-space and kernel-space kills much of the performance on those cheap routers. We are currently using OpenVPN and I doubt Tinc will improve much in this aspect. Or am I mistaken?
Mitar
Hi!
I have a feature request.
On Fri, Apr 13, 2012 at 7:59 AM, Andrew Lunn andrew@lunn.ch wrote:
- BATMAN is not a daemon. Its a kernel module. Hence it cannot play
around with routes, even if it wanted to. Also, adding/removing routes is policy, and the Linux/Unix view of the world is that user space is responsible for deciding policy.
I have problems with an idea of running another routing protocol on top of Batman. I think this is creates unnecessary additional bandwidth consumption: 1) routing protocol has to find which nodes are reachable 2) routing protocol has to exchange information about announced routes
But 1) Batman already knows. So we could have a netlink protocol where user-land daemon could subscribe to events and get for example two events: "node reachable" and "node unreachable". Then it can do some user-land mingling. :-)
And for 2), Batman could allow a general key/value additional information to be announced by nodes. From its point of view that information can be anything.
And the in our case we could add to each node information which IP it has and which subnets it has behind. This information would then be announced to other nodes. And then we could run a simple daemon on other nodes which would listen to netlink events and once a node is reachable, read that information and setup necessary routes.
I like the idea because it allows general hooks into Batman reachability awareness. So no duplication of this is necessary. Also, Batman already exchanges information, so we could just allow custom additional information from nodes to be exchanged. Maybe even stored in that DHT you have.
Mitar
On Monday, April 16, 2012 12:25:42 Mitar wrote:
I have problems with an idea of running another routing protocol on top of Batman. I think this is creates unnecessary additional bandwidth consumption:
- routing protocol has to find which nodes are reachable
- routing protocol has to exchange information about announced routes
But 1) Batman already knows. So we could have a netlink protocol where user-land daemon could subscribe to events and get for example two events: "node reachable" and "node unreachable". Then it can do some user-land mingling. :-)
Batman-adv already has an event reporting system (via sysfs): http://www.open-mesh.org/wiki/batman-adv/Uevent It could be easily extended to also throw node events (reachable/unreachable).
Note: Whoever is going to implement that should think about the implications of mesh clouds periodically connecting and disconnecting. A lot of events could be fired over and over again.
And for 2), Batman could allow a general key/value additional information to be announced by nodes. From its point of view that information can be anything.
And the in our case we could add to each node information which IP it has and which subnets it has behind. This information would then be announced to other nodes. And then we could run a simple daemon on other nodes which would listen to netlink events and once a node is reachable, read that information and setup necessary routes.
There is no need to implement such a thing in batman-adv. You can have this feature today by using any broadcast system (avahi, routing daemon, self- implemented thingy, etc) you like. Here also lies the misunderstanding: You don't need to run another routing protocol "on top" of batman-adv. You only run it to announce "stuff". You can simply run your routing protocol of choice to announce the routes. This was explained several times in this thread - maybe it still is not clear?
Cheers, Marek
Hi!
On Mon, Apr 16, 2012 at 12:37 PM, Marek Lindner lindner_marek@yahoo.de wrote:
Note: Whoever is going to implement that should think about the implications of mesh clouds periodically connecting and disconnecting. A lot of events could be fired over and over again.
We made such extension to olsrd (which runs given bash script) and I think it is the consumer who needs to take care to aggregate/average those events and so on. Batman logic could be very simple.
You don't need to run another routing protocol "on top" of batman-adv. You only run it to announce "stuff". You can simply run your routing protocol of choice to announce the routes.
Announcing and checking for node reachability, no? And the second part still require some additional bandwidth, no?
Mitar
On Monday, April 16, 2012 12:46:35 Mitar wrote:
Note: Whoever is going to implement that should think about the implications of mesh clouds periodically connecting and disconnecting. A lot of events could be fired over and over again.
We made such extension to olsrd (which runs given bash script) and I think it is the consumer who needs to take care to aggregate/average those events and so on. Batman logic could be very simple.
If you already have a shell script uevent definitely is the way to go. It is far easier to script than netlink.
How does your logic handle network splits / join ? Do you have the source somewhere ?
You don't need to run another routing protocol "on top" of batman-adv. You only run it to announce "stuff". You can simply run your routing protocol of choice to announce the routes.
Announcing and checking for node reachability, no? And the second part still require some additional bandwidth, no?
Batman would also need to announce the route. What are we saving? How reachability is handled / verified depends on the routing daemon you choose. You could even connect your routing daemon to the uevents mentioned above.
I don't see any benefit of putting this into batman-adv - only a loss of flexibility / choice.
Cheers, Marek
Hi!
On Mon, Apr 16, 2012 at 12:58 PM, Marek Lindner lindner_marek@yahoo.de wrote:
If you already have a shell script uevent definitely is the way to go. It is far easier to script than netlink.
Yes, but are there uevent events for node reachable/unreachable?
How does your logic handle network splits / join ? Do you have the source somewhere ?
Our current logic is very simple: if route to 0.0.0.0 disappears, display to the user error message through splash screen ("sorry, node is separated from the rest of the mesh" message) when user tries to access external IPs. When it comes back, return to normal behavior.
I am not sure what you mean by splits / joins. Yes, source is available. But I am not sure if this is what you had in mind?
I was just saying that route flapping (on/off) can be dealt with some aggregation of events on a consumer side. (But I am now unsure, maybe we are doing this already on the plugin side.)
For olsrd plugin:
http://dev.wlan-si.net/browser/firmware/packages/opkg/net/olsrd/plugins/acti...
You could even connect your routing daemon to the uevents mentioned above.
Are those uevents already available? Do we know of any existing routing protocol being capable of using them?
Mitar
On Mon, Apr 16, 2012 at 02:32:09PM +0200, Mitar wrote:
Hi!
On Mon, Apr 16, 2012 at 12:58 PM, Marek Lindner lindner_marek@yahoo.de wrote:
If you already have a shell script uevent definitely is the way to go. It is far easier to script than netlink.
Yes, but are there uevent events for node reachable/unreachable?
Actually there is only one class of event: Gateway event. A Uevent is sent to the userspace each time the node ADD/DELETE/CHANGE its own batman-adv-GW. Actually, if the batman-adv-GW is co-located with the L3 GW, you can use this event to understand whether you have or not a default route in the network (assuming that I correctly understood your scenario).
Are those uevents already available? Do we know of any existing routing protocol being capable of using them?
Uevents support is in the batman-adv code, but as I told you before we only have GW events right now. But the code is easily extendible for other uevents :)
Then, in the userspace you can use UDEV to handle such uevents and do something you like (e.g. triggeringa bash script, relaunching another daemon, whatever..)
Cheers,
Hi!
On Mon, Apr 16, 2012 at 2:40 PM, Antonio Quartulli ordex@autistici.org wrote:
you can use this event to understand whether you have or not a default route in the network (assuming that I correctly understood your scenario).
Yes. This is one feature we need. To detect gateway availability.
But also general subnet routing so that we do not need to bridge home networks into the mesh, but route to them.
So, about my question from before about other routing protocols, like OSPF, it probably still consumes bandwidth to detect node availability, no?
Mitar
On Monday, April 16, 2012 15:59:11 Mitar wrote:
But also general subnet routing so that we do not need to bridge home networks into the mesh, but route to them.
So, about my question from before about other routing protocols, like OSPF, it probably still consumes bandwidth to detect node availability, no?
Unless somebody shows up with real life experience regarding the routing daemons you consider using you have to test it yourself. I am sure there are mechanisms as network failures were not a new invention that came with wifi. ;)
Cheers, Marek
On Monday, April 16, 2012 14:32:09 Mitar wrote:
I am not sure what you mean by splits / joins. Yes, source is available. But I am not sure if this is what you had in mind?
I was just saying that route flapping (on/off) can be dealt with some aggregation of events on a consumer side. (But I am now unsure, maybe we are doing this already on the plugin side.)
I was talking about the scenario in which you have two mesh clouds consisting of roughly 100 nodes. These two clouds are barely connected via a single flaky link. Whenever this link vanishes you have 100 unreachable events and a minute later 100 reachable events and so on ..
Cheers, Marek
On Mon, Apr 16, 2012 at 12:46:35PM +0200, Mitar wrote:
Hi!
On Mon, Apr 16, 2012 at 12:37 PM, Marek Lindner lindner_marek@yahoo.de wrote:
Note: Whoever is going to implement that should think about the implications of mesh clouds periodically connecting and disconnecting. A lot of events could be fired over and over again.
We made such extension to olsrd (which runs given bash script) and I think it is the consumer who needs to take care to aggregate/average those events and so on. Batman logic could be very simple.
You don't need to run another routing protocol "on top" of batman-adv. You only run it to announce "stuff". You can simply run your routing protocol of choice to announce the routes.
Announcing and checking for node reachability, no? And the second part still require some additional bandwidth, no?
We are not talking about another "wireless oriented" routing protocol, which would need to measure link quality (cost) and announce it. We are talking about something like OSPF/BGP/RIP which can be configured to announce routes only (actually I think this is the standard way they work and this is also the reason why they are not suitable for wireless networks).
Cheers,
Hi!
On Mon, Apr 16, 2012 at 1:03 PM, Antonio Quartulli ordex@autistici.org wrote:
We are not talking about another "wireless oriented" routing protocol, which would need to measure link quality (cost) and announce it. We are talking about something like OSPF/BGP/RIP which can be configured to announce routes only (actually I think this is the standard way they work and this is also the reason why they are not suitable for wireless networks).
I admit I am not familiar how those protocols work, but I imagine they need to check if node is available/reachable in some regular intervals? Or it is that node is just announcing its routes in regular intervals and once it stops, after some timeout other daemons remove the route?
Yes, I am aware that wireless cost is unneeded. But reachability checking should still be done?
BTW, if multiple nodes are announcing the same subnet/prefix, how does the L3 routing protocol daemon knows which one to select? From its point of view everything is one hope away, but in reality there could be huge differences?
For DHCP you sniff packets. But could be there some general way?
Mitar
b.a.t.m.a.n@lists.open-mesh.org
-
Andrew Lunn -
Antonio Quartulli -
Gioacchino Mazzurco -
Jernej Kos -
Marek Lindner -
Mitar -
Nicolás Echániz