Hi,
Mi name is NicoEchániz, this is my first post to the list.
********** INTRO, please skip it if you find it long to read **********
I've been doing free-network stuff for a while (around 10 years). I started in Buenos Aires, Argentina, where I built one of the first nodes for BuenosAiresLibre.org
I was the coordinator of the first JRRL (Free Networks Regional Meeting), where Ramón Roca (from guifi.net) was our guest keynote and members from Latin American networks intentionally gathered for the first time.
I've moved last year to a small town in another province, where I'm experimenting whith a completely different network. BuenosAiresLibre (a.k.a. BAL) run in infrastructure mode and used OLSR for routing.
In this town (Quintana), we are now building an ad-hoc mesh and as you must have guessed we're using batman-adv for routing.
Some friends from BAL came along and we worked for a couple of weeks on this experiment.
You may find information here (spanish but g.translator might help): http://wiki.arraigodigital.org.ar/RedLibre/QuintanaCamp/Documentaci%C3%B3n
And some interesting pictures here: http://www.lavecindaria.org.ar/category/quintanacamp/
QuintanaLibre.org is the first experimental implementation of a cheap design of ad-hoc mesh network that we intend to re-use in hundreds of small towns around the country, in colaboration with the national Ministry of Education, through the Arraigo Digital plan. Some info on that can be found here: http://www.arraigodigital.org.ar
***********************************************************************
Well... after this not-so-brief introduction, here's what I'd like to ask about.
We have been experimenting with different setups for our nodes. Some of them are regular TP-Link MR-3220 (thanks Elektra for the recommendation) with a PoE modification; others have a USB/Wi-fi adapter added (TP-Link WN722N) and there's a third kind where we connected an Ubiquiti BulletM2 to an MR-3220.
This third kind of setup is the one that has been giving us trouble.
There's a total 5 nodes running in the test network; the longest inter-node distance is 1.5 Km.
In the example below, the MR is called "marisa-mr" and the bulletM2 is called "marisa-blt" (this is the only node with this setup so far). They are connected by ethernet and ad-hoc and share the same tower space.
The network seemed to work quite well, but from time to time, pings would sort of fall into a hole for a while... so we started looking at traceroutes and this is what we found.
Traceroute is done from a notebook connected (with batman active on eth0) to the node called "nogal" and trying to trace the route to czuk_wlan1 (the other end of the net).
running the command several times, from time to time we get this sort of output:
$ sudo batctl tr czuk_wlan1 traceroute to czuk_wlan1 (f8:d1:11:0b:76:4b), 50 hops max, ... 1: nogal_wlan0 (00:15:6d:d6:24:7a) 0.395 ms 0.214 ms 0.416 ms 2: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.702 ms 3.291 ms 1.694 ms 3: cisterna_wlan0 (54:e6:fc:b9:be:e8) 25.282 ms * * 4: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 2.694 ms 1.722 ms 5.462 ms 5: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 1.758 ms 1.584 ms 5.409 ms 6: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.335 ms 4.584 ms 3.111 ms ... 47: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 2.090 ms 2.180 ms 2.138 ms 48: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 4.738 ms 2.898 ms 2.735 ms 49: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 2.325 ms 2.269 ms 2.249 ms
$ sudo batctl tr czuk_wlan1 traceroute to czuk_wlan1 (f8:d1:11:0b:76:4b), 50 hops max, ... 1: nogal_wlan0 (00:15:6d:d6:24:7a) 0.541 ms 0.193 ms 0.188 ms 2: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.347 ms 1.221 ms 1.215 ms 3: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 1.281 ms 5.103 ms 1.255 ms 4: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 5.944 ms 1.705 ms 2.238 ms ... 33: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 1.920 ms 1.890 ms 1.815 ms 34: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 2.859 ms 1.778 ms 4.499 ms 35: * * * * 36: czuk_wlan1 (f8:d1:11:0b:76:4b) 299.941 ms 37.911 ms *
and this is what a correct traceroute looks like:
$ sudo batctl tr czuk_wlan1 traceroute to czuk_wlan1 (f8:d1:11:0b:76:4b), 50 hops max, 20 byte packets 1: nogal_wlan0 (00:15:6d:d6:24:7a) 0.292 ms 0.211 ms 0.209 ms 2: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.541 ms 1.407 ms 2.508 ms 3: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 1.464 ms 1.593 ms 1.466 ms 4: cisterna_wlan0 (54:e6:fc:b9:be:e8) 5.275 ms 18.669 ms 4.106 ms 5: czuk_wlan1 (f8:d1:11:0b:76:4b) 3.505 ms 5.681 ms 5.198 ms
or this one, when the chosen route skips the cisterna node: $ sudo batctl tr czuk_wlan1 traceroute to czuk_wlan1 (f8:d1:11:0b:76:4b), 50 hops max, ... 1: nogal_wlan0 (00:15:6d:d6:24:7a) 0.293 ms 0.184 ms 1.990 ms 2: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.638 ms 1.687 ms 1.442 ms 3: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 1.635 ms 1.334 ms 1.463 ms 4: czuk_wlan1 (f8:d1:11:0b:76:4b) 8.783 ms 3.844 ms 3.704 ms
this happens with the nodes configured according to: http://www.open-mesh.org/wiki/batman-adv/Bridge-loop-avoidance
or so we understand!
here are the relevant portions of the nodes config. we have a central repo for configurations, thus the "strange" syntax.
MARISA-BLT
$ uci show wireless.@wifi-iface[0] -c marisa-blt/etc/config/ wireless.cfg033579=wifi-iface wireless.cfg033579.device=radio0 wireless.cfg033579.encryption=none wireless.cfg033579.mode=adhoc wireless.cfg033579.ssid=mesh.quintanalibre.org.ar wireless.cfg033579.bssid=02:12:34:56:78:9A
# wlan0 here is a master mode interface $ uci show network.lan.ifname -c marisa-blt/etc/config/ network.lan.ifname=bat0 wlan0 eth0
# wlan0-1 here is the mesh interface $ uci show batman-adv.bat0.interfaces -c marisa-blt/etc/config/ batman-adv.bat0.interfaces=wlan0-1 br-lan
MARISA-MR
$ uci show wireless.@wifi-iface[0] -c marisa-mr/etc/config/ wireless.cfg033579=wifi-iface wireless.cfg033579.encryption=none wireless.cfg033579.device=radio0 wireless.cfg033579.bssid=02:12:34:56:78:9A wireless.cfg033579.mode=adhoc wireless.cfg033579.ssid=mesh.quintanalibre.org.ar (this is wlan0 in this router)
#eth1 is connected to a router inside the owner's house; no batman $ uci show network.lan.ifname -c marisa-mr/etc/config/ network.lan.ifname=bat0 eth1 eth0
#wlan0 is the only wireless interface here $ uci show batman-adv.bat0.interfaces -c marisa-mr/etc/config/ batman-adv.bat0.interfaces=wlan0 br-lan
We have also tried adding eth0 to bat0 and take it out of br-lan, which also "works" but gives routing loops from time to time.
OpenWRT is trunk from a couple of weeks ago, where batman-adv version was 2011.4
We hoped that adding a second router through ethernet would give the mesh an alternate path and more redundancy, but we get much lower throughputs because of these loops.
We would very much appreciate any insight on this matter.
Hoping to read from you bat-friends soon :)
Cheers, NicoEchániz
On Saturday, March 03, 2012 15:39:09 Nicolás Echániz wrote:
this happens with the nodes configured according to: http://www.open-mesh.org/wiki/batman-adv/Bridge-loop-avoidance
or so we understand!
If the ethernet is used for mesh purposes uniquely why do you bother running the bridge loop avoidance at all instead of adding the ethernet interface as a regular batman interface ? The only reason why you would want the bridge loop avoidance is when you have ethernet clients that are bridged into the mesh. This does not seem to be the case. As far as I understand the ethernet cable is used as reliable "wireless cable".
Cheers, Marek
On 03/03/2012 04:53 AM, Marek Lindner wrote:
On Saturday, March 03, 2012 15:39:09 Nicolás Echániz wrote:
this happens with the nodes configured according to: http://www.open-mesh.org/wiki/batman-adv/Bridge-loop-avoidance
or so we understand!
If the ethernet is used for mesh purposes uniquely why do you bother running the bridge loop avoidance at all instead of adding the ethernet interface as a regular batman interface ? The only reason why you would want the bridge loop avoidance is when you have ethernet clients that are bridged into the mesh. This does not seem to be the case. As far as I understand the ethernet cable is used as reliable "wireless cable".
Maerk thanks for your fast reply. In fact, that's how we started, that's what I meant by:
"We have also tried adding eth0 to bat0 and take it out of br-lan, which also "works" but gives routing loops from time to time."
but my english sometimes fails me :)
we also had routing loops with that setup and that's where we started to look at loop avoidance docs.
I haven't saved traceroutes from that moment; I've actually turned off the wlan0 on the MR now, but I can turn it back on and add eth0 as a batman interface to reproduce it. Is this a known issue or should this not happen in a setup where both interfaces are added to batman?
On Saturday, March 03, 2012 16:16:15 Nicolás Echániz wrote:
Maerk thanks for your fast reply. In fact, that's how we started, that's what I meant by:
"We have also tried adding eth0 to bat0 and take it out of br-lan, which also "works" but gives routing loops from time to time."
but my english sometimes fails me :)
Your English is quite good. To be honest I lost the overview about what interface is added where and what is connected how. Some people might be able to simply look at the uci config to understand the setup but I am not one of those. :-)
Let me write down what I understood - feel free to correct me.
some nodes -- node1 -- adhoc[wlan0-1] + ethernet[eth0] -- node2 -- more nodes
Each node also has wlan0 for non-mesh clients but this is of no relevance at this point. What is eth1 for ?
You are saying that from time to time the packets loop between node1 and node2 using adhoc & ethernet to fly forth and back ?
Is this a known issue or should this not happen in a setup where both interfaces are added to batman?
You should not have loops either way but it is easy to build loops in complicated setups. At first we should understand your setup and configuration. Drawing a little picture that shows what interface is connected to what other interface also can help.
Regards, Marek
On 03/03/2012 05:43 AM, Marek Lindner wrote:
On Saturday, March 03, 2012 16:16:15 Nicolás Echániz wrote:
Maerk thanks for your fast reply. In fact, that's how we started, that's what I meant by:
"We have also tried adding eth0 to bat0 and take it out of br-lan, which also "works" but gives routing loops from time to time."
but my english sometimes fails me :)
Your English is quite good. To be honest I lost the overview about what interface is added where and what is connected how. Some people might be able to simply look at the uci config to understand the setup but I am not one of those. :-)
Let me write down what I understood - feel free to correct me.
some nodes -- node1 -- adhoc[wlan0-1] + ethernet[eth0] -- node2 -- more nodes
Each node also has wlan0 for non-mesh clients but this is of no relevance at this point. What is eth1 for ?
in the node marisa-mr, eth1 is the WAN port of the router, that is not being used as wan but just as a non-batman ethernet interface to connect to a router inside the house.
You are saying that from time to time the packets loop between node1 and node2 using adhoc & ethernet to fly forth and back ?
exactly so.
Is this a known issue or should this not happen in a setup where both interfaces are added to batman?
You should not have loops either way but it is easy to build loops in complicated setups. At first we should understand your setup and configuration. Drawing a little picture that shows what interface is connected to what other interface also can help.
alright, I'll try some ascii art, but I'm afraid you'll find out my english is better :P
(non-batman) wlan0 | ------ | czuk | ------ /wlan1 / / / / 1500m / / / wlan1 ---------- / | cisterna | ---------- (batman enabled | cabled clients) wlan0 | / \ eth0 / \ | / \ ------- / \ | nogal | / 350m \ 600m / ------- / \ --------wlan0 / \ / / \ / wlan0--------------wlan0-1 / \ ------------ ------------ | marisa-mr | 0m | marisa-blt | ------------ ------------ \ | \eth0__________________eth0/ wlan0 eth1 (non-batman) (non-batman) | | | [in-house router]
where not indicated otherwise, links are between interfaces added to batman.
I hope it's understandable!
... at the moment I've disabled marisa-mr_wlan0 and everything works fine.
The routing loop problem appears when marisa-mr_wlan0 is enabled, as shown in the drawing.
The setup makes more sense when you add the antennae information. marisa-blt has a panel/sector antena, but marisa-mr has a 24dbi grid pointed at cisterna, which in turn has another grid pointed back (on wlan0) and a second grid (on wlan1) points at czuk on the opposite side, where a grid points back.
I've drawn the "desired" links; the wireless portion is ad-hoc, so there are many more links present between the nodes, in fact cisterna, czuk and both marisa's can all "see" each other.
cisterna_wlan1 has a reasonable link through the back lobe of the grid antenna to both marisa-blt_wlan0-1 and marisa-mr_wlan0.
If you'd like me to send any more information just ask please; and if you get bored, I'll understand!!
-- NicoEchániz
On 03/03/2012 07:14 AM, Nicolás Echániz wrote:
On 03/03/2012 05:43 AM, Marek Lindner wrote:
You should not have loops either way but it is easy to build loops in complicated setups. At first we should understand your setup and configuration. Drawing a little picture that shows what interface is connected to what other interface also can help.
alright, I'll try some ascii art, but I'm afraid you'll find out my english is better :P
holy s... I knew this wouldn't work
I've uploaded the image here: http://www.lavecindaria.org.ar/picture/esquema-de-enlaces/
(non-batman) wlan0 | ------ | czuk | ------ /wlan1 / / / / 1500m / / / wlan1 ---------- / | cisterna | ---------- (batman enabled | cabled clients) wlan0 | / \ eth0 / \ | / \ ------- / \ | nogal | / 350m \ 600m / ------- / \ --------wlan0 / \ / / \ / wlan0--------------wlan0-1 / \ ------------ ------------| marisa-mr | 0m | marisa-blt | ------------ ------------ \ | \eth0__________________eth0/ wlan0 eth1 (non-batman) (non-batman) | | | [in-house router]
On Saturday, March 03, 2012 18:24:40 Nicolás Echániz wrote:
On 03/03/2012 07:14 AM, Nicolás Echániz wrote:
On 03/03/2012 05:43 AM, Marek Lindner wrote:
You should not have loops either way but it is easy to build loops in complicated setups. At first we should understand your setup and configuration. Drawing a little picture that shows what interface is connected to what other interface also can help.
alright, I'll try some ascii art, but I'm afraid you'll find out my english is better :P
holy s... I knew this wouldn't work
I've uploaded the image here: http://www.lavecindaria.org.ar/picture/esquema-de-enlaces/
I got your drawing. Would it possible to also post the output of: * brctl show * batctl if * batctl o
We don't need it from all nodes. cisterna, marisa-mr and marisa-blt should be enough. Please configure these nodes, so that they theoretically would produce the routing loop although it is not necessary to wait for the loop itself. As I understand it that loop occurs from time to time only?
Regards, Marek
On 03/03/2012 08:32 AM, Marek Lindner wrote:
On Saturday, March 03, 2012 18:24:40 Nicolás Echániz wrote:
On 03/03/2012 07:14 AM, Nicolás Echániz wrote:
On 03/03/2012 05:43 AM, Marek Lindner wrote:
You should not have loops either way but it is easy to build loops in complicated setups. At first we should understand your setup and configuration. Drawing a little picture that shows what interface is connected to what other interface also can help.
alright, I'll try some ascii art, but I'm afraid you'll find out my english is better :P
holy s... I knew this wouldn't work
I've uploaded the image here: http://www.lavecindaria.org.ar/picture/esquema-de-enlaces/
I got your drawing. Would it possible to also post the output of:
- brctl show
- batctl if
- batctl o
We don't need it from all nodes. cisterna, marisa-mr and marisa-blt should be enough.
Here you go:
############## marisa-mr ###############
root@marisa-mr:~# brctl show bridge name bridge id STP enabled interfaces br-lan 8000.54e6fcb9cb39 no eth1 bat0 root@marisa-mr:~# batctl if eth0: active wlan0: active
root@marisa-mr:~# batctl o [B.A.T.M.A.N. adv 2011.4.0, MainIF/MAC: wlan0/54:e6:fc:b9:cb:38 (bat0)] Originator last-seen (#/255) Nexthop [outgoingIF]: Potential nexthops ... nicoyjesi_adhoc 0.140s (231) marisa-blt_eth0 [ eth0]: cisterna_wlan1 (179) cisterna_wlan0 (180) marisa-blt_adhoc (188) marisa-blt_eth0 (231) nogal_wlan0 (131) nogal_wlan0 0.620s (245) marisa-blt_eth0 [ eth0]: cisterna_wlan1 (191) cisterna_wlan0 (188) marisa-blt_adhoc (199) marisa-blt_eth0 (245) nogal_wlan0 (139) cisterna_wlan0 0.640s (245) marisa-blt_eth0 [ eth0]: marisa-blt_adhoc (199) marisa-blt_eth0 (245) cisterna_wlan1 (220) nogal_wlan0 (118) cisterna_wlan0 (215) cisterna_wlan1 0.390s (222) cisterna_wlan1 [ wlan0]: marisa-blt_adhoc (199) marisa-blt_eth0 ( 0) cisterna_wlan1 (222) czuk_wlan1 0.440s (143) marisa-blt_eth0 [ eth0]: nogal_wlan0 ( 84) marisa-blt_adhoc (117) marisa-blt_eth0 (143) cisterna_wlan0 (153) cisterna_wlan1 (138) marisa-blt_eth0 0.240s (253) marisa-blt_eth0 [ eth0]: cisterna_wlan1 (202) cisterna_wlan0 (201) marisa-blt_adhoc (208) nogal_wlan0 (123) marisa-blt_eth0 (253) marisa-blt_adhoc 0.450s (209) marisa-blt_adhoc [ wlan0]: nogal_wlan0 (119) cisterna_wlan1 (187) cisterna_wlan0 (186) marisa-blt_adhoc (209)
############## marisa-blt ###############
root@marisa-blt:~# brctl show bridge name bridge id STP enabled interfaces br-lan 8000.00156d3e2c4f no wlan0 bat0 root@marisa-blt:~# batctl if eth0: active wlan0-1: active
root@marisa-blt:~# batctl o [B.A.T.M.A.N. adv 2012.0.0, MainIF/MAC: eth0/00:15:6d:3f:2c:4f (bat0)] Originator last-seen (#/255) Nexthop [outgoingIF]: Potential nexthops ... nicoyjesi_adhoc 0.620s (170) nogal_wlan0 [ wlan0-1]: cisterna_wlan1 (144) cisterna_wlan0 (139) czuk_wlan1 ( 0) nogal_wlan0 (170) marisa-mr_eth0 0.950s (129) marisa-mr_eth0 [ eth0]: marisa-mr_eth0 (129) nogal_wlan0 0.040s (182) nogal_wlan0 [ wlan0-1]: marisa-mr_wlan0 ( 77) czuk_wlan1 ( 0) cisterna_wlan1 (153) cisterna_wlan0 (149) marisa-mr_eth0 ( 74) nogal_wlan0 (182) cisterna_wlan0 0.780s (238) cisterna_wlan1 [ wlan0-1]: marisa-mr_wlan0 (118) marisa-mr_eth0 (110) nogal_wlan0 (131) czuk_wlan1 ( 0) cisterna_wlan1 (238) cisterna_wlan0 (231) cisterna_wlan1 0.420s (238) cisterna_wlan1 [ wlan0-1]: marisa-mr_wlan0 (116) czuk_wlan1 (174) cisterna_wlan1 (238) marisa-mr_wlan0 0.000s (138) marisa-mr_wlan0 [ wlan0-1]: czuk_wlan1 ( 0) nogal_wlan0 (109) cisterna_wlan1 (113) cisterna_wlan0 (112) marisa-mr_wlan0 (138) marisa-mr_eth0 (133) czuk_wlan1 0.940s (214) cisterna_wlan1 [ wlan0-1]: marisa-mr_wlan0 (103) marisa-mr_eth0 ( 95) cisterna_wlan0 (210) cisterna_wlan1 (214) czuk_wlan1 (204)
############## cisterna ###############
root@cisterna:~# brctl show bridge name bridge id STP enabled interfaces br-lan 8000.54e6fcb9bee7 no eth0 bat0 root@cisterna:~# batctl if wlan0: active wlan1: active
root@cisterna:~# batctl o [B.A.T.M.A.N. adv 2011.4.0, MainIF/MAC: wlan0/54:e6:fc:b9:be:e8 (bat0)] Originator last-seen (#/255) Nexthop [outgoingIF]: Potential nexthops ... nicoyjesi_adhoc 0.390s (217) marisa-mr_wlan0 [ wlan0]: czuk_wlan1 ( 58) czuk_wlan1 (175) marisa-mr_wlan0 (207) marisa-mr_wlan0 (217) marisa-blt_adhoc (205) marisa-blt_adhoc (209) nogal_wlan0 0.850s (223) marisa-mr_wlan0 [ wlan0]: czuk_wlan1 (186) czuk_wlan1 ( 62) nogal_wlan0 ( 0) marisa-mr_wlan0 (216) marisa-mr_wlan0 (223) marisa-blt_adhoc (214) marisa-blt_adhoc (216) marisa-mr_wlan0 0.050s (250) marisa-mr_wlan0 [ wlan0]: czuk_wlan1 ( 67) czuk_wlan1 (201) marisa-blt_adhoc (213) marisa-blt_adhoc (215) marisa-mr_wlan0 (239) marisa-mr_wlan0 (250) czuk_wlan1 0.600s (255) czuk_wlan1 [ wlan1]: marisa-mr_wlan0 (170) marisa-mr_wlan0 (178) czuk_wlan1 ( 85) marisa-blt_adhoc (172) marisa-blt_adhoc (168) czuk_wlan1 (255) marisa-blt_eth0 0.090s (232) marisa-mr_wlan0 [ wlan0]: czuk_wlan1 (198) czuk_wlan1 ( 66) marisa-mr_wlan0 (225) marisa-mr_wlan0 (232) marisa-blt_adhoc (231) marisa-blt_adhoc (227) marisa-blt_adhoc 0.730s (231) marisa-blt_adhoc [ wlan0]: marisa-mr_wlan0 (216) marisa-mr_wlan0 (227) czuk_wlan1 ( 63) czuk_wlan1 (188) marisa-blt_adhoc (231) marisa-blt_adhoc (226)
######################################
Please configure these nodes, so that they theoretically would produce the routing loop although it is not necessary to wait for the loop itself. As I understand it that loop occurs from time to time only?
By "from time to time" I meant that not every packet gets trapped in the loop, but it's easily reproducible.
With marisa-mr_wlan0 activated, I get a looped traceroute just by repeating the command a few times.
And here's a loop:
# batctl tr czuk_wlan1 traceroute to czuk_wlan1 (f8:d1:11:0b:76:4b), 50 hops max, 20 byte packets 1: nogal_wlan0 (00:15:6d:d6:24:7a) 0.128 ms 0.133 ms 0.136 ms 2: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 6.755 ms 4.468 ms 5.130 ms 3: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 2.119 ms 1.603 ms 1.180 ms 4: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 1.794 ms 1.544 ms 1.596 ms 5: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 9.078 ms 6.598 ms 1.844 ms
[...]
44: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 10.385 ms 11.837 ms 13.225 ms 45: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 9.998 ms 11.833 ms 19.300 ms 46: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 10.659 ms 12.662 ms 11.318 ms 47: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 11.114 ms 17.455 ms 10.274 ms 48: marisa-mr_wlan0 (54:e6:fc:b9:cb:38) 17.981 ms 20.367 ms 14.605 ms 49: marisa-blt_eth0 (00:15:6d:3f:2c:4f) 11.862 ms 12.518 ms 14.096 ms
transfer speed (measured with iperf) between marisa and czuk drops to 0.5Mbit/s with this setup, from the 10Mbit/s we get when there are no loops.
The whole network experience is very poor in this setup, ssh sessions lag a lot and drop from timeout, pings fail regularly and transfer speed is impossible.
When marisa-mr_wlan0 is disabled, pings loose 0% end-to-end even with big packets an 0.1s interval and transfer speed is quite good, around 5Mbit/s end to end and 30Mbit/s on the best links.
One other thing that took some time to learn (the hard way) is that the network won't fully recover when we revert the changes in marisa-mr. Loops are gone and pings are normal, but transfer speed sucks. All related nodes (marisa-mr, marisa-blt, cisterna and czuk) need to be restarted for the transfer speed to get back to normal. We assumed this is some kind of problem with the wireles modules, ath9k and ath9k_htc that we use, but it's just an assumption.
Let me know if you find anything unusual in the setup I sent you.
-- NicoEchániz
Regards, Marek
On Sunday, March 04, 2012 10:30:14 Nicolás Echániz wrote:
When marisa-mr_wlan0 is disabled, pings loose 0% end-to-end even with big packets an 0.1s interval and transfer speed is quite good, around 5Mbit/s end to end and 30Mbit/s on the best links.
One other thing that took some time to learn (the hard way) is that the network won't fully recover when we revert the changes in marisa-mr. Loops are gone and pings are normal, but transfer speed sucks. All related nodes (marisa-mr, marisa-blt, cisterna and czuk) need to be restarted for the transfer speed to get back to normal. We assumed this is some kind of problem with the wireles modules, ath9k and ath9k_htc that we use, but it's just an assumption.
Let me know if you find anything unusual in the setup I sent you.
The setup looks good as far as I can tell. I backported 2 patches for 2011.4.0 that we currently have in the pipeline. They are meant to address temporary routing loops. Somewhat similar to what you are experiencing but I am not 100% sure it will help in your case. Could you please apply these patches to batman-adv and install the patched binary on all nodes ? It is essential that all the nodes run the same version to avoid odd routing behavior.
Let us know how it goes!
Cheers, Marek
On 03/04/2012 05:13 AM, Marek Lindner wrote:
On Sunday, March 04, 2012 10:30:14 Nicolás Echániz wrote:
Let me know if you find anything unusual in the setup I sent you.
The setup looks good as far as I can tell. I backported 2 patches for 2011.4.0 that we currently have in the pipeline. They are meant to address temporary routing loops. Somewhat similar to what you are experiencing but I am not 100% sure it will help in your case. Could you please apply these patches to batman-adv and install the patched binary on all nodes ? It is essential that all the nodes run the same version to avoid odd routing behavior.
Let us know how it goes!
Marek,
these experimental nodes were installed from daily snapshot. and batman installed with: opkg install kmod-batman-adv. Are these patches already present in batman-adv 2012.0.0?
If so, we might just re-flash the routers with current trunk; right?
This will take a while because routers are already installed on roof-tops and the network is in use by some neighbors, but it's still experimental, so we will do what needs to be donde to get the best results possible.
On Sunday, March 04, 2012 17:32:38 Nicolás Echániz wrote:
these experimental nodes were installed from daily snapshot. and batman installed with: opkg install kmod-batman-adv. Are these patches already present in batman-adv 2012.0.0?
If so, we might just re-flash the routers with current trunk; right?
No, these patches are not part of any release yet. You need to compile your own batman-adv with the patches applied. We have a document explaining how you should proceed: http://www.open-mesh.org/wiki/batman-adv/Building-with-openwrt
This will take a while because routers are already installed on roof-tops and the network is in use by some neighbors, but it's still experimental, so we will do what needs to be donde to get the best results possible.
There is no need to reflash all the routers. You can simply build a patched batman-adv package and install it on top of your current image. The important part is that you compile the batman-adv kernel module for exactly the same kernel you are running.
Regards, Marek
On 03/04/2012 07:52 AM, Marek Lindner wrote:
On Sunday, March 04, 2012 17:32:38 Nicolás Echániz wrote:
these experimental nodes were installed from daily snapshot. and batman installed with: opkg install kmod-batman-adv. Are these patches already present in batman-adv 2012.0.0?
If so, we might just re-flash the routers with current trunk; right?
No, these patches are not part of any release yet. You need to compile your own batman-adv with the patches applied. We have a document explaining how you should proceed: http://www.open-mesh.org/wiki/batman-adv/Building-with-openwrt
This will take a while because routers are already installed on roof-tops and the network is in use by some neighbors, but it's still experimental, so we will do what needs to be donde to get the best results possible.
There is no need to reflash all the routers. You can simply build a patched batman-adv package and install it on top of your current image. The important part is that you compile the batman-adv kernel module for exactly the same kernel you are running.
Hi Marek,
I've finally had the time to look into this again.
The routers have been updated to current OpenWRT trunk and batman-adv version has changed, would it be too much inconvenience for you to send me your patches for 2012.0.0?
thanks in advance.
NicoEchániz
On Sunday, March 18, 2012 07:31:57 Nicolás Echániz wrote:
I've finally had the time to look into this again.
The routers have been updated to current OpenWRT trunk and batman-adv version has changed, would it be too much inconvenience for you to send me your patches for 2012.0.0?
Here you go.
Regards, Marek
b.a.t.m.a.n@lists.open-mesh.org
-
Marek Lindner -
Nicolás Echániz