Hello,
Are there any documented cases (aside from the aging graph on the wiki) on batman-adv bonding setups?
I'm testing batman-adv 2014.3.0 in an effort to experiment and test with multi-link optimizations. There is no wifi in the picture at the moment, this is being done with tap interfaces right now.
My two nodes have this network config: lan0: bridge of eth0 and bat0, 192.168.100.1 on node1, 192.168.100.2 on node2 bat0: includes tap0 and tap1 active, no IP. eth1: "wan" link, 10.10.10.1 on node1, 10.10.10.2 on node2 tap0/1: openvpn bridge links over eth1, no IPs assigned to these interfaces.
This is a VMware environment for testing. With the above setup, I can ping/iperf/whatever back and forth on the 192.168.100.x network. It works great just extending a lan transparently on a wan link. This was the original idea, with redundant connections (hence the multi tap interfaces).
Now, I'm trying to isolate the test down to bonding. However, enabling bonding in batctl on both nodes has no apparent effect whatsoever. Watching an iperf session in ifstat, I will see constant traffic on bat0, and then either tap0 or tap1 depending on how it's feeling at the time. Sometimes it will shift all the traffic to the other tap interface. Sometimes the incoming traffic will be on one interface and the outgoing traffic will be on the other.
Note: I'm running iperf/etc on the nodes themselves, not on separate devices on the lan bridge.
Ultimately I am looking to try bonding several 3G or 4G devices together with batman-adv to achieve higher throughput to a single destination.
Thank you for any comments or suggestions. I find batman-adv very useful in other projects and look to extend its use into this one.
Ray
Hello Ray,
thanks a lot for your mail.
On Sunday 10 August 2014 15:00:51 Ray Gibson wrote:
Hello,
Are there any documented cases (aside from the aging graph on the wiki) on batman-adv bonding setups?
Not as far as I know. The bonding feature does not seem to be the most popular one. :) (If there is anyone, please tell us!)
I'm testing batman-adv 2014.3.0 in an effort to experiment and test with multi-link optimizations. There is no wifi in the picture at the moment, this is being done with tap interfaces right now.
My two nodes have this network config: lan0: bridge of eth0 and bat0, 192.168.100.1 on node1, 192.168.100.2 on node2 bat0: includes tap0 and tap1 active, no IP. eth1: "wan" link, 10.10.10.1 on node1, 10.10.10.2 on node2 tap0/1: openvpn bridge links over eth1, no IPs assigned to these interfaces.
This is a VMware environment for testing. With the above setup, I can ping/iperf/whatever back and forth on the 192.168.100.x network. It works great just extending a lan transparently on a wan link. This was the original idea, with redundant connections (hence the multi tap interfaces).
Now, I'm trying to isolate the test down to bonding. However, enabling bonding in batctl on both nodes has no apparent effect whatsoever. Watching an iperf session in ifstat, I will see constant traffic on bat0, and then either tap0 or tap1 depending on how it's feeling at the time. Sometimes it will shift all the traffic to the other tap interface. Sometimes the incoming traffic will be on one interface and the outgoing traffic will be on the other.
Note: I'm running iperf/etc on the nodes themselves, not on separate devices on the lan bridge.
Hmm. What you should see after enabling bonding would be a similar usage of both interfaces. I guess each tap interfaces of node1 is "directly connected" to the other tap interface of node2, right? Could you please share your outputs of:
batctl originators batctl originators -i tap0 batctl originators -i tap1
Ultimately I am looking to try bonding several 3G or 4G devices together with batman-adv to achieve higher throughput to a single destination.
Please note that the bonding will only benefit under some circumstances, as far as my experiments have shown:
* since its round robin, you'll only see a benefit if the worst link does not have less than 50% throughput of the best one - otherwise it will slow the other links down. * different latencies or buffering delays in the links may lead to out of order packets, and not every payload traffic likes that.
I could see some improvement when having two equal wifi links though. In any case, please thoroughly test it before applying that to your 3g/4g application, there may be some pitfalls. I'd be also very interested in your findings. :)
Thanks, Simon
Simon,
Thank you for looking at this with me.
On Mon, Aug 11, 2014 at 2:38 PM, Simon Wunderlich sw@simonwunderlich.de wrote:
Hmm. What you should see after enabling bonding would be a similar usage of both interfaces. I guess each tap interfaces of node1 is "directly connected" to the other tap interface of node2, right? Could you please share your outputs of:
Yes, the tap interfaces behave as if they are directly connected together.
batctl originators batctl originators -i tap0 batctl originators -i tap1
I've put all the output in a pastebin as to not clog up everyone's inbox:
Please note that the bonding will only benefit under some circumstances, as far as my experiments have shown:
- since its round robin, you'll only see a benefit if the worst link does not
have less than 50% throughput of the best one - otherwise it will slow the other links down.
- different latencies or buffering delays in the links may lead to out of order
packets, and not every payload traffic likes that.
I could see some improvement when having two equal wifi links though. In any case, please thoroughly test it before applying that to your 3g/4g application, there may be some pitfalls. I'd be also very interested in your findings. :)
Thanks, Simon
Regardless of its potential impact on performance, I think it's worth a look and test to see if there may be some potential benefit. Currently this setup is working nice in that it's a true case of point to point redundant links. If additional links could be added or removed at will to enhance throughput I would see this useful in a variety of applications. Currently, I'd like to just reproduce what the function was originally designed to do and then go from there.
Thanks again for the help.
Ray
Ray,
On Tuesday 12 August 2014 11:21:18 Ray Gibson wrote:
I've put all the output in a pastebin as to not clog up everyone's inbox:
Thanks! The output actually looks fine. I've tried a little bit myself in a few VMs and noticed two problems which were preventing bonding to work, at least for me:
* There was a trivial (and rather stupid) logic bug in the check whether bonding should be considered or not. See "[PATCH] batman-adv: fix and simplify condition when bonding should be used" * Another not so trivial problem is that bonding with the multi interface optimization checks the originator tables for the different interfaces. However since you don't use WiFi and all links appear perfect, the same router is often chosen for the different interfaces and the bonding has not enough candidates to choose and do round robin. I've sent an experimental patch "[RFC] batman-adv: experimental sysfs variable to always apply half duplex penalty" which allows to turn on the interface / half duplex penalty for all interfaces, not just wifi interfaces. That patch will most probably not make it upstream, but should be good enough for testing. Enable it with:
echo 1 > /sys/devices/virtual/net/bat0/mesh/always_half_duplex_penalty
Please apply these two patches, set the new option and try again!
[...]
Regardless of its potential impact on performance, I think it's worth a look and test to see if there may be some potential benefit. Currently this setup is working nice in that it's a true case of point to point redundant links. If additional links could be added or removed at will to enhance throughput I would see this useful in a variety of applications. Currently, I'd like to just reproduce what the function was originally designed to do and then go from there.
OK, sure, it can't hurt to try it out - maybe you can even find ways to optimize it. In any case, we would be happy to hear about your test results and experiences. :)
Thanks, Simon
b.a.t.m.a.n@lists.open-mesh.org