Hi,

             Good Afternoon from Boston. I really love Batman-Adv ...

brilliant layer 2 functionality.

I want to use batman-adv in a wired (gigabit and 10-gigabit) only mesh and wanted to know your insights.

makes me happy to hear you love our project. Typically, we communicate via our public mailing list allowing various sources to chime in at any point. Since I don't see any reason for privacy I am cc'ing the mailing list in my answer.

The example case scenario is as follows:

1. 4 to 6 AMD servers with 6 10-Gigabit NICs each.
   
   

2. 2 or 3 10-Gigabit NICs used for batman-adv, which are then connected
   

in ring or torus topology directly (no external switch involved)

3. the remaining interfaces on the server are connected to the LAN
   

(switches, routers etc)

4. the virtual machine (qemu-kvm) tap interfaces, the physical
   

non-batman-adv ethernet and bat0 interfaces are put in a bridge (brctl), so now we have the ability for virtual machines, wired hosts on the lan to go via batman-adv and talk to each other.

Is there any, down size to doing this? I see at the most 2 - 100 servers in one network....

From what i understand:

1. that the live migration of virtual machines (qemu-kvm) will be seen

just as a migrating non-mesh client so my assumption is that live migration should work from that perspective. Also, what if the tap interfaces of the virtual machines are given to bat0 itself (if it might help in live migration / increasing throughput) ?

2. The MTU if set for 1500 or 9000 or higher (eg barman-adv reads --

"define ETHERMTU ETH_DATA_LEN") would be taken automatically by batman-adv and anything below 1500 would be fragmented, which gives me the idea that higher MTUs would not be a problem for batman-adv to handle.

3. There is no restriction to the number of clients in batman-adv.

am i somewhat close in understanding batman-adv? .... apologies if not...

Also would layer 2 forwarding by batman-adv would be close, same or better when compared to bridge (linux brctl) packet forwarding?

I have built converged-unified distributed qemu-kvm system (all metadata less design, with web-interface and cli, quite the opposite of vmware and open-stack type centralized approaches) and was in the preliminary stage of looking at the possibility of integrating batman-adv into the design.

Your input will be valuable for me to give server and desktop virtualization a mesh architecture on top of already distributed design.

I keep your description intact to allow other people to comment as well.

Before we dive into the batman-adv details I'd like to understand what advantage batman-adv brings to the table in your scenario. The batman-adv project aims to facilitate layer2 routing in primarily wireless setups with dynamically changing links due to link quality changes or links being modified in an uncontrolled fashion (community mesh network). While batman-adv also is able to run on wired backbones this never was the main target and bears a number of drawbacks compared to other technologies. A simple example to picture this: The standard Linux bridge (configurable via brctl) does not run any link layer protocol to estimate the quality of one link compared to another. This will give you huge advantages in terms of overhead with the cost of all links being treated equal. While this work fine on an all-wired setup it represents an unacceptable trade-off for wireless networks.

From what I can gather you are not running wireless but high throughput wired links. What has brought you to batman-adv ?

Cheers, Marek