Repository : ssh://git@open-mesh.org/doc

On branches: backup-redmine/2019-11-07,master

commit a7e40aba3effa4151a337da1f01635743bb2a1a3 Author: Sven Eckelmann sven@narfation.org Date: Sun Oct 27 14:41:25 2019 +0000

doc: open-mesh/Vde_switch_virtual_network

a7e40aba3effa4151a337da1f01635743bb2a1a3 open-mesh/Vde_switch_virtual_network.textile | 191 +++++++-------------------- 1 file changed, 47 insertions(+), 144 deletions(-)

diff --git a/open-mesh/Vde_switch_virtual_network.textile b/open-mesh/Vde_switch_virtual_network.textile index b75777d..7a24312 100644 --- a/open-mesh/Vde_switch_virtual_network.textile +++ b/open-mesh/Vde_switch_virtual_network.textile @@ -1,87 +1,39 @@ -h1. Emulation Setup +h1. vde_switch interconnect

-To give an answer to the often asked question "How to test/evaluate/debug mesh network protocols?", this document explains a virtual machine setup which can be used to test batman-adv in a controlled environment. The idea is to use "QEMU":http://www.qemu.org/ or "KVM":http://www.linux-kvm.org (instead of pure simulation systems like "NS-2":http://www.isi.edu/nsnam/ns/) to run an unmodified Linux system with the unmodified source code as it is used in real world setups. Besides B.A.T.M.A.N., you could evaluate any routing protocol. +The simple interconnect from [[open-mesh:OpenWrt in QEMU#interconnect-initialization]] is a simple bridge which allows full communication between all devices. This is not optimal to create not fully meshed setups. One alternative is to use a modified vde_switch to create a user defined links between each virtual instance.

-h2. Architecture +h2. Components

The test stack consists of the following components:

-* "OpenWrt":https://openwrt.org/, any recent stable release for x86 with minimal modifications (see below) -* one (unmodified) "QEMU":http://www.qemu.org/ instance per host (KVM or any other virtual machine) * one (modified, see below) "vde_switch":http://wiki.virtualsquare.org/wiki/index.php/VDE_Basic_Networking per instance. You need multiple instances to allow interconnection with wirefilter. * one (unmodified) "wirefilter":http://wiki.virtualsquare.org/wiki/index.php/VDE#wirefilter per link between hosts.

+h2. vde setup

-h3. OpenWrt - -A standard OpenWrt can be downloaded and configured for X86. Some packages (tcpdump, netcat, batman-adv...) should be selected and kernel commandline parameters were modified to make sure that qemu boots up properly: -<pre> -noapic acpi=off -</pre> - -Furthermore, to enjoy an automatic network device setup on boot you can use the following script (save it as './files/etc/rc.local' in your local OpenWrt build directory): - -<pre> -#!/bin/sh - -# kill default openwrt network config -NUM=$(ip link show eth0 | awk '/ether/ {print $2}'| sed -e 's/.*://' -e 's/[\n\ ].*//') -ip addr add 192.168.$((0x$NUM)).2/24 dev eth1 -ip link set up dev eth1 -ip link set down dev br-lan -ip link del dev br-lan - - -# setup batman, this step should be done before setting up the log server, otherwise it'll can't find log file -ip link set up dev eth0 -echo "" > /proc/net/batman-adv/interfaces -echo eth0 > /proc/net/batman-adv/interfaces -echo 15 > /proc/net/batman-adv/log_level -ip addr add 192.168.0.${NUM}/24 dev bat0 -ip link set up dev bat0 - -# set up log server -cat << EOF > /tmp/logserver.sh -#!/bin/sh -while [ 1 ]; do - trace-cmd stream -e batadv:batadv_dbg | nc -l -p 2050 -done -EOF -chmod 755 /tmp/logserver.sh -/tmp/logserver.sh& - -</pre> - -The script above is far from perfect, it takes some minutes after the rc.local file is finally started. Suggestions welcome. - -h3. prepare a working directory for your operations - -$mkdir qemu_work -$cd qemu_work - -h3. vde setup - -Download vde2.3.1: https://sourceforge.net/projects/vde/files/vde2/ -Download vde colour patch: attachment:vde2-2.3.1_colour.patch +Download "vde2.3.1":https://sourceforge.net/projects/vde/files/vde2/ and the vde colour patch attachment:vde2-2.3.1_colour.patch

h3. patch the "vde2-2.3.1_colour.patch" for vde.

-$ cd vde-2.3.1 -$ cp vde2-2.3.1_colour.patch . -$ patch -p1 < vde2-2.3.1_colour.patch +<pre><code class="shell"> +cd vde-2.3.1 +cp vde2-2.3.1_colour.patch . +patch -p1 < vde2-2.3.1_colour.patch +</code></pre>

h3. compile and install vde

-$ cd vde-2.3.1 -$ ./configure -$ make -$ sudo make install +<pre><code class="shell"> +cd vde-2.3.1 +./configure +make +sudo make install +</code></pre>

-h3. download and cp colourful.rc +h3. Create colourful.rc

-colourful.rc: - -<pre> +<pre><code class="shell"> +cat > colourful.rc << "EOF" port/setcolourful 1 port/create 1 port/create 2 @@ -89,15 +41,13 @@ port/create 3 port/create 4 port/create 5 port/setcolour 1 1 -</pre> - +EOF +</code></pre>

-$cp colourful.rc qemu_work

h3. vde_switch

-You can get detailed vde_switch parameters documentation here: http://www.linuxhowtos.org/manpages/1/vde_switch.htm -The main advantage of "vde_switch":http://wiki.virtualsquare.org/wiki/index.php/VDE_Basic_Networking over "uml_switch":http://user-mode-linux.sourceforge.net/old/networking.html is that any clients can be attached to this virtual switch: QEMU, KVM, UML, tap interfaces, virtual interconnections, and not just UML instances. +You should read the "detailed vde_switch parameters documentation":http://www.linuxhowtos.org/manpages/1/vde_switch.htm. The main advantage of "vde_switch":http://wiki.virtualsquare.org/wiki/index.php/VDE_Basic_Networking over "uml_switch":http://user-mode-linux.sourceforge.net/old/networking.html is that any clients can be attached to this virtual switch: QEMU, KVM, UML, tap interfaces, virtual interconnections, and not just UML instances.

If the vde_switches were just connected with wirefilter "patch cables" without modification, we would end up creating a broadcast domain and switch loops which we don't want: The goal is to allow the packets to travel only from one host to it's neighbor, not farther.

@@ -146,101 +96,54 @@ wirefilter --ether 0x4305:02 -l 50 You can specify up to 10 packet types (separated by colon). The patch against vde2-2.3.1 (current latest stable version) can be found here: * attachment:vde2-2.3.1-wirefilter-ethertype.patch

-copy openwrt-x86-generic-combined-ext4.img to your qemu_work directory. +h2. Start of the environment

-h2. Scripts - - -The following script is used to start up all the qemus. It is a good idea to start the script inside a screen to have the QEMU instances in screen windows (which can be switch with ctrl+a n, ctrl+a p). Make sure that you have the correct sudo priveleges or alternatively run this script as root. +h3. interconnect initialization

The script does: * kill old instances * start up vde_switch instances for each host -* start up QEMU hosts (one Ethernet tap device is created per instance to allow logging etc) * install the links between the hosts. The resulting topology will be similar to this: !mesh.gif!

-<pre> +<pre><code class="shell"> +cat > interconnect.sh << "EOF" #!/bin/sh -QEMU=qemu-system-x86 -VDESWITCH=vde_switch -IMAGE=openwrt-x86-generic-combined-ext4.img - -if [ "$TERM" != "screen" ]; -then -echo "Must be run inside a screen session" 1>&2 -exit 1 -fi

-# you can set this if you are running as root and don't need sudo: -# SUDO= -SUDO=sudo +NUM_SESSIONS=9 +VDESWITCH=vde_switch

-${SUDO} killall -q qemu killall -q wirefilter killall -q vde_switch

-for i in $(seq 1 9); -do - ${VDESWITCH} \ - -d --hub --sock num${i}.ctl -f colourful.rc - ${SUDO} ip tuntap add tapwrt${i} mode tap - ${SUDO} ip addr add 192.168.${i}.1/24 dev tapwrt${i} - ${SUDO} ip link set up dev tapwrt${i} +for i in $(seq 1 "${NUM_SESSIONS}"); do + ${VDESWITCH} -d --hub --sock num${i}.ctl -f colourful.rc done

-for i in $(seq 1 9); -do - cp ${IMAGE} num${i}.image - screen -dmS num${i} ${QEMU} \ - -no-acpi -m 32M \ - -net vde,sock=num${i}.ctl,port=1 -net nic,macaddr=fe:fe:00:00:$(printf %02x $i):01 \ - -net nic -net tap,ifname=tapwrt${i},script=no,downscript=no \ - -nographic num${i}.image - sleep 3 +for i in $(seq 1 $((${NUM_SESSIONS} - 1))); do + wirefilter --daemon -v num${i}.ctl:num$((${i} + 1)).ctl done

-wirefilter --daemon -v num1.ctl:num2.ctl -wirefilter --daemon -v num2.ctl:num3.ctl -wirefilter --daemon -v num3.ctl:num4.ctl -wirefilter --daemon -v num4.ctl:num5.ctl -wirefilter --daemon -v num5.ctl:num6.ctl -wirefilter --daemon -v num6.ctl:num7.ctl -wirefilter --daemon -v num7.ctl:num8.ctl -wirefilter --daemon -v num8.ctl:num9.ctl - -wirefilter --daemon -v num1.ctl:num3.ctl -l 60 -wirefilter --daemon -v num3.ctl:num5.ctl -l 60 -wirefilter --daemon -v num5.ctl:num7.ctl -l 60 -wirefilter --daemon -v num7.ctl:num9.ctl -l 60 -wirefilter --daemon -v num2.ctl:num4.ctl -l 60 -wirefilter --daemon -v num4.ctl:num6.ctl -l 60 -wirefilter --daemon -v num6.ctl:num8.ctl -l 60 -</pre> +for i in $(seq 1 $((${NUM_SESSIONS} - 2))); do + wirefilter --daemon -v num${i}.ctl:num$((${i} + 2)).ctl -l 60 +done +EOF

-About IMAGE variable: +chmod + interconnect.sh +</code></pre>

-create this script in the "qemu_work" directory that you have created before. -when you use openwrt chaos_calmer version, you should use openwrt-x86-generic-combined-ext4.img. +h3. VM instances bringup

-The script is using the switch configuration file "colourful.rc". It creates the ports (create more if your topology demands this) and sets the host port (port no. 1) colour to 1. Put the following text in this file: +The [[open-mesh:OpenWrt_in_QEMU#VM-instances-bringup|run.sh from the OpenWrt environment]] can mostly be reused. There are only minimal adjustments required. The interconnect tap NIC has to be replaced with a vde NIC.

-h3. About screen command +<pre><code class="diff"> +- -nic tap,ifname=tap$i,script=no,model=virtio-net-pci,mac=02:ba:de:af:fe:"${twodigit_id}" \ ++ -nic vde,sock=num${i}.ctl,port=1,script=no,model=virtio-net-pci,mac=02:ba:de:af:fe:"${twodigit_id}" \ +</code></pre>

-<pre> -#show all screen instances -$screen -ls +It is also possible to have both (just with different mac addresses) to have SSH access to the virtual instance over the tap interface.

-#you can login a specific openwrt system with following command -$screen -r num1 #(num1~num9) -</pre> +h3. Automatic test initialization

-To collect the batman logs from the individual hosts, you might want to use this script after all nodes have completed booting and started batman: +The [[open-mesh:OpenWrt_in_QEMU#Automatic-test-initialization|test-init.sh from the OpenWrt environment]] is always test specific. But it can just be reused again with this interconnect.

-<pre> -#!/bin/sh -for i in $(seq 1 9) -do - nc 192.168.${i}.2 2050 > num$i.log & -done -</pre> \ No newline at end of file