[doc] master: Merge branch 'backup-redmine/2020-02-08' (cee3a86)
by postmaster@open-mesh.org
Repository : ssh://git@open-mesh.org/doc
On branch : master
>---------------------------------------------------------------
commit cee3a86f22e98abd854674d34e1990d627a21624
Merge: cbe4884 a6a1d67
Author: Sven Eckelmann <sven(a)narfation.org>
Date: Sat Feb 8 21:05:19 2020 +0100
Merge branch 'backup-redmine/2020-02-08'
>---------------------------------------------------------------
cee3a86f22e98abd854674d34e1990d627a21624
open-mesh/Kernel_hacking_Debian_image.rst | 1 +
1 file changed, 1 insertion(+)
diff --cc open-mesh/Kernel_hacking_Debian_image.rst
index 652104c,0000000..0797814
mode 100644,000000..100644
--- a/open-mesh/Kernel_hacking_Debian_image.rst
+++ b/open-mesh/Kernel_hacking_Debian_image.rst
@@@ -1,430 -1,0 +1,431 @@@
+.. SPDX-License-Identifier: GPL-2.0
+
+Kernel hacking Debian image
+===========================
+
+The :doc:`OpenWrt image <OpenWrt_in_QEMU>` is an easy way to start multiple
+virtual instances. But these instances usually don’t provide the
+required infrastructure to test kernel modules extensively. And it also
+depends on special toolchains to prepare the used tools/modules which
+should tested.
+
+It is often easier to use the same operating system in the virtual
+environment and on the host. Only the kernel is modified here to provide
+the necessary helpers for in-kernel development.
+
+An interested reader might even extend this further to only provide a
+modified kernel and use the currently running rootfs also in the virtual
+environment. Such an approach is used in `hostap’s test
+vm <https://w1.fi/cgit/hostap/tree/tests/hwsim/vm>`__ but it is out of
+scope for this document.
+
+Create an Image
+---------------
+
+The debian root filesystem is used here to a minimal system to boot and
+run the test programs. It is a simple ext4 filesystem with only
+userspace components from Debian. The configuration is changed to:
+
+* automatically mount the shared folder
+* automatically set up a static IPv4 address and hostname on bootup
+* start a test-init.sh script from the shared folder on bootup
+* disable root password
+* prefer batctl binary from shared folder’s batctl subdirectory instead
+ of virtual environment binary
+
+The installation is also cleaned up at the end to reduce the required
+storage space
+
+.. code-block:: sh
+
+ qemu-img create debian.img 8G
+ sudo mkfs.ext4 -O '^has_journal' -F debian.img
+ sudo mkdir debian
+ sudo mount -o loop debian.img debian
+ sudo debootstrap buster debian
+ sudo chroot debian apt update
+ sudo chroot debian apt install --no-install-recommends build-essential vim openssh-server less \
+ pkg-config libnl-3-dev libnl-genl-3-dev libcap-dev tcpdump \
+ trace-cmd flex bison libelf-dev libdw-dev binutils-dev libunwind-dev libssl-dev libslang2-dev liblzma-dev libperl-dev
+
+ sudo mkdir debian/root/.ssh/
+ ssh-add -L | sudo tee debian/root/.ssh/authorized_keys
+
+ sudo mkdir debian/host
+ sudo sh -c 'cat > debian/etc/fstab << EOF
+ host /host 9p trans=virtio,version=9p2000.L 0 0
+ EOF'
+
+ sudo sh -c 'cat > debian/etc/rc.local << "EOF"
+ #!/bin/sh -e
+
+ MAC_PART="$(ip link show enp0s1 | awk "/ether/ {print \$2}"| sed -e "s/.*://" -e "s/[\\n\\ ].*//"|awk "{print (\"0x\"\$1)*1 }")"
+ IP_PART="$(echo $MAC_PART|awk "{ print \$1+50 }")"
+ NODE_NR="$(echo $MAC_PART|awk "{ printf(\"%02d\", \$1) }")"
+ ip addr add 192.168.251.${IP_PART}/24 dev enp0s1
+ ip link set up dev enp0s1
+ hostname "node"$NODE_NR
+ ip link set up dev lo
+ [ ! -x /host/test-init.sh ] || /host/test-init.sh
+ exit 0
+ EOF'
+ sudo chmod a+x debian/etc/rc.local
+
+ sudo sed -i 's/^root:[^:]*:/root::/' debian/etc/shadow
+
+ ## optionally: allow ssh logins without passwords
+ # sudo sed -i 's/^#PermitRootLogin.*/PermitRootLogin yes/' debian/etc/ssh/sshd_config
+ # sudo sed -i 's/^#PermitEmptyPasswords.*/PermitEmptyPasswords yes/' debian/etc/ssh/sshd_config
+ # sudo sed -i 's/^UsePAM.*/UsePAM no/' debian/etc/ssh/sshd_config
+
+ ## optionally: enable autologin for user root
+ #sudo mkdir debian/etc/systemd/system/serial-getty(a)hvc0.service.d/
+ #sudo sh -c 'cat > debian/etc/systemd/system/serial-getty(a)hvc0.service.d/autologin.conf << EOF
+ #[Service]
+ #ExecStart=
+ #ExecStart=-/sbin/agetty --autologin root -s %I 115200,38400,9600 vt102
+ #EOF'
+
+ sudo sh -c 'echo '\''PATH="/host/batctl/:$PATH"'\'' >> debian/etc/profile'
+ sudo rm debian/var/cache/apt/archives/*.deb
+ sudo rm debian/var/lib/apt/lists/*
+ sudo e4defrag -v debian/
+ sudo umount debian
+ sudo zerofree -v debian.img
++ sudo fallocate --dig-holes debian.img
+
+
+ ## optionally: convert image to qcow2
+ #sudo qemu-img convert -c -f raw -O qcow2 debian.img debian.qcow2
+ #sudo mv debian.qcow2 debian.img
+
+Kernel compile
+--------------
+
+Any recent kernel can be used for the setup. We will use linux-next here
+to get the most recent development kernels. It is also assumed that the
+sources are copied to the same directory as the debian.img and a x86_64
+image will be used.
+
+The kernel will be build to enhance the virtualization and debugging
+experience. It is configured with:
+
+* basic kernel features
+* support for necessary drivers
+* kernel hacking helpers
+* kernel address + undefined sanitizers
+* support for hwsim
+
+.. code-block:: sh
+
+ # make sure that libelf-dev is installed or module build will fail with something like "No rule to make target 'net/batman-adv/bat_algo.o'"
+
+ git clone git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
+ cd linux-next
+
+ make allnoconfig
+ cat >> .config << EOF
+
+ # small configuration
+ CONFIG_SMP=y
+ CONFIG_EMBEDDED=n
+ # CONFIG_EXPERT is not set
+ CONFIG_MODULES=y
+ CONFIG_MODULE_UNLOAD=y
+ CONFIG_MODVERSIONS=y
+ CONFIG_MODULE_SRCVERSION_ALL=y
+ CONFIG_64BIT=y
+ CONFIG_X86_VSYSCALL_EMULATION=n
+ CONFIG_IA32_EMULATION=n
+ CONFIG_VOP_BUS=y
+ CONFIG_VOP=y
+ CONFIG_HW_RANDOM_VIRTIO=y
+ CONFIG_NET_9P_VIRTIO=y
+ CONFIG_VIRTIO_MENU=y
+ CONFIG_SCSI_VIRTIO=y
+ CONFIG_VIRTIO_BALLOON=y
+ CONFIG_VIRTIO_BLK=y
+ CONFIG_VIRTIO_CONSOLE=y
+ CONFIG_VIRTIO_INPUT=y
+ CONFIG_VIRTIO_NET=y
+ CONFIG_VIRTIO_PCI=y
+ CONFIG_VIRTIO_PCI_LEGACY=y
+ CONFIG_VIRTIO_BALLOON=y
+ CONFIG_VIRTIO_BLK_SCSI=y
+ CONFIG_VIRTIO_INPUT=y
+ CONFIG_VIRTIO_MMIO=y
+ CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
+ CONFIG_RPMSG_VIRTIO=y
+ CONFIG_VSOCKETS=y
+ CONFIG_VIRTIO_VSOCKETS=y
+ CONFIG_DRM=y
+ CONFIG_DRM_VIRTIO_GPU=y
+ CONFIG_CAIF=y
+ CONFIG_CAIF_VIRTIO=y
+ CONFIG_CRYPTO_DEV_VIRTIO=y
+ CONFIG_FUSE_FS=y
+ CONFIG_VIRTIO_FS=y
+ CONFIG_IOMMU_SUPPORT=y
+ CONFIG_VIRTIO_IOMMU=y
+ CONFIG_LIBNVDIMM=y
+ CONFIG_VIRTIO_PMEM=y
+ CONFIG_CRC16=y
+ CONFIG_LIBCRC32C=y
+ CONFIG_CRYPTO_SHA512=y
+ CONFIG_NET=y
+ CONFIG_INET=y
+ CONFIG_DEBUG_FS=y
+ CONFIG_IPV6=y
+ CONFIG_BRIDGE=y
+ CONFIG_VLAN_8021Q=y
+ CONFIG_WIRELESS=n
+ CONFIG_NET_9P=y
+ CONFIG_NETWORK_FILESYSTEMS=y
+ CONFIG_9P_FS=y
+ CONFIG_9P_FS_POSIX_ACL=y
+ CONFIG_9P_FS_SECURITY=y
+ CONFIG_BLOCK=y
+ CONFIG_BLK_DEV=y
+ CONFIG_EXT4_FS=y
+ CONFIG_EXT4_USE_FOR_EXT23=y
+ CONFIG_TTY=y
+ CONFIG_HW_RANDOM=y
+ CONFIG_VHOST_RING=y
+ CONFIG_GENERIC_ALLOCATOR=y
+ CONFIG_SCSI_LOWLEVEL=y
+ CONFIG_SCSI=y
+ CONFIG_NETDEVICES=y
+ CONFIG_NET_CORE=y
+ CONFIG_DEVTMPFS=y
+ CONFIG_HYPERVISOR_GUEST=y
+ CONFIG_PVH=y
+ CONFIG_PARAVIRT=y
+ CONFIG_PARAVIRT_TIME_ACCOUNTING=y
+ CONFIG_PARAVIRT_SPINLOCKS=y
+ CONFIG_KVM_GUEST=y
+ CONFIG_BINFMT_ELF=y
+ CONFIG_BINFMT_SCRIPT=y
+ CONFIG_BINFMT_MISC=y
+ CONFIG_PCI=y
+ CONFIG_SYSVIPC=y
+ CONFIG_POSIX_MQUEUE=y
+ CONFIG_CROSS_MEMORY_ATTACH=y
+ CONFIG_UNIX=y
+ CONFIG_TMPFS=y
+ CONFIG_CGROUPS=y
+ CONFIG_BLK_CGROUP=y
+ CONFIG_CGROUP_CPUACCT=y
+ CONFIG_CGROUP_DEVICE=y
+ CONFIG_CGROUP_FREEZER=y
+ CONFIG_CGROUP_HUGETLB=y
+ CONFIG_CGROUP_NET_CLASSID=y
+ CONFIG_CGROUP_NET_PRIO=y
+ CONFIG_CGROUP_PERF=y
+ CONFIG_CGROUP_SCHED=y
+ CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
+ CONFIG_INOTIFY_USER=y
+ CONFIG_FHANDLE=y
+ CONFIG_CFG80211=y
+ CONFIG_DUMMY=y
+ CONFIG_PACKET=y
+ CONFIG_VETH=y
+ CONFIG_IP_MULTICAST=y
+ CONFIG_NET_IPGRE_DEMUX=y
+ CONFIG_NET_IP_TUNNEL=y
+ CONFIG_NET_IPGRE=y
+ CONFIG_NET_IPGRE_BROADCAST=y
+ # CONFIG_LEGACY_PTYS is not set
+ CONFIG_NO_HZ_IDLE=y
+ CONFIG_CPU_IDLE_GOV_HALTPOLL=y
+ CONFIG_PVPANIC=y
+
+ # makes boot a lot slower but required for shutdown
+ CONFIG_ACPI=y
+
+
+ #debug stuff
+ CONFIG_CC_STACKPROTECTOR_STRONG=y
+ CONFIG_LOCKUP_DETECTOR=y
+ CONFIG_DETECT_HUNG_TASK=y
+ CONFIG_SCHED_STACK_END_CHECK=y
+ CONFIG_DEBUG_RT_MUTEXES=y
+ CONFIG_DEBUG_SPINLOCK=y
+ CONFIG_DEBUG_MUTEXES=y
+ CONFIG_PROVE_LOCKING=y
+ CONFIG_LOCK_STAT=y
+ CONFIG_DEBUG_LOCKDEP=y
+ CONFIG_DEBUG_ATOMIC_SLEEP=y
+ CONFIG_DEBUG_LIST=y
+ CONFIG_DEBUG_PI_LIST=y
+ CONFIG_DEBUG_SG=y
+ CONFIG_DEBUG_NOTIFIERS=y
+ CONFIG_PROVE_RCU_REPEATEDLY=y
+ CONFIG_SPARSE_RCU_POINTER=y
+ CONFIG_DEBUG_STRICT_USER_COPY_CHECKS=y
+ CONFIG_X86_VERBOSE_BOOTUP=y
+ CONFIG_DEBUG_RODATA=y
+ CONFIG_DEBUG_RODATA_TEST=n
+ CONFIG_DEBUG_SET_MODULE_RONX=y
+ CONFIG_PAGE_EXTENSION=y
+ CONFIG_DEBUG_PAGEALLOC=y
+ CONFIG_DEBUG_OBJECTS=y
+ CONFIG_DEBUG_OBJECTS_FREE=y
+ CONFIG_DEBUG_OBJECTS_TIMERS=y
+ CONFIG_DEBUG_OBJECTS_WORK=y
+ CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
+ CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER=y
+ CONFIG_DEBUG_KMEMLEAK=y
+ CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=8000
+ CONFIG_DEBUG_STACK_USAGE=y
+ CONFIG_DEBUG_STACKOVERFLOW=y
+ CONFIG_DEBUG_INFO=y
+ CONFIG_DEBUG_INFO_DWARF4=y
+ CONFIG_GDB_SCRIPTS=y
+ CONFIG_READABLE_ASM=y
+ CONFIG_STACK_VALIDATION=y
+ CONFIG_WQ_WATCHDOG=y
+ CONFIG_DEBUG_KOBJECT_RELEASE=y
+ CONFIG_DEBUG_WQ_FORCE_RR_CPU=y
+ CONFIG_OPTIMIZE_INLINING=y
+ CONFIG_ENABLE_MUST_CHECK=y
+ CONFIG_ENABLE_WARN_DEPRECATED=y
+ CONFIG_DEBUG_SECTION_MISMATCH=y
+ CONFIG_UNWINDER_ORC=y
+ CONFIG_FTRACE=y
+ CONFIG_FUNCTION_TRACER=y
+ CONFIG_FUNCTION_GRAPH_TRACER=y
+ CONFIG_FTRACE_SYSCALLS=y
+ CONFIG_TRACER_SNAPSHOT=y
+ CONFIG_TRACER_SNAPSHOT_PER_CPU_SWAP=y
+ CONFIG_STACK_TRACER=y
+ CONFIG_UPROBE_EVENTS=y
+ CONFIG_DYNAMIC_FTRACE=y
+ CONFIG_FUNCTION_PROFILER=y
+ CONFIG_HIST_TRIGGERS=y
+
+ # for GCC 5+
+ CONFIG_KASAN=y
+ CONFIG_KASAN_INLINE=y
+ CONFIG_UBSAN_SANITIZE_ALL=y
+ CONFIG_UBSAN=y
+ CONFIG_UBSAN_NULL=y
+ EOF
+ make olddefconfig
+
+ cat >> .config << EOF
+ # allow to use unsigned regdb with hwsim
+ CONFIG_EXPERT=y
+ CONFIG_CFG80211_CERTIFICATION_ONUS=y
+ # CONFIG_CFG80211_REQUIRE_SIGNED_REGDB is not set
+ EOF
+ make olddefconfig
+
+ make all -j$(nproc || echo 1)
+
+Build the BIOS
+--------------
+
+The (sea)bios used by qemu is nice to boot all kind of legacy images but
+reduces the performance for booting a paravirtualized Linux system.
+Something like qboot works better for this purpose:
+
+.. code-block:: sh
+
+ git clone https://github.com/bonzini/qboot.git
+ make -C qboot
+
+.. _open-mesh-kernel-hacking-debian-image-building-the-batman-adv-module:
+
+Building the batman-adv module
+------------------------------
+
+The kernel module can be build outside the virtual environment and
+shared over the 9p mount. The path to the kernel sources have to be
+provided to the make process
+
+.. code-block:: sh
+
+ make KERNELPATH="$(pwd)/../linux-next"
+
+The kernel module can also be compiled in a way which creates better
+stack traces and increases the usability with (k)gdb:
+
+.. code-block:: sh
+
+ make EXTRA_CFLAGS="-fno-inline -Og -fno-optimize-sibling-calls" KERNELPATH="$(pwd)/../linux-next" V=1
+
+Start of the environment
+------------------------
+
+virtual network initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The
+:ref:`virtual-network.sh from the OpenWrt environment <open-mesh-openwrt-in-qemu-virtual-network-initialization>`
+can be reused again.
+
+VM instances bringup
+~~~~~~~~~~~~~~~~~~~~
+
+The
+:ref:`run.sh from the OpenWrt environment <open-mesh-openwrt-in-qemu-vm-instances-bringup>`
+can mostly be reused. There are only minimal
+adjustments required.
+
+The BASE_IMG is of course no longer the same because a new image
+“debian.img” was created for our new environment. The image also doesn’t
+contain a bootloader or kernel anymore. The kernel must now be supplied
+manually to qemu.
+
+.. code-block:: sh
+
+ BASE_IMG=debian.img
+ BOOTARGS+=("-bios" "qboot/bios.bin")
+ BOOTARGS+=("-kernel" "linux-next/arch/x86/boot/bzImage")
+ BOOTARGS+=("-append" "root=/dev/vda rw console=hvc0 nokaslr tsc=reliable no_timer_check noreplace-smp rootfstype=ext4 rcupdate.rcu_expedited=1 reboot=t pci=lastbus=0 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1")
+ BOOTARGS+=("-device" "virtconsole,chardev=charconsole0,id=console0")
+
+It is also recommended to use linux-next/vmlinux instead of bzImage with
+qemu 4.0.0 (or later)
+
+Automatic test initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The
+:ref:`test-init.sh from the OpenWrt environment <open-mesh-openwrt-in-qemu-automatic-test-initialization>`
+is always test specific. But its main
+functionality is still the same as before. A simple example would be:
+
+.. code-block:: sh
+
+ cat > test-init.sh << "EOF"
+ #! /bin/sh
+
+ set -e
+
+ ## get internet access
+ dhclient enp0s2
+
+ ## Simple batman-adv setup
+
+ # ip link add dummy0 type dummy
+ ip link set up dummy0
+
+ rmmod batman-adv || true
+ insmod /host/batman-adv/net/batman-adv/batman-adv.ko
+ /host/batctl/batctl routing_algo BATMAN_IV
+ /host/batctl/batctl if add dummy0
+ /host/batctl/batctl it 5000
+ /host/batctl/batctl if add enp0s1
+ ip link set up dev enp0s1
+ ip link set up dev bat0
+ EOF
+
+ chmod +x test-init.sh
+
+Start
+-----
+
+The startup method
+:ref:`from the OpenWrt environment <open-mesh-openwrt-in-qemu-start>`
+should be used here.