VM migration call stack (libvirt and qemu-KVM)

libvirt-0.7.7/tools/virsh.c: cmdMigrate() -> virDomainMigrate() -> virDomainMigrateVersion2() -> domain->conn->driver->domainMigratePerform() ->

libvirt-0.7.7/src/qemu/qemu_driver.c: qemudDomainMigratePerform() -> doNativeMigrate() -> qemuMonitorMigrateToHost() -> qemuMonitorTextMigrateToHost() -> qemuMonitorCommandWithHandler:230:
Send command 'migrate -d "tcp:tsurugi8.il.is.s.u-tokyo.ac.jp:49157"'

qemu-kvm-0.12.3/migration.c: do_migrate() -> tcp_start_outgoing_migration() -> migrate_fd_connect()

migrate_fd_connect():

qemu_fopen_ops_buffered(): creates a QEMUFileBuffered object and sets up a timer in qemu_new_timer(), where buffered_rate_tick() is the actual callback function.

buffered_rate_tick(): sets the new timer deadline and calls the QEMUFileBuffered obj's put_buffer() and put_ready() functions that were registered as migrate_fd_put_buffer() and migrate_fd_put_ready() respectively.

migrate_fd_put_ready(): calls qemu_savevm_state_iterate() that iterates the savevm_handlers and calls save_live_state() on each.

two live save handlers are registered: ram_save_live() and block_save_live() for the memory and the disk respectively.

ram_save_live():

KVM bridge network + VM live migration

/etc/network/interfaces on the host machine:
#auto eth0
#iface eth0 inet dhcp

auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0

domain description XML for the guest (interface bridge is the point!):

<domain type='kvm' id='3'>
  <name>karmic_qemu</name>
  <uuid>11111111-50c2-1590-c9fc-0d3afa1c1b97</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-0.11'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='cdrom'>
      <target dev='hdc' bus='ide'/>
      <readonly/>
    </disk>
    <disk type='file' device='disk'>
      <source file='/home/bgerofi/qemu-images/karmic.qcow2'/>
      <target dev='vda' bus='virtio'/>
    </disk>
        <interface type='bridge'>
                <source bridge='br0'/>
                <mac address="00:6E:01:69:3A:11"/>
                <model type='virtio'/>
        </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
    </video>
  </devices>
</domain>

/etc/network/interfaces on the guest machine (for tsurugi network):

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
iface eth1 inet static
address 172.16.16.99
netmask 255.255.0.0
gateway 172.16.0.1

(I have no clue why it doesn't show up as eth0...)

on both machines we need to set up sasl:
bgerofi@tsurugi8:~$ sudo saslpasswd2 -a libvirt bgerofi -f /etc/libvirt/passwd.db
(type pasword)

(Note: it is possible you need to add the user/pass to /etc/sasldb2 as well!)

bgerofi@tsurugi8:~$ sudo sasldblistusers2 -f /etc/libvirt/passwd.db
bgerofi@tsurugi8: userPassword


edit libvirt.conf so that it uses TCP and no TSL:
bgerofi@tsurugi8:~$ sudo vim /usr/local/etc/libvirt/libvirtd.conf

# This is enabled by default, uncomment this to disable it
listen_tls = 0 <<<----- THIS CHANGED

# Listen for unencrypted TCP connections on the public TCP/IP port.
# NB, must pass the --listen flag to the libvirtd process for this to
# have any effect.
#
# Using the TCP socket requires SASL authentication by default. Only
# SASL mechanisms which support data encryption are allowed. This is
# DIGEST_MD5 and GSSAPI (Kerberos5)
#
# This is disabled by default, uncomment this to enable it.
listen_tcp = 1 <<<----- THIS CHANGED


start libvirtd in listening mode:
root@tsurugi8:~# libvirtd --listen


live migration:
bgerofi@tsurugi7:~$ sudo virsh list --all
Id Name State
----------------------------------
1 karmic_qemu running

bgerofi@tsurugi7:~$ sudo virsh migrate --live karmic_qemu qemu+tcp://tsurugi8/system
Please enter your authentication name: bgerofi
Please enter your password:

bgerofi@tsurugi7:~$ sudo virsh list --all
Id Name State
----------------------------------
- karmic_qemu shut off

what we got on tsurugi8:
bgerofi@tsurugi8:~$ sudo virsh list
Id Name State
----------------------------------
1 karmic_qemu running



QEMU log file: /usr/local/var/log/libvirt/qemu/karmic_qemu.log

replication:
sudo virsh replicate karmic_qemu qemu+tcp://tsurugi8/system