Xen Hvm Multi Home Different Subnet, Guest Multi Home Different Subnet Crap!

Setup:

Centos 5.8 HVM Xen (Use Default Installation)

My ethernets are eth1 and eth2 since somethings wrong with eth0 so change eth numbers to your needs

1. Edit Xen Config : xend-config.sxp
edit the network script to (network-script network-xen-multi-bridge)

2. Copy network-bridge to network-bridge.xen in scripts folder

---- network-xen-multi-bridge script----



#!/bin/sh
# network-xen-multi-bridge
# Exit if anything goes wrong.
set -e
# First arg is the operation.
OP=$1
shift
script=/etc/xen/scripts/network-bridge.xen
case ${OP} in
start)
        $script start vifnum=1 bridge=xenbr1 netdev=eth1
        $script start vifnum=2 bridge=xenbr2 netdev=eth2
        ;;
stop)
        $script stop vifnum=1 bridge=xenbr1 netdev=eth1
        $script stop vifnum=2 bridge=xenbr2 netdev=eth2
        ;;
status)
        $script status vifnum=1 bridge=xenbr1 netdev=eth1
        $script status vifnum=2 bridge=xenbr2 netdev=eth2
        ;;
*)
        echo 'Unknown command: ' ${OP}
        echo 'Valid commands are: start, stop, status'
        exit 1
esac

-------------------------

Restart

-------------------------

First VM created is ubuntu see config below:

name = "ubuntu"
uuid = "711a0907-4f18-5398-c2aa-bf94b7175f28"
maxmem = 1024
memory = 1024
vcpus = 2
builder = "hvm"
kernel = "/usr/lib/xen/boot/hvmloader"
boot = "c"
pae = 1
acpi = 1
apic = 1
localtime = 0
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
device_model = "/usr/lib64/xen/bin/qemu-dm"
sdl = 0
vnc = 1
vncunused = 1
keymap = "en-us"
disk = [ "file:/var/lib/xen/images/ubuntu.img,hda,w", ",hdc:cdrom,r" ]

vif = [ "mac=00:16:3e:00:00:10,bridge=xenbr2,script=vif-bridge","mac=00:16:3e:00:00:11,bridge=xenbr1,script=vif-bridge" ]

parallel = "none"
serial = "pty"

---------------------------------------------------------------------------------------------
The hard part:

Creating a vm that will do dual public ip address my interface config below:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.1.1
        netmask 255.255.255.0
        gateway 192.168.1.254

auto eth1
iface eth1 inet static
        address 192.168.2.1
        netmask 255.255.255.0
        gateway 192.168.2.254

======================================================

Do a dual wan thanks to http://chris.olstrom.com/howto/setup-dual-wan/ below is his work:

Simple Configuration
 Add two lines to /etc/iproute2/rt_tables

1 first
2 second

And then set up the routing for those tables.

# ip route add 192.168.1.0/24 dev eth0 src 192.168.1.254 table first
# ip route add default via 192.168.1.254 table first
# ip route add 192.168.2.0/24 dev eth1 src 192.168.2.254 table second
# ip route add default via 192.168.2.254 table second
# ip rule add from 192.168.1.254 table first
# ip rule add from 192.168.2.254 table second

Set up evenly weighted round-robin routing for the interface.

# ip route add default scope global nexthop via 192.168.1.254 dev eth0 weight 1 nexthop via 192.168.2.254 dev eth1 weight 1

Fixes and workarounds
  “RTNETLINK answers: File exists” error, replace the last entry

# ip route append default scope global nexthop via 192.168.1.254 dev eth0 weight 1 nexthop via 192.168.2.254 dev eth1 weight 1

Then remove the earlier route:

# ip route del

Alternatively, omitting both

# ip route add default via 192.168.1.254 table first
# ip route add default via 192.168.2.254 table second