ubuntu conference 2007 talks

see talk on clonezilla and virtual machines.

qemu creation of images

How do you create your images? Do you want to have a BIG file or a SMALL file

qemu-img create debian.img  5G

this creates a large file
which is SPARSE thus is capable of being compressed
via
tar -cjSf debian.tar.bz2 debian.img

which can be moved.

On the other hand you can make a qcow small file to begin with.

qemu-img create debian.img -f qcow 5G

this is a small file.
And can make a copy file to use.

qemu-img create -b debian.img -f qcow stable.ovl

this second is a very tiny copy file, just stores the diffs.

qemu and vmware and kvm and live view

I have been playing with qemu. I have a vmware machine and now I would like to get a qemu version. When I get a new cpu with virtualization in hardware I can try out kvm as well.

Here is some nice stuff.
http://www.robertpeaslee.com/index.php/converting-a-physical-disk-to-a-virtual-disk-for-free/

He wrote:

I have a Windows XP SP2 installation that I use for work, while my current personal OS of choice is Windows Vista. I only work weekends, so rebooting wasn’t too bothersome initially. Over time it has become more and more of a chore, as I can’t easily switch from one development environment to another without rebooting. I decided it was time to make my work installation a virtual machine.

I had some additional complications that made the process a little non-standard. I originally tried using VMWare’s converter tool, but it would fail at 97% of the creation of the disk. I then tried using a Windows port of the Unix utility ‘dd’ to create a raw image of the disk, but because the Windows volume manager was accessing the disk, dd would give me access errors. Additionally, the VMWare converter doesn’t support converting from a raw image (…grr…), so Qemu’s qemu-img tool would have to be used to convert the raw image to a disk in VMWare’s vmdk format.

The steps to reach our goal aren’t too complicated, and can be replicated by others easily. To do it, I used:

• VMWare Workstation
• A Linux installation or LiveCD (I used my existing Debian installation, but something like Knoppix would work fine)
• Qemu
• NTFS-3G (if you plan on writing out to an NTFS partition from Linux, as I did)

I started by booting into my Linux install. Linux only mounts the disks it uses (hint hint, Microsoft) so we can access all sectors of the partition to make a dump of the disk with dd. I first had to mount the partition where I wanted the output file to reside, which uses NTFS:

ntfs-3g /dev/sdb1 /mnt/external

Next, create the image. I did this with the following (substitute your device/partition and output file):

dd if=/dev/hdc3 of=/mnt/external/diskImage/XPSP2.img bs=1024

When that finishes, the file specified with the ‘of’ option in dd will contain a block-by-block exact copy of your partition. However, it is in a raw format - we need it in a format VMWare can read. This is where Qemu comes in. Qemu is distributed with qemu-img, a tool used for creating, manipulating, and converting images. Specifically, our goal is to use qemu-img’s convert functionality to convert from a raw image format to the vmdk format. This is accomplished with:

qemu-img convert -f raw /mnt/external/diskImage/XPSP2.img -O vmdk /mnt/external/diskImage/XPSP2.vmdk

Be prepared to wait. For a 40gb image, this process took roughly 12 hours. Since qemu-img provides no status as to how far it has come, I kept tabs on it just by monitoring the filesize of the output image. This is entirely unnecessary, but if you want to do the same, just open a new terminal and type the following:

while [1 -gt 0 ]; do du -hs /mnt/external/diskImage/XPSP2.vmdk; sleep 10; clear; done

This will just print out the size of the file on your screen so you can watch it grow.

When the process is completed, boot back into Windows (or if you are using Linux as the host, stay put) and create a virtual machine around your new disk image. Don’t forget to remove the original img created with dd, it is a huge waste of disk space :)

[Notes]

1. This process is really only feasible if you have a lot of disk space. At worst, the disk requirements are greater than 2*P, where P is the partition size of the virtual machine you wish to create. However, qemu-img only writes out actual data, not empty sectors, so your output image will be the size of the used space in the input image. For my conversion, I used over 40gb (input)+15gb (output) of disk space, which was reclaimed with the deletion of the output of dd, and resizing another partition to use the old physical installation’s space.
2. qemu-img doesn’t support stream input, which is why we can’t pipe dd’s output directly into qemu-img convert. This would have reduced the disk requirements to only the size of the vmdk image, and sped up the process substantially. Bug the Qemu developers to implement this feature :)

then a comment said

1. u may be able to skip the dd command. Try this command:
   qemu-img convert -f raw /dev/hdc3 -O vmdk boot.vmdk

   I think this should work just fine. dd is just reading from the disk and qemu-img would not know the difference.

   I tested on /dev/hda1 (/boot) and it worked just fine.

   t

   
   

2. Further to my previous post - I think you may have stumbled on the single most simple way of converting the system from physical to virtual.

   I think the following procedure would work, however I have not had time to test it.

   on the machine to be converted (physical machine) nfs mount vmware host drive where you will be putting the virtual disks. Then run the qemu-img convertion from /dev/hda putting the output on the nfs mounted partition. This will be a one step conversion directly to the final location. It might be best to shut down as many processes as possible on the source host.

   I’ll give this a go and report back.

   
   
4. ran into the same problems you did. VMWare apparently *really* doesn’t like using images created from individual partitions. The reason isn’t documented, but I believe it is because the partition table for the disk is located at the beginning of the disk, which isn’t being copied. So you are left with a single partition that VMWare believes is the whole disk, and no information as to the partition’s geometry. Additionally, there is no room for an MBR at the beginning of the partition, because it jumps straight into the data section. My suggestion is to use LiveView, which I wrote about here:

   http://www.robertpeaslee.com/index.php/raw-image-to-vmdk-the-easy-way/

   If that doesn’t work, then… well, I have ideas, but they aren’t fun :) Let me know if LiveView works for you. Good luck mate!

   1 March 2008, 5:07 pm

Then look at this

Raw Image to VMDK… the Easy Way

In my last post I chronicled the time consuming process of converting an image produced using the Unix dd utility to a vmware VMDK virtual disk. The process does work, but VMWare will only load it if the image contains a master boot record that still contains the appropriate information when removed from the rest of the system. Additionally, I think it is appropriate to mention again how terribly slow it is. We can circumvent these problems with Live View for Windows, a program developed for forensic work by Carnegie Mellon University.

The process for creating the image with dd is the same as last time, IE, by doing something along the lines of:


dd if=/dev/hdc3 of=/mnt/external/diskImage/XPSP2.img bs=1024

Once that image is generated and you are back in Windows, simply launch the Live View utility and… the rest is pretty self explanatory. Once the dd image is generated, the Live View program takes only a couple minutes to create the new vmdk, configure the virtual machine, and launch it. Sweet, sweet efficiency!

now what is live view?
http://liveview.sourceforge.net/

Live View 0.6 is now available for download.

This version adds support for third party image formats such as EnCase with the use of image mounting software such as Mount Image Pro or Physical Disk Emulator. This release also adds support for the newest versions of Java and VMware, improves the boot preparation process to allow a wider variety of system configurations to boot successfully, and fixes a number of bugs. For more information, see the changelog.

We appreciate the feedback and encourage you to continue to report bugs and make suggestions to help make this tool more useful for everyone. The best way to do this is to post to the forums.

---

Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to "boot up" the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra "throw away" copies of the disk or image to create the virtual machine.

Live View is capable of booting

• Full disk raw images
• Bootable partition raw images
• Physical Disks (attached via a USB or Firewire bridge)
• Specialized and closed image formats (using 3rd party image mounting software)

Containing the following operating systems

• Windows 2003, XP, 2000, NT, Me, 98
• Linux (limited support)

Behind the scenes, Live View automates a wide array of technical tasks. Some of these include: resolving hardware conflicts resulting from booting on hardware other than that on which the OS was originally installed; creating a customized MBR for partition-only images; and correctly specifying a virtual disk to match the original image or physical disk.

***************
which is amazing because boot failure due to changing underlying hardware machines is the main issue i have often with windows. you can't just plug a machine into a different machine! now you can

i have not gotten this working myself.
my machine just reboots over and over again.

dyndns

free dyndns is available at
http://freedns.afraid.org/
reverse-dns.org
or
reverse-dns.net is the target