Sunday, March 23, 2008

qemu and vmware and kvm and live view

I have been playing with qemu. I have a vmware machine and now I would like to get a qemu version. When I get a new cpu with virtualization in hardware I can try out kvm as well.

Here is some nice stuff.
http://www.robertpeaslee.com/index.php/converting-a-physical-disk-to-a-virtual-disk-for-free/

He wrote:

I have a Windows XP SP2 installation that I use for work, while my current personal OS of choice is Windows Vista. I only work weekends, so rebooting wasn’t too bothersome initially. Over time it has become more and more of a chore, as I can’t easily switch from one development environment to another without rebooting. I decided it was time to make my work installation a virtual machine.

I had some additional complications that made the process a little non-standard. I originally tried using VMWare’s converter tool, but it would fail at 97% of the creation of the disk. I then tried using a Windows port of the Unix utility ‘dd’ to create a raw image of the disk, but because the Windows volume manager was accessing the disk, dd would give me access errors. Additionally, the VMWare converter doesn’t support converting from a raw image (…grr…), so Qemu’s qemu-img tool would have to be used to convert the raw image to a disk in VMWare’s vmdk format.

The steps to reach our goal aren’t too complicated, and can be replicated by others easily. To do it, I used:

  • VMWare Workstation
  • A Linux installation or LiveCD (I used my existing Debian installation, but something like Knoppix would work fine)
  • Qemu
  • NTFS-3G (if you plan on writing out to an NTFS partition from Linux, as I did)

I started by booting into my Linux install. Linux only mounts the disks it uses (hint hint, Microsoft) so we can access all sectors of the partition to make a dump of the disk with dd. I first had to mount the partition where I wanted the output file to reside, which uses NTFS:

ntfs-3g /dev/sdb1 /mnt/external

Next, create the image. I did this with the following (substitute your device/partition and output file):

dd if=/dev/hdc3 of=/mnt/external/diskImage/XPSP2.img bs=1024

When that finishes, the file specified with the ‘of’ option in dd will contain a block-by-block exact copy of your partition. However, it is in a raw format - we need it in a format VMWare can read. This is where Qemu comes in. Qemu is distributed with qemu-img, a tool used for creating, manipulating, and converting images. Specifically, our goal is to use qemu-img’s convert functionality to convert from a raw image format to the vmdk format. This is accomplished with:

qemu-img convert -f raw /mnt/external/diskImage/XPSP2.img -O vmdk /mnt/external/diskImage/XPSP2.vmdk

Be prepared to wait. For a 40gb image, this process took roughly 12 hours. Since qemu-img provides no status as to how far it has come, I kept tabs on it just by monitoring the filesize of the output image. This is entirely unnecessary, but if you want to do the same, just open a new terminal and type the following:

while [1 -gt 0 ]; do du -hs /mnt/external/diskImage/XPSP2.vmdk; sleep 10; clear; done

This will just print out the size of the file on your screen so you can watch it grow.

When the process is completed, boot back into Windows (or if you are using Linux as the host, stay put) and create a virtual machine around your new disk image. Don’t forget to remove the original img created with dd, it is a huge waste of disk space :)

[Notes]

  1. This process is really only feasible if you have a lot of disk space. At worst, the disk requirements are greater than 2*P, where P is the partition size of the virtual machine you wish to create. However, qemu-img only writes out actual data, not empty sectors, so your output image will be the size of the used space in the input image. For my conversion, I used over 40gb (input)+15gb (output) of disk space, which was reclaimed with the deletion of the output of dd, and resizing another partition to use the old physical installation’s space.
  2. qemu-img doesn’t support stream input, which is why we can’t pipe dd’s output directly into qemu-img convert. This would have reduced the disk requirements to only the size of the vmdk image, and sped up the process substantially. Bug the Qemu developers to implement this feature :)
  1. u may be able to skip the dd command. Try this command:
    qemu-img convert -f raw /dev/hdc3 -O vmdk boot.vmdk

    I think this should work just fine. dd is just reading from the disk and qemu-img would not know the difference.

    I tested on /dev/hda1 (/boot) and it worked just fine.

    t


  2. Further to my previous post - I think you may have stumbled on the single most simple way of converting the system from physical to virtual.

    I think the following procedure would work, however I have not had time to test it.

    on the machine to be converted (physical machine) nfs mount vmware host drive where you will be putting the virtual disks. Then run the qemu-img convertion from /dev/hda putting the output on the nfs mounted partition. This will be a one step conversion directly to the final location. It might be best to shut down as many processes as possible on the source host.

    I’ll give this a go and report back.


  3. ran into the same problems you did. VMWare apparently *really* doesn’t like using images created from individual partitions. The reason isn’t documented, but I believe it is because the partition table for the disk is located at the beginning of the disk, which isn’t being copied. So you are left with a single partition that VMWare believes is the whole disk, and no information as to the partition’s geometry. Additionally, there is no room for an MBR at the beginning of the partition, because it jumps straight into the data section. My suggestion is to use LiveView, which I wrote about here:

    http://www.robertpeaslee.com/index.php/raw-image-to-vmdk-the-easy-way/

    If that doesn’t work, then… well, I have ideas, but they aren’t fun :) Let me know if LiveView works for you. Good luck mate!

Then look at this

In my last post I chronicled the time consuming process of converting an image produced using the Unix dd utility to a vmware VMDK virtual disk. The process does work, but VMWare will only load it if the image contains a master boot record that still contains the appropriate information when removed from the rest of the system. Additionally, I think it is appropriate to mention again how terribly slow it is. We can circumvent these problems with Live View for Windows, a program developed for forensic work by Carnegie Mellon University.

The process for creating the image with dd is the same as last time, IE, by doing something along the lines of:


dd if=/dev/hdc3 of=/mnt/external/diskImage/XPSP2.img bs=1024

Once that image is generated and you are back in Windows, simply launch the Live View utility and… the rest is pretty self explanatory. Once the dd image is generated, the Live View program takes only a couple minutes to create the new vmdk, configure the virtual machine, and launch it. Sweet, sweet efficiency!

now what is live view?
http://liveview.sourceforge.net/

Live View 0.6 is now available for download.

This version adds support for third party image formats such as EnCase with the use of image mounting software such as Mount Image Pro or Physical Disk Emulator. This release also adds support for the newest versions of Java and VMware, improves the boot preparation process to allow a wider variety of system configurations to boot successfully, and fixes a number of bugs. For more information, see the changelog.

We appreciate the feedback and encourage you to continue to report bugs and make suggestions to help make this tool more useful for everyone. The best way to do this is to post to the forums.


Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to "boot up" the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra "throw away" copies of the disk or image to create the virtual machine.

Live View is capable of booting

  • Full disk raw images
  • Bootable partition raw images
  • Physical Disks (attached via a USB or Firewire bridge)
  • Specialized and closed image formats (using 3rd party image mounting software)

Containing the following operating systems

  • Windows 2003, XP, 2000, NT, Me, 98
  • Linux (limited support)

Behind the scenes, Live View automates a wide array of technical tasks. Some of these include: resolving hardware conflicts resulting from booting on hardware other than that on which the OS was originally installed; creating a customized MBR for partition-only images; and correctly specifying a virtual disk to match the original image or physical disk.

***************
which is amazing because boot failure due to changing underlying hardware machines is the main issue i have often with windows. you can't just plug a machine into a different machine! now you can

i have not gotten this working myself.
my machine just reboots over and over again.





10 comments:

Rob said...

Hey,

I'm really happy that you found what I wrote about useful. It is always nice to see your efforts helping others!

In the future, you might want to only extract snippets of text and provide a link back to the original article instead of copying the entire post and the comments. I don't particularly mind, and references to my blog are definitely appreciated, but it was my original work and I would like to see it in its entirety only on my blog :)

--Robert Peaslee

debuser said...

Sorry,

I meant to copy only the important information I needed so I could find it later on. I also put your url in the text. It was late at night and I was running from link to link trying to figure out what to do. I was was also worried I wouldnt find all the stuff again:)

It is tough with the web being an evanescent place where stuff is here today and gone tomorrow.

I realize that there are limits to allowable free use, and I apologize if I hurt your feelings. I think I gave you full credit if you look at what I wrote....

Rackmountsales said...

I’ve been checking out this blog it’s been really helpful. I was not understanding what I was doing. But your blog helped me a lot. Thank you very much and all the best in future.
KVM Switches

Dear Coupons said...

Great Blog!You have mentioned very useful information.I liked it.Network
solution coupons
provides you a great schemes for network devices.

eblogkpn said...

彌月禮盒團購美食 印刷 彩色印刷 包裝 設計pe膜 冬令營 food processing equipment frying machine Telecom PCB 泡菜 團購美食油飯麻糬
創業加盟 水餃 壯陽食品 早洩漆彈情趣用品 湯包 搬家公司 會場設計紙盒訂裝 展覽設計展場設計 展覽設計 消防設備 消防設備 機電 崴立機電 牙齒美白 植牙

eblogkpn said...

出軌 清潔公司台北搬家公司 整形 韓風整形 整形 韓風整形 老人癡呆症 情緒管理 訂房網 線上訂房 室內裝潢 裝潢 裝潢工程 房屋裝潢 舊屋翻新 裝潢施工 木工裝潢 居家裝潢 室內裝潢設計 裝潢設計馬桶 馬桶不通 國外旅遊 國外機票 團體旅遊 直航機票 簽證熱水器

eblogkpn said...

DC Jack tact switch Slide switch Phone Jack USB connector RCA Jack開 關 製網 菱形網 不鏽鋼網 金屬網 無塵室射出 Disposable plastic cups Disposable plastic cups Disposable products 宜蘭住宿 ECO products Biodegradable plastic PLA 律師事務所 律師 寵物醫院polylactic acid Biodegradable
搬家公司 月子中心 seo 關鍵字廣告 關鍵字 google關鍵字廣告 關鍵字行銷 網路行銷 通姦 徵信社 外遇 桃園房屋仲介 桃園房屋買賣 桃園房屋 醫學美容診所 淨膚雷射 雷射溶脂 飛梭雷射 微晶瓷 植髮 團體服 團體服訂做 醫學美容診所 肉毒桿菌 肉毒桿菌瘦臉 醫學美容 整型診所 美國月子中心 徵信 徵信公司

eblogkpn said...

素食餐廳 氣球佈置二手車健檢 醫學美容 淨膚雷射 汽車美容 法拍屋 水餃 清潔公司 塑膠袋批發 塑膠袋工廠 實驗動物 到府坐月子 坐月子 坐月子中心 坐月子餐 孕婦 月子餐 美國月子中心 團體服 OBU 投審會 會計師事務所 會計師 工商登記 彌月蛋糕 公司登記 保時捷 法拉利 福利 <包子 肉粽 宅配美食 四神湯 搬家公司 訂房網

eblogkpn said...

喜餅禮盒 < 塑膠射出 塑膠射出廠 模具設計 Precision Mold加盟創業 冷凍宅配 宅配美食 Light guide panels Plastic Products Mold design 沖孔網
飛梭雷射 柔膚雷射 獸醫師 創業鞋之澡堂 洗鞋子 洗包包加盟彌月禮盒 彌月禮 玻尿酸 皮膚科 皮膚科診所 肉毒桿菌 肉毒桿菌瘦臉 脈衝光 除斑 Flex PCB PCB Electronic PCB中華湯包湯包桃園房屋買賣桃園房屋仲介桃園房屋買賣桃園房屋仲介乳癌 全身健康檢查 自由行 三久 太陽能 三久太陽能 太陽能熱水器 環保袋 慈善基金會 慈善機構 租辦公室 租店面 買辦公室電波拉皮 hand dryer 電波拉皮

eblogkpn said...

房屋貸款 剖腹生產 姓名配對 星座 星座運勢 算命 素食料理 素食水餃 開運印章 風水 外遇 徵信 壁癌 屋頂防水 屋頂隔熱 抓漏 油漆 徵信社 外遇 徵信 徵信社 外遇 徵信 徵信社 外遇 徵信 徵信社 外遇 徵信 徵信社 外遇