Virtually Aware

Blog writing has been light here for a while since I have been spending my spare time documenting many of the tips and tricks I have learned throughout the years with Hyper-V and server virtualizations in general.  The articles range from true how-to to licensing considerations to marketing a server virtualization deployment to your management.  If you get a chance take a look at the articles.

Virtualization don'ts- Neglecting VM resource allocation

Using Diskshadow for backing up Hyper-V workloads

Gathering workload data essential to P2V migration

Winning management buy-in for server virtualization projects

How to make Hyper-V server virtualization licensing work for you

Using Windows Server Backup for Hyper-V workloads

Creating virtual machine templates aids Windows server provisioning

Fixed vs. dynamic disks for Hyper-V virtual machines

How Hyper-V snapshots work

The top four Hyper-V virtualization problems that plague admins

The pros and cons of clustered virtual host environments

Recently I was reading Seth Godin’s blog, which has nothing to do with virtualization, but everything to do about the mindset of a consumer.  He points out something very basic in his entry, that Microsoft needs to address in Hyper-V and other virtualization management apps.  I won’t reiterate his whole entry but the last two lines point to what Microsoft really needs provide to truly make market gains.

“The chances that you can top a trusted provider on the very thing the provider is trusted for are slim indeed.”

“Instead, you gain converts by winning at something the existing provider didn't think was so important. “

To some degree, the standalone Hyper-V Server R2 gives the consumer something new,  A free hypervisor with capabilities which its main competitor does not provide in its free offering i.e. Live Migration of VMs.  Where as this is a valid product differentiation, it is not a feature differentiation.  This has been tried in the past, offering something free that used to be a licensing cost.  This usually does not have staying power.  At best this is just a momentary advantage that competitors soon follow i.e. Virtual Server 2005, VMware Server, Hyper-V, XenServer

So much of Microsoft’s server virtualization strategy has been to compare itself to VMware on a feature by feature basis.  To some degree I think this was important. If they came out with a product that diverged too much from established definition of server virtualization, no one would take them seriously in adopting the product.  Now however, since the feature set between the products is converging, where do they go next.  VMware has grand plans to encompass the total datacenter.  Whether this plan succeeds or not, is still to be debated, but it is a unique product offering.  What will Microsoft do?  I see them making inroads into the small to medium businesses with the current product set, but for those that are enterprise customers, will there be any real innovation?

My random thoughts on virtualization can be found here.  Add to the conversation.

While writing an article for SearchServerVirtualiztaion on snapshots one of the editors posed the question. What does AVHD stand for?  Honestly, I didn’t know.  I knew a VHD stands for “Virtual Hard Disk”, but what about the “A?”  The customary search through popular search engines came up mostly empty except for one.  Actually four, but the all had a variation of the same title (search results) so I will call them one for the sake of argument.  Anyway, it wasn’t something that was easily found.  There were plenty of references of .AVHD files and what they are, snapshot differencing disks, but very limited reference to what the “A” represents.  So what did this one search reference say the “A” stood for?  “Automatic Virtual Hard Disk.”  Seems to fit, but I was still skeptical given the limited results to back it up.  This led me to pose the question to others in the Virtualization field.  Here is what they came up with:

Ben Armstrong (Virtual PC Guy) - “We have never (AFAIK) actually documented what the "A" is for.  But it is "automatic". “

John Howard - Hyper-V and virtualization blog  “Rob - yes, the A stands for Automatic. Under the covers, an AVHD is simply a differencing VHD with a different extension. We use the different extension to make it more obvious that users shouldn't be manipulating VHDs directly, rather use the snapshot capability directly.”

Mike Briggs - Senior Support Escalation Engineer (Microsoft System Center Support) “Although I haven't seen any official documentation anywhere that spells it out clearly, we have always referred to it as "automatic" which I think stems from Rob Larson's Resource Kit for Hyper-V. (Isbn# 978-0-7356-2517-4)”

A little more clarity on the subject.  Not sure if this makes you or me any smarter, but maybe there will be at least one more search result in the list.

I have been working with Microsoft over the past year in the Hyper-V Rapid Deployment Program.  Here is our story.  Works great, saves money, space and energy.  There are lots of good case studies out there.  If you get a chance, take a look at a few.

As I mentioned in a previous post, I have been involved with Microsoft with the Rapid Deployment Program for Hyper-V.  It has been a great way to get immersed in the product with full premier support and the ability to give feedback directly to Microsoft.  Along with those benefits, I have had the aid of John Wilson, from Microsoft Consulting Services, on-site one day a week to help guide and provide an inside track to configuration and troubleshooting. 

As a result of my involvement, the organization where I work has been mentioned in one of the growing number of case studies for Hyper-V.  Take a look.

Dartmouth-Hitchcock Medical Center - Solution Brief

All Hyper-V Case Studies

-Rob

I have been working with Volume GUIDs more and more in our Hyper-V implementation and there has been a little bit of a learning curve in viewing, modifying, deleting etc... the files that now live on this volume without a drive letter.  I haven't seen much out there on how to manage Volume GUIDs so I thought it would be a good topic for the blog. 

I have to plug Adam Fazio for leading to some of the info I have gained on this topic.

Like a normal volume, with a drive letter, there are 2 ways to view, modify, copy etc... files that reside therein. Explore in the GUI and Command line access.

Explorer View:  Just paste the Volume GUID in a RUN command.  This will give you the common Explorer view of your files that reside on the volume.

Command Line: dir \\?\Volume\\   i.e. dir \\?\Volume{fc247e42-0a5e-11dd-94db-001b785788b0}\\  The key to this one is to make sure you are using the second backslash at the end of the Volume GUID if you want to get to the root of the volume.  If you want to go to a known subfolder however all you need to do is \\?\Volume{fc247e42-0a5e-11dd-94db-001b785788b0}\FOLDERNAME.  No second backslash needed.

So this is a good step to being able to manage these types of volumes, but how do you get the Volume GUIDs.  In a webcast from Adam Fazio today he presented a Powershell script that displays all the Volume GUIDs on a local or remote server. [For remote, change localhost to a computer name or computer names. Separated by a comma.]

Get-WmiObject -class Win32_Volume -computername localhost | where-object {$_.name -like "\\?\*"} | select-object label, name, capacity, freespace | format-list

I have created a command file that runs this as a powershell script with a pause at the end to allow me to copy off the Volume GUID I am looking for.

NOTE:  Even though you can output all the Volume GUIDs of a remote server, at this time you cannot access the Volume  remotely. The above ways to access your Volume GUIDs can only be done from the localhost where they reside.

Hope this helps you out.

-Rob

I have been working with Hyper-V for a while and have had this problem with mounting ISO files that sit on a common file share within our organization.  The problem was when I tried to mount them from my remote Hyper-V management console I would get the error below.  If I mounted it from the Hyper-V host Hyper-V management console everything worked as expected.

This stumped me for a while, but then I got to thinking.  Where had I seen this before....  Virtual Server 2005.  Constrained delegation of authority settings on the Virtual Host objects was necessary there in order to both mount remove ISO images and in order to manage the virtual hosts from a common management web page.  So I made similar modifications to my Hyper-V host objects and now all seems to be working correctly.  Below are the steps I took to make mounting ISO images with my remote Hyper-V management console, work like a charm

Open Active Directory Users and Computers, Right-Click the Hyper-V host server object and select properties.  Then click on the Delegation tab. 

Select "Trust this computer for delegation to specified services only" and "Use any authentication protocol."  Then click "Add"  to select the service necessary to pass credentials to the remote file server where your ISOs are located.

Click on "Users or Computers"

Enter the name of the remote resource you need to pass credentials to through the Hyper-V Host and click OK. (i.e. File Server that houses your ISO images)

Select CIFS as the service type and click OK twice.

The refresh on this procedure can take a few minutes to propagate to the Hyper-V host object.  Once this is completed you will be able to successfully mount an ISO from your remote Hyper-V management console.  I suspect that this type of information will be in the final Hyper-V documentation, but as of yet, I have not seen any public documentation outlining this.  If I missed anything or you have any questions, feel free to comment or send an email along. 

I usually do not write much about what I am doing with certifications, but I found out about this one while at TechEd and decided to see if I could get in on the exam while I was here.  It is a 3 hour test with 75 questions that took me about 2 hours to complete.  There is a voucher code that you can use to take the test for free before June 23rd, 2008.  Not sure if I am supposed to give that out, but if you put "71-652 voucher" in your favorite search engine you will find it easily.  As far as the test, it had quite a fair number questions on Snapshotting within Hyper-V, iSCSI configuration for failover clustering, SCVMM 2008, and versioning information for both VMs and Host Servers.  It was a fair test and if you have experience with Hyper-v and SCVMM 2008 it might be a way to knock off a test while it is fresh in your mind.

Good Luck

UPDATE: Just check Prometric tonight and looks like I passed.  Not sure what the released exam looks like, but if you have a working knowledge of Hyper-V and SCVMM and have worked with 2008 clustering, that should cover the main topics of the test. 

The organization I belong to is part of the Hyper-V rapid deployment program which is a program run by Microsoft to get organizations to deploy production virtual servers on early RC and RTM code so that when the code is deployed to the rest of the world, they can note that they have X number of organizations already running it in production.  If you have been to any release event, these are the early adopter stories.  It also comes with great support from Microsoft and Partners who really want the early implementations succeed.  If you ever get the chance to be part of a TAP or RDP, I would recommend it.

So back to the backup script.  Being part of the rapid deployment program, and the desire to test out Hyper-V with production based virtual servers, we needed a way to protect the data that is the servers incase of problems.  Of course we could have just put a backup agent on the child servers and completed file based backups, but that can be a scheduling and resource nightmare once you get a large number of child servers on a particular parent server.  It also did not meet our standard of how we backup our servers in our Virtual Server 2005 R2 environments.  So, to mimic the success we had with backing up our Virtual Server 2005 R2 environment using the VShadow SDK, we looked for a similar solution. 

What we discovered right off the bat is, the VShadow SDK does not work on Windows 2008.  After a little reading of Windows 2008 documentation and a some Googling we came to realization that there was  a new and improved utility called DiskShadow.  What we came up with is a fairly simple compilation of a Diskshadow script and CMD files that takes live snapshots (using the VSWriter) of child servers, mounts those snapshots and then uses rob0copy to copy the files to an alternate location for backups to tape that happen during the daytime hours. 

Below are the scripts that consist of the backup procedures as it stands today.  This has already gone through many different derivatives, and I suspect this will evolve over time or maybe as others add their input.   Randy Pausch makes a good point about feedback from your peers and the inability to grow without it.  Please feel free to leave feedback.

Take note of line wraps.

VSBACKUP.CMD

*****************************************

REM Date Formatting
FOR /F "TOKENS=1* DELIMS= " %%A IN ('echo %date%') DO SET CDATE=%%B
FOR /F "TOKENS=1,2 eol=/ DELIMS=/ " %%A IN ('echo %date%') DO SET mm=%%B
FOR /F "TOKENS=1,2 DELIMS=/ eol=/" %%A IN ('echo %CDATE%') DO SET dd=%%B
FOR /F "TOKENS=2,3 DELIMS=/ eol= " %%A IN ('echo %CDATE%') DO SET yyyy=%%B
SET dateStamp=%mm%%dd%%yyyy%
c:
cd vsbackup
Echo %date% %time% >>c:\vsbackup\logs\%dateStamp%_%computername%_VSBackup.log
REM Execute DiskShadow script
diskshadow /s c:\vsbackup\vsbackup_step.dsh >>c:\vsbackup\logs\%dateStamp%_%computername%_VSBackup.log
Echo %date% %time% >>c:\vsbackup\logs\%dateStamp%_%computername%_VSBackup.log

VSBACKUP_STEP.DSH (coordinates backup proceedure)

*****************************************

#Diskshadow script file

#Maintenance task to clear all previous Volume Shadow Copies
DELETE SHADOWS ALL

#Allow for shadow copies to be persist across program exit, reset or reboot.
SET CONTEXT PERSISTENT

#Cab location for process
SET METADATA c:\vsbackup\cab\Backup.cab

#Verbose on because knowing what is going on is good.
SET VERBOSE ON

#Start Backup process
BEGIN BACKUP

#Alias volumes for easier use in process. 
#Both Drive letters and volume GUIDs are used.
#Volume GUIDs are used for those virtual guests
# on volumes with out a drive letter.
ADD VOLUME D: ALIAS G1
ADD VOLUME \\?\Volume{ac9658ca-fa80-11dc-85fa-001b785788b0}\ ALIAS G2
ADD VOLUME H: ALIAS G3
ADD VOLUME J: ALIAS G4
ADD VOLUME \\?\Volume{6019c063-1540-11dd-b48d-001b785788b1}\ ALIAS G5

#Create Snaps
CREATE

#Maintenance script that deletes previous days file backup from
#secondary backup drives
EXEC c:\vsbackup\backupscript_maint.cmd

#Exposing of shadows and coping to secondary backup drives
#for each volume alias above.
#Using UNEXPOSE as a fallback proceedure incase Volume Shadow
#Copy malfunctions.
EXPOSE %G1% X:
EXEC c:\vsbackup\backupscript_step.cmd
UNEXPOSE X:
EXPOSE %G2% X:
EXEC c:\vsbackup\backupscript_step.cmd
UNEXPOSE X:
EXPOSE %G3% X:
EXEC c:\vsbackup\backupscript_step.cmd
UNEXPOSE X:
EXPOSE %G4% X:
EXEC c:\vsbackup\backupscript_step.cmd
UNEXPOSE X:
EXPOSE %G5% X:
EXEC c:\vsbackup\backupscript_step.cmd
UNEXPOSE X:

END BACKUP
#End of script

BACKUPSCRIPT_MAINT.CMD

*****************************************

FOR /F "TOKENS=1* DELIMS= " %%A IN ('echo %date%') DO SET CDATE=%%B
FOR /F "TOKENS=1,2 eol=/ DELIMS=/ " %%A IN ('echo %date%') DO SET mm=%%B
FOR /F "TOKENS=1,2 DELIMS=/ eol=/" %%A IN ('echo %CDATE%') DO SET dd=%%B
FOR /F "TOKENS=2,3 DELIMS=/ eol= " %%A IN ('echo %CDATE%') DO SET yyyy=%%B
SET dateStamp=%mm%%dd%%yyyy%
Echo %date% %time% >> c:\vsbackup\logs\%dateStamp%_%computername%_maint.log
REM Delete directory with previous nights backup.
rd /s /q f:\%computername%
Echo %date% %time% >> c:\vsbackup\logs\%dateStamp%_%computername%_maint.log

BACKUPSCRIPT_STEP.CMD

******************************************

REM Reusable file copy proceedure for mounted shadows.

FOR /F "TOKENS=1* DELIMS= " %%A IN ('echo %date%') DO SET CDATE=%%B
FOR /F "TOKENS=1,2 eol=/ DELIMS=/ " %%A IN ('echo %date%') DO SET mm=%%B
FOR /F "TOKENS=1,2 DELIMS=/ eol=/" %%A IN ('echo %CDATE%') DO SET dd=%%B
FOR /F "TOKENS=2,3 DELIMS=/ eol= " %%A IN ('echo %CDATE%') DO SET yyyy=%%B
SET dateStamp=%mm%%dd%%yyyy%
Echo %date% %time% >> c:\vsbackup\logs\%dateStamp%_%computername%_xcopy.log

REM Check to see if backup drive and directory exist.
if not exist f:\%computername% md f:\%computername%

REM Copy mounted volume data to secondary backup drives.
REM I used Robocopy for its logging and retry capabilities.
REM Any Copy proceedure would work
REM Note the exclusions that could cause problems with coping files.
robocopy X:\ f:\%computername%\ /E /V /NP /LOG+:"c:\vsbackup\logs\%dateStamp%_%computername%_xcopy.log" /ZB /R:2 /W:30 /XJD /XJF /XD $RECYCLE.BIN SYSTEM* MP* /XA:SHO
Echo %date% %time% >> c:\vsbackup\logs\%dateStamp%_%computername%_xcopy.log

So, here is the code we are using to backup the servers.  In the next post, I will go through the scripts in more detail.  Until then, send your feedback.

Rob

The troubleshooting continues from the problems I have been having with Virtual Server 2005 SP1 (see http://virtuallyaware.spaces.live.com/blog/cns!549C424F228D6040!125.entry for the background). 

It looks like the root cause to the "Hang" problem is the hypervisor.  Not sure who's fault it is, Microsoft or HP, but if I uncheck ALL the "Enable hardware-assisted virtualization if available" check boxes from the general properties of each guest, the "Hanging" of the guest during startups and save state restores goes away.  I also do not get the "Hang" if I disable processor virtualization on the processor in the BIOS.  I have a ticket open with MS at the moment and hopefully will get a fix.  When I do i will pass it on.  I know there is a growing number of you that are seeing similar problems.

-Rob

I was looking through the Microsoft new groups today and answered a question that I have done a few times so I thought it would be a good time to blog about it here.  The task is how to expand a system drive that is running out of space.  I used to do this by creating a second VHD that was larger than my original then mount up both to the guest and boot  up a BartPE disk and then use ghost to image one disk to the larger disk.  This was slow and painful, especially since BartPE didn't really have any knowledge of virtual machine additions.  As more tools for virtual servers came about, this task became easier.  Below I outline different ways that I do this.

VMtoolkit VHD Resizer is your friend.  All of the techniques below use this utility

The problem with increasing the size of a System partition is that using the built in Windows Server 2003 utility diskpart, will not work for an "active" system drive.  This is how it can be done.  The trick is to make the system drive a non-active drive or secondary drive.

Option #1

Use the VMToolkit VHD Resizer utility to increase the VHD file to what you want. This actually creates a secondary VHD file and moves the data over to it. Then mount this VHD file to a drive letter using the VHDMount utility in VS 2005 SP1.  Then go to a command prompt, and using the diskpart utility, select the mounted volume and use the diskpart /extend command.  This assumes you are using Windows Server 2003 however. Then unmount the vhd, make sure it is attach to the correct virtual guest, and boot it up.  You should have an expanded system drive.

Option #2
Another way to do it would be to use the VMToolkit Resizer and then attach the new VHD to another virtual guest as a secondary drive.  Boot the other virtual guest.  Once it is booted, go to a command prompt and use the same diskpart routine on the secondary drive as above.  Then shut down this secondary guest, detach the drive, reattach it to the right guest and boot.  Your system drive should now be expanded.

Option #3

Still using ghost but faster:  Like the old way  of creating a secondary VHD file, but instead of booting BartPE in the virtual guest, you will mount both VHDs with the vhdmount utility of VS 2005 SP1 on the virtual host system.  Then you will put your BartPE CD that has ghost as an application into the host.  Then use ghost to do a disk to disk transfer.  The benefits of this is that you use the native speed of the host system drives to do the image from one disk to the expanded disk.  I don't use this option all that often, but it is a viable option for expanding a vhd system disk. 

If you have methods of expanding virtual system disks, please drop me a line and let me know.

-Rob

You can now get to my spaces site by just typing http://www.virtuallyaware.net.  I may host this on my own web server soon, but for now, it will point to where you are now.  Anyway, thanks for stopping by whichever way you got here.

*****************************************************************************************************************************************************

Update 12/24/2007

After some further testing, the problem with Virtual Server 2005 SP1 continues. 

Notes:

KB941125 did help some.  It eliminated the complete freezing of the web console, but the guests still get hung up and are unresponsive to any commands.

Setting the resource allocation to the default 100 does not seem to be any help.  All of my guests have been at the default 100 and I have seen the re-occurrence of the guest freezing. 

Other options tried:

AMD Opteron™ Processor with AMD PowerNow!™ Technology Driver Version 1.3.2.0053 for Windows XP and Windows Server 2003 (x86 and x64) -  This was to solve the problem with time shift between processor cores that would give negative ping times and would show event id 1054.  Below are some articles that outline this.

http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182_871_9033,00.html

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=463637&prodTypeId=12169&objectID=c01075682

http://support.microsoft.com/kb/938448/en-us

*********************************************************************************************************************************************************

Let me start out by saying that my experience with Virtual Server and all the subsequent versions that followed have been relatively bug free.  This is the first one that really stopped me from progressing to Virtual Server 2005 R2 SP1.  At first I thought what I had was a corrupt install, but after a few rebuilds of the systems and looking at firmware and drivers everything looked fine.  The main symptom I saw was that every time I put the servers into saved state either manually or with a script, when restoring to running state, the web interface would not be accessible nor would the VMRC or VMRCplus or any type of communication.  The guest servers would not ping or be able to be shutdown with a script.  The whole Virtual Server COM API was not accessible.  The only way to fix this would be to kill the vssrvc.exe process, restart the virtual server service and then discard the saved states of the guests.  If you tried to restore the saved state of the guests, you would go into the inaccessibility again.  Upon starting the guest servers back up from an off state, everything would work just fine, until the next saved state which in our case is sometimes during our scripted backups or on a shutdown script we have in place or when I felt like save stating any guest.  We do have 2 other virtual hosts running Virtual Server 2005 R2 SP1 that seem to be working just fine.  The 2  hosts that were showing this problem are DL585G2 systems with AMD-V.  Not sure if this is linked to the overall problem, but the DL585G1(AMD without AMD-V) and DL380G4 (Intel Xeon without VT) have not experienced this problem at all.  Lets hope this is the last of the bugs for now.  At least until Hyper-V.

*****************************************************************

Attached is the news group string.

So, do you have to install the patch and do one of the workarounds or
"should" the patch be all that is necessary?  Do you have any experience
with the effectiveness of KB941125?
Rob McShinsky
"Benjamin Armstrong [MSFT]" <benarm@online.microsoft.com> wrote in message
news:uMg6X8eNIHA.3516@TK2MSFTNGP02.phx.gbl...
> Yup.
>
> Cheers,
> Benjamin Armstrong
> ============================
> Windows Virtualization Program Manager
> Blog: http://blogs.msdn.com/Virtual_PC_Guy
> Book: http://tinyurl.com/ysxcbm
>
> This posting is provided "AS IS" with no warranties, and confers no
> rights. You assume all risk for your use.
>
> Microsoft News Groups wrote:
>> Do you know anything about KB941125?  Sounds alot like what you are
>> outlining.
>>
>> Rob McShinsky.
>>
>> "Benjamin Armstrong [MSFT]" <benarm@online.microsoft.com> wrote in
>> message news:O8tUZheNIHA.484@TK2MSFTNGP06.phx.gbl...
>>> First:  I would recommend that you contact Microsoft Product Support so
>>> that we can get formal cases open against this.
>>>
>>> Second:  Try creating a blank virtual machine (with no VHD) and connect
>>> it to all of your virtual networks.  Configure that virtual machine to
>>> start first, and then have the other virtual machines start with a delay
>>> after that.
>>>
>>> Cheers,
>>> Benjamin Armstrong
>>> ============================
>>> Windows Virtualization Program Manager
>>> Blog: http://blogs.msdn.com/Virtual_PC_Guy
>>> Book: http://tinyurl.com/ysxcbm
>>>
>>> This posting is provided "AS IS" with no warranties, and confers no
>>> rights. You assume all risk for your use.
>>>
>>> Microsoft News Groups wrote:
>>>> Hey Ben;  I am seeing that you are also seeing this from other people
>>>> as well 9/5/2007 "CGI Timeout - VS 2005 SP1 Admin Site".  We are seeing
>>>> this also when we do a startup script to restore saved guests to a
>>>> running state (We do a shutdown script prior to put them in saved state
>>>> and create a log file the startup script reads).  Seems as more and
>>>> more people are popping up with this issue with VS 2005 R2 SP1.  What
>>>> is your recommendation?
>>>>
>>>> Thanks
>>>>
>>>> Rob McShinsky
>>>>
>>>>
>>>> "Microsoft News Groups" <Please@UseGroup.com> wrote in message
>>>> news:eb7zYe8MIHA.1188@TK2MSFTNGP04.phx.gbl...
>>>>> I will certainly check this out on my system.  I made another
>>>>> adjustment today after I say a single server "freeze" today.  I still
>>>>> had to shutdown the other guests and then kill the vssrc services to
>>>>> get control again. This was with all the guests with the same, default,
>>>>> values.  So what I did was remove all guests from the console, then
>>>>> delete the VMC files for all of guests, and then did a Create again to
>>>>> re-establish the vmc again.  I did this because I was doing a little
>>>>> comparison between VMC files that were created for guests native on
>>>>> box, vs. guest that were migrated over from another SP1 box  but
>>>>> without  AMD-V or from a machine that is not running SP1 or AMD-V.  It
>>>>> seemed that the VMC (xml) files were structured different. They have
>>>>> not been running long enough to determine if this will help, but it
>>>>> seems a little strange.  Maybe Ben might have an idea on if this could
>>>>> potentially be causing the problem.
>>>>>
>>>>> Here's to more testing.  Let me know if you have any further progress
>>>>> or want to try some different testing scenarios.  I have 2 DL585 G2
>>>>> systems to test on. 1 built an the other not.  I moved to the current
>>>>> testing host because I experienced the same problem on the other host.
>>>>> I thought it was a corrupt install.  The same guests that were on that
>>>>> guest are now on the new problematic host.  My hopes is that one of
>>>>> the guests is just messed.
>>>>>
>>>>> Rob McShinsky
>>>>> Dartmouth Hitchcock Medical Center
>>>>> Lebanon, NH
>>>>>
>>>>>
>>>>> "Stephan Kaufmann" <StephanKaufmann@discussions.microsoft.com> wrote
>>>>> in message news:791DF571-457B-4089-8CF0-519AFDE173ED@microsoft.com...
>>>>>> Hi Rob
>>>>>>
>>>>>> Very intresting!
>>>>>>
>>>>>> I have 6 VMs (2 Windows Server 2003 =SV000x  and 4 Win XP=VP000x) on
>>>>>> my
>>>>>> DL385 G2 with 10GB RAM. If I set the following values for "Resource
>>>>>> Allocation", I always get the freezing problem as soon as one of the
>>>>>> VMs with
>>>>>> lower priority boots:
>>>>>>
>>>>>> 1st value: Relative Weight
>>>>>> 2nd value: Reserved Capacity (% of one CPU)
>>>>>> 3rd value: Maximum Capacity (% of one CPU)
>>>>>>
>>>>>> SV0001: 100, 5%, 100%
>>>>>> SV0002: 100, 5%, 100%
>>>>>> VP0001: 50, 0%, 80%
>>>>>> VP0002: 50, 0%, 80%
>>>>>> VP0003: 50, 0%, 80%
>>>>>> VP0004: 50, 0%, 80%
>>>>>>
>>>>>> All VMs have the hardware-assisted virtualization active. As nearer
>>>>>> the
>>>>>> values are on the default, the problem appears seldom.
>>>>>>
>>>>>> It would be great, if  you can try it on your machine with these
>>>>>> values!
>>>>>>
>>>>>> Stephan
>>>>>>
>>>>>> "Microsoft News Groups" wrote:
>>>>>>
>>>>>>> I am trying this now.  I have 7 guests (all with additions 13.813)
>>>>>>> on this
>>>>>>> host for testing.  Before I saw this post, I had three of them with
>>>>>>> a
>>>>>>> relative weight of 99 and the other 4 with a relative weight of 100.
>>>>>>> All of
>>>>>>> them had the default maximum capacity of 100% and no reserve.  This
>>>>>>> is
>>>>>>> usually our default.  development or test servers get a relative
>>>>>>> weight of
>>>>>>> 99 or less and production servers all get 100.  Since I have changed
>>>>>>> all of
>>>>>>> the guests over to a relative weight of 100, 2 days ago, I have not
>>>>>>> seen the
>>>>>>> problem, but I have seen the machine go days without experiencing
>>>>>>> the
>>>>>>> problem.  I used to think it had something to do with our backup
>>>>>>> scripts
>>>>>>> that happen everynight, but the combination of the "freezing" not
>>>>>>> lining up
>>>>>>> with backup time period and this only happening with the new DL585
>>>>>>> G2
>>>>>>> systems with AMD-V makes me think the problem is somewhere else.  We
>>>>>>> have 3
>>>>>>> other DL585 G1 systems without AMD-V technology that are running
>>>>>>> 2005 R2 SP1
>>>>>>> and the same scripting options that have never had any problems.
>>>>>>>
>>>>>>> I will keep you up to date on any future freezing.  I am relectant
>>>>>>> to load
>>>>>>> the hosts with any more guests since the remediation of the
>>>>>>> "freezing"
>>>>>>> problem is to basically crash the guests.   If you want or need any
>>>>>>> information offline, please let me know.  I have 3 of the DL585 G2
>>>>>>> systems
>>>>>>> that I am eager to get stable so will try just about anything to get
>>>>>>> the
>>>>>>> service level to the same as the other 8 hosts I have.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Rob McShinsky
>>>>>>>
>>>>>>>
>>>>>>> "Benjamin Armstrong [MSFT]" <benarm@online.microsoft.com> wrote in
>>>>>>> message
>>>>>>> news:O%23k8WNhMIHA.5988@TK2MSFTNGP02.phx.gbl...
>>>>>>>> Rob -
>>>>>>>>
>>>>>>>> Can you try this and see if it affects your system?
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Benjamin Armstrong
>>>>>>>> ============================
>>>>>>>> Windows Virtualization Program Manager
>>>>>>>> Blog: http://blogs.msdn.com/Virtual_PC_Guy
>>>>>>>> Book: http://tinyurl.com/ysxcbm
>>>>>>>>
>>>>>>>> This posting is provided "AS IS" with no warranties, and confers no
>>>>>>>> rights. You assume all risk for your use.
>>>>>>>>
>>>>>>>> Stephan Kaufmann wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I also have a HP DL385 G2 with 2 CPUs and Virtual Server 2005 R2
>>>>>>>>> SP1 x64
>>>>>>>>> installed on Win Server 2003 Enterprise x64. I have the same
>>>>>>>>> problem and
>>>>>>>>> found a solution with active AMD-V!
>>>>>>>>>
>>>>>>>>> All VMs must have the default values in the "Resource Allocation"
>>>>>>>>> config.
>>>>>>>>> As soon as one or more VMs have other values and the first VM
>>>>>>>>> boots up,
>>>>>>>>> it freezes. If you set all VMs to the default values, all VMs work
>>>>>>>>> fine
>>>>>>>>> with AMD-V active in BIOS and also "Enable hardware-assisted
>>>>>>>>> virtualization if available" active in the "General Properties" of
>>>>>>>>> the
>>>>>>>>> VM.
>>>>>>>>>
>>>>>>>>> If you really want a VM with other "Resource Allocation" config,
>>>>>>>>> you have
>>>>>>>>> to disable "Enable hardware-assisted virtualization if available"
>>>>>>>>> of this
>>>>>>>>> single VM and it works too.
>>>>>>>>>
>>>>>>>>> Can anyone confirm this?
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> "Microsoft News Groups" wrote:
>>>>>>>>>
>>>>>>>>>> Ben;  From what I am seeing, since this happened again last
>>>>>>>>>> night, all
>>>>>>>>>> systems become unresponsive through what ever remote capabilities
>>>>>>>>>> are
>>>>>>>>>> out there.  Looking at the event log, all systems went into a
>>>>>>>>>> saved
>>>>>>>>>> state right before t his happened.  The website is inaccessable,
>>>>>>>>>> and it
>>>>>>>>>> cannot be reached with the VMRC client to go to the
>>>>>>>>>> administrative
>>>>>>>>>> display.    We have 8 other hosts running the same set of scripts
>>>>>>>>>> with a
>>>>>>>>>> mix of SP1 and non-SP1 VS 2005 R2.  All of those are working
>>>>>>>>>> perfectly.
>>>>>>>>>> The only difference is that these are DL585 G2 servers with the
>>>>>>>>>> AMD-V
>>>>>>>>>> technology.  We  have some DL585 G1 systems that are running very
>>>>>>>>>> well.with no problems.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is a snipit right before the Host went out to lunch.
>>>>>>>>>>
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:25 PM Virtual Machine "DH350" was saved.
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:25 PM Setting Change The setting "hardware/bios/time_bytes"
>>>>>>>>>> for the
>>>>>>>>>> virtual machine configuration "DH350" was changed.
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:25 PM Setting Change The setting
>>>>>>>>>> "settings/configuration/saved_state/path/absolute" for the
>>>>>>>>>> virtual
>>>>>>>>>> machine configuration "DH350" was changed from NULL to
>>>>>>>>>> "D:\B\ServersE\DH350\DH350.vsv".
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:25 PM Setting Change The setting
>>>>>>>>>> "settings/configuration/saved_state/path/relative" for the
>>>>>>>>>> virtual
>>>>>>>>>> machine configuration "DH350" was changed from NULL to
>>>>>>>>>> ".\DH350.vsv".
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:13 PM Setting Change The setting
>>>>>>>>>> "settings/shutdown/quit/was_running" for the virtual machine
>>>>>>>>>> configuration "DH344" was changed from true to false.
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:13 PM Virtual Machine "DH344" was saved.
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:13 PM Setting Change The setting "hardware/bios/time_bytes"
>>>>>>>>>> for the
>>>>>>>>>> virtual machine configuration "DH344" was changed.
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:13 PM Setting Change The setting
>>>>>>>>>> "settings/configuration/saved_state/path/absolute" for the
>>>>>>>>>> virtual
>>>>>>>>>> machine configuration "DH344" was changed from NULL to
>>>>>>>>>> "D:\A\ServersD\DH344\DH344.vsv".
>>>>>>>>>>  11/20/2007
>>>>>>>>>> 5:28:13 PM Setting Change The setting
>>>>>>>>>> "settings/configuration/saved_state/path/relative" for the
>>>>>>>>>> virtual
>>>>>>>>>> machine configuration "DH344" was changed from NULL to
>>>>>>>>>> ".\DH344.vsv".
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Looks liike it  protected the guests well before it died.  A
>>>>>>>>>> funny thing
>>>>>>>>>> is however, after a reboot I can get to all consoles and all
>>>>>>>>>> looks fine.
>>>>>>>>>> I can see the systems in saved state.  When I try to restore one
>>>>>>>>>> of
>>>>>>>>>> those systems from saved state, the server hangs again and does
>>>>>>>>>> not
>>>>>>>>>> allow me any access to the web or vmrc.  The server allocates RAM
>>>>>>>>>> which
>>>>>>>>>> I can see in task manager, but the guest never truely comes
>>>>>>>>>> online. The
>>>>>>>>>> only way to get the system to come back it to reboot one more
>>>>>>>>>> time or
>>>>>>>>>> kill the vssrvc.exe process, restart the service and discard all
>>>>>>>>>> previous save states for those systems that were saved during the
>>>>>>>>>> hang
>>>>>>>>>> time.
>>>>>>>>>>
>>>>>>>>>> Anythoughts
>>>>>>>>>>
>>>>>>>>>> Rob McShinsky.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> "Benjamin Armstrong [MSFT]" <benarm@online.microsoft.com> wrote
>>>>>>>>>> in
>>>>>>>>>> message news:%239qN2JWJIHA.5468@TK2MSFTNGP05.phx.gbl...
>>>>>>>>>>> For number 1 - are there any entries in the event log that might
>>>>>>>>>>> indicate what is happening here?
>>>>>>>>>>>
>>>>>>>>>>> For number 3 - are they using the web site or the VMRC client?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Benjamin Armstrong
>>>>>>>>>>> ============================
>>>>>>>>>>> Windows Virtualization Program Manager
>>>>>>>>>>> Blog: http://blogs.msdn.com/Virtual_PC_Guy
>>>>>>>>>>> Book: http://tinyurl.com/ysxcbm
>>>>>>>>>>>
>>>>>>>>>>> This posting is provided "AS IS" with no warranties, and confers
>>>>>>>>>>> no
>>>>>>>>>>> rights. You assume all risk for your use.
>>>>>>>>>>>
>>>>>>>>>>> TheWall wrote:
>>>>>>>>>>>> Hi, 1.When i click on reset nothing happens. Doesnt reset, just
>>>>>>>>>>>> keep
>>>>>>>>>>>> on running.
>>>>>>>>>>>> 2. Connection to the server has been lost
>>>>>>>>>>>> 3. Client being the platform being logged onto when we need to
>>>>>>>>>>>> do
>>>>>>>>>>>> testing (Virtual Machine) Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> "Benjamin Armstrong [MSFT]" wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Some questions:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. After reseting a virtual machine what do you have to do to
>>>>>>>>>>>>> get the
>>>>>>>>>>>>> website back?
>>>>>>>>>>>>> 2. What problems / errors do you see from the 32-bit client.
>>>>>>>>>>>>> 3.  What do you mean by 'client' in this context?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Benjamin Armstrong
>>>>>>>>>>>>> ============================
>>>>>>>>>>>>> Windows Virtualization Program Manager
>>>>>>>>>>>>> Blog: http://blogs.msdn.com/Virtual_PC_Guy
>>>>>>>>>>>>> Book: http://tinyurl.com/ysxcbm
>>>>>>>>>>>>>
>>>>>>>>>>>>> This posting is provided "AS IS" with no warranties, and
>>>>>>>>>>>>> confers no
>>>>>>>>>>>>> rights. You assume all risk for your use.
>>>>>>>>>>>>>
>>>>>>>>>>>>> TheWall wrote:
>>>>>>>>>>>>>> I am running a 64Bit virtual Server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a few issues: 1. In the server admin Website it does
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> respond if i try to reset or save Machines
>>>>>>>>>>>>>> 2. Cant connect to server from 32 bit client
>>>>>>>>>>>>>> 3. Cant add excisting clients created in Virtual server
>>>>>>>>>>>>>> 32bit.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>
>>

This is part one of the various VSMT Speed Tweaks that I have found in my experiences with the Virtual Server Migration Toolkit. Each one will save you a little time from beginning to end that adds up to significant time savings. This not only allows you to mitigate the application downtime necessary to migrate a system into your virtual environment, but also allows you to get home to your wife and kids at a reasonable hour and may even keep you out of the dog house.

Before running the GatherHW command on a system, disable or set to manual non-essential and application specific services. Once the server has been deployed into the virtual environment install virtual machine additions and uninstall apps that are not needed anymore i.e. various HP Management Agents or other hardware based software. Then, re-enabled the application specific services again. Leaving these set to automatic, especially during the first boot into the new virtual environment after the deploy process, can significant increase the amount of time it takes for the system to rediscover hardware and allow access to the system. Below is the documentation section we use to record these steps.