WINDOWS 2008 R2 REMOTE DESKTOP AND XENAPP 6 TUNING TIPS UPDATE  

6 December, 2010 (41336 reads)

Following the great article Terminal Server & XenApp Tuning Tips published on this website by Pierre Marmignon, this article defined all tips that I’ve found, test and validate for tuning Windows 2008 R2 and XenApp6.

Please note that :
–       These informations are provided “as is” and that using these tips is at your own risks.
–       All this tuning tips have been tested and validated only on VMs running on Vmware Vsphere 4, let us know your feeback related to any other platform (either Hypervisor or physical server).
Windows 2008 R2 OS Tuning Tips for Remote Desktop Service and XenApp 6
 
Registry Hive
Value
Type
Data
Purpose(s)
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
 KeepAliveTime
 REG_DWORD
180000
Determines how often TCP sends keep-alive transmissions
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
 KeepAliveInterval
 REG_DWORD
 100
Determines how often TCP repeats keep-alive transmissions when no response is received
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
 TcpMaxDataRetransmissions
 REG_DWORD
 10
Determines how many times TCP retransmits an unacknowledged data segment on an existing connection
HKLMSystemCurrentControlSetServicesLanmanServerParameters
 MaxWorkItems
 REG_DWORD
 512
Server Service Optimization
HKLMSystemCurrentControlSetServicesLanmanServerParameters
 MaxMpxCt
 REG_DWORD
 2048
Server Service Optimization
HKLMSystemCurrentControlSetServicesLanmanServerParameters
 MaxFreeConnections
 REG_DWORD
 100
Server Service Optimization
HKLMSystemCurrentControlSetServicesLanmanServerParameters
 MinFreeConnections
 REG_DWORD
 32
Server Service Optimization
HKLMSystemCurrentControlSetServicesLanmanWorkstationParameters
 UtilizeNTCaching
 REG_DWORD
 0
Disable Caching
HKLMSYSTEMCurrentControlSetServicesMRXSmbParameters
 OplocksDisabled
 REG_DWORD
 1
Disables Opportunistic Locking
HLMSYSTEMCurrentControlSetServicesLanmanworkstationParameters
 UseOpportunisticLocking
 REG_DWORD
 0
Disables Opportunistic Locking
HKLMSYSTEMCurrentControlSetServicesLanmanserverParameters
 EnableOplocks
 REG_DWORD
 0
Disables Opportunistic Locking
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
EnableRSS
 REG_DWORD
0
Disable Receive Side Scaling
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
EnableTCPA
 REG_DWORD
0
Disable TCP-acceleration
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
EnableTCPChimney
 REG_DWORD
0
Disable TCP Chimney Offload
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
DisableBandwidthThrottling
 REG_DWORD
1
The default is 0. By default, the SMB redirector throttles throughput across high-latency network connections in some cases to avoid network-related timeouts. Setting this registry value to 1 disables this throttling, enabling higher file transfer throughput over high-latency network connections
HKLMSystemCurrentControlSetServicesLanmanWorkstationParameters
 MaxThreads
 REG_DWORD
 17
Maximum Concurrent Threads
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
DisableLargeMtu
 REG_DWORD
0
The default is 1. By default, the SMB redirector does not transfer payloads larger than approximately 64 KB per request. Setting this registry value to 0 enables larger request sizes, which can improve file transfer speed.
HKLMSYSTEMCurrentControlSetServicesTcpipparameters
 EnableWsd
 REG_DWORD
 0
The default is 1 for client operating systems. By default, Windows Scaling Diagnostics (WSD) automatically disables TCP receive window auto tuning when heuristics suspect a network switch component might not support the required TCP option (scaling). Setting this registry setting to 0 disables this heuristic and allows auto tuning to stay enabled. When no faulty networking devices are involved, applying the setting can enable more reliable high-throughput networking via TCP receive window auto tuning. For more information about disabling this setting
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
FileInfoCacheEntriesMax
 REG_DWORD
32768
The default is 64 with a valid range of 1 to 65536. This value is used to determine the amount of file metadata that can be cached by the client. Increasing the value can reduce network traffic and increase performance when a large number of files are accessed
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
DirectoryCacheEntriesMax
 REG_DWORD
4096
The default is 16 with a valid range of 1 to 4096. This value is used to determine the amount of directory information that can be cached by the client. Increasing the value can reduce network traffic and increase performance when large directories are accessed
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
FileNotFoundCacheEntriesMax
 REG_DWORD
32768
The default is 128 with a valid range of 1 to 65536. This value is used to determine the amount of file name information that can be cached by the client. Increasing the value can reduce network traffic and increase performance when a large number of file names are accessed.
HKLMSYSTEMCurrentControlSetServicesLanmanWorkstationParameters
MaxCmds
 REG_DWORD
32768
The default is 15. This parameter limits the number of outstanding requests on a session. Increasing the value can use more memory, but can improve performance by enabling deeper request pipelining. Increasing the value in conjunction with MaxMpxCt can also eliminate errors encountered due to large numbers of outstanding long-term file requests, such as FindFirstChangeNotification calls. This parameter does not affect connections with SMB 2 servers.
Windows 2008 CPU Tuning (for VM only)
 
Because I’m working with VMware VSphere Server 4, I Supposed that the Hypervisor manage my processor and I don’t want to have any CPU management in the VM, I force all my VM to the “min  Power Scheme“  (High Performance) with the following command “owercfg -setactive scheme_min“
And force my Processor Performance Boost Policy, Minimum and Maximum Processor Performance State and Processor Performance Core Parking Maximum and Minimum Cores to the maximum. (http://www.microsoft.com/whdc/system/sysperf/Perf_tun_srv-R2.mspx)
The following commands set Processor Performance Boost Policy to 100 percent on the current power plan:
Powercfg -setacvalueindex scheme_current sub_processor 45bcc044-d885-43e2-8605-ee0ec6e96b59 100
Powercfg -setactive scheme_current
The following commands set Processor Performance State parameters to 100 %
Powercfg -setacvalueindex scheme_current sub_processor 893dee8e-2bef-41e0-89c6-b55d0929964c 100
Powercfg -setactive scheme_current
Core parking is a new feature in Windows Server 2008 R2. The processor power management (PPM) engine and the scheduler work together to dynamically adjust the number of cores available to execute threads, to turn off core parking, set the Minimum Cores parameter to 100 percent by using the following commands:
Powercfg -setacvalueindex scheme_current sub_processor bc5038f7-23e0-4960-96da-33abaf5935ec 100
Powercfg -setactive scheme_current
Additional Windows Explorer Tuning
 
Registry Hive
Value
Type
Data
Purpose(s)
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 UseDesktopIniCache
 REG_DWORD
 1
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 NoRemoteRecursiveEvents
 REG_DWORD
 1
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 NoRemoteChangeNotify
 REG_DWORD
 1
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 StartRunNoHOMEPATH
 REG_DWORD
 1
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 NoRecentDocsNetHood
 REG_DWORD
 1
HKLMSOFTWAREMicrosoftWindowsCurrentVersionPoliciesExplorer
 NoDetailsThumbnailOnNetwork
 REG_DWORD
 1
HKLMSystemCurrentControlSetServicesMRXSmbParameters
 InfoCacheLevel
 REG_DWORD
 16
HKCR*shellexPropertySheetHandlersCryptoSignMenu
 SuppressionPolicy
 REG_DWORD
 1048576
HKCR*shellexPropertySheetHandlers{3EA48300-8CF6-101B-84FB-666CCB9BCD32}
 SuppressionPolicy
 REG_DWORD
 1048576
HKCR*shellexPropertySheetHandlers{883373C3-BF89-11D1-BE35-080036B11A03}
 SuppressionPolicy
 REG_DWORD
 1048576
HKLMSOFTWAREMicrosoftWindowsCurrentVersionexplorerSCAPI
 Flags
 REG_DWORD
 1051650
HKLMSYSTEMCurrentControlSetControlSession Manager
 SafeDllSearchMode
 REG_DWORD
 1
HKLMSYSTEMCurrentControlSetControlSession Manager
 SafeProcessSearchMode
 REG_DWORD
 1
Windows 2008 R2 – RDP Tuning  
Registry Hive
Value
Type
Data
Purpose(s)
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 AutoEndTasks
 REG_SZ
 1
Determines whether user processes end automatically when the user either logs off.
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 WaitToKillAppTimeout
 REG_SZ
 20000
Determines how long the system waits for user processes to end after the user attempts to log off
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 MenuShowDelay
 REG_SZ
 10
Changes the Start menu display interval
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 CursorBlinkRate
 REG_SZ
 -1
Specifies how much time elapses between each blink of the selection cursor
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 DisableCursorBlink
 REG_DWORD
 1
Enables / Disables Cursor Blink
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 DragFullWindows
 REG_SZ
 0
Specifies what appears on the screen while a user drags a window / Only the outline of the window moves
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 SmoothScroll
 REG_DWORD
 0
Scrolls using smooth scrolling
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 Wallpaper
 REG_SZ
 (none)
Sets Wallpaper to “None”
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktopWindowsMetrics
 MinAnimate
 REG_SZ
 0
Disabled. Window does not animate while being resized
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverrideControl PanelDesktop
 InteractiveDelay
 REG_DWORD
 40
Optimizes Explorer and Start Menu responses Times
XenApp 6.0 – ICA tuning
  
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 AutoEndTasks
 REG_SZ
 1
 Determines whether user processes end automatically when the user either logs off.
Registry Hive
Value
Type
Data
Purpose(s)
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 WaitToKillAppTimeout
 REG_SZ
 20000
Determines how long the system waits for user processes to end after the user attempts to log off
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 MenuShowDelay
 REG_SZ
 10
Changes the Start menu display interval
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 CursorBlinkRate
 REG_SZ
 -1
Specifies how much time elapses between each blink of the selection cursor
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
DisableCursorBlink
REG_DWORD
 1
Enables / Disables Cursor Blink
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 DragFullWindows
 REG_SZ
 0
Specifies what appears on the screen while a user drags a window / Only the outline of the window moves
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 SmoothScroll
 REG_DWORD
 0
Scrolls using smooth scrolling
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 Wallpaper
 REG_SZ
 (none)
Sets Wallpaper to “None”
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktopWindowsMetrics
 MinAnimate
 REG_SZ
 0
Disabled. Window does not animate while being resized
HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverrideControl PanelDesktop
 InteractiveDelay
 REG_DWORD
 40
Optimizes Explorer and Start Menu responses Times
Please note that using these tips is at your own risks.
All these tips have been test with XenApp6 server running on VMware Vsphere 4 and should be test on your own environment.
Sources :
Author : Julien Sybille
CCEA & CCA XenDesktop
14 comments
By: Jonathan Pitre (JakeLD)
10 December, 2010
Very nice article Julien!

According to Thomas Koetzing paper on “Optimizing Logon and Logoff 1.4” available at http://www.thomaskoetzing.de/index.php?option=com_docman&task=doc_download&gid=135

the registry key HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverride dosen’t work anymore with Windows 2003 and 2008/2008 R2 I assume.

Here what he had to say on page 22: “With Windows 2003 those global keys don’t work anymore and has to be set on a per user basis.”

I tested it my self with existing profiles and it didn’t work…but I just read this morning that the profile must be NEW. So who’s right, you or Thomas ? 🙂

Also I suggest the following registry keys:

HKLMSSYSTEMCurrentControlSetControlProcessor
Key: “Capabilities” (dword)
Value: 0007e666
http://support.microsoft.com/kb/2000977

HKLMSYSTEMCurrentControlSetServicesTCPIPParameters
Key: “DisableTaskOffload” (dword)
Value: “1”

http://support.microsoft.com/kb/904946
http://support.citrix.com/article/CTX117491

Regards,

Jonathan Pitre

By: Helmut Hauser (Houzer)
16 December, 2010
Other tuning options are:

1st) Turn off TCP Offloading (at the NIC AND the OS)

OS:
%SYSTEMROOT%SYSTEM32
etsh.exe int tcp set global chimney=disabled
%SYSTEMROOT%SYSTEM32
etsh.exe int tcp set global rss=disabled

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters]
“EnableRSS”=dword:00000000
“EnableTCPChimney”=dword:00000000
“EnableTCPA”=dword:00000000
“DisableTaskOffload”=dword:00000001

2nd) If dealing with APP-V 4.6 64 Bit RDS Client on W2K8R2 this one could be for you [SCCM related]:

[HKEY_LOCAL_MACHINESOFTWAREMicrosoftSoftGrid4.5ClientConfiguration]
“RequireAuthorizationIfCached”=dword:00000000
[HKEY_LOCAL_MACHINESOFTWAREMicrosoftSoftGrid4.5ClientNetwork]
“AllowDisconnectedOperation”=dword:00000001
“Online”=dword:00000000
“DOTimeoutMinutes”=dword:ffffff
“LimitDisconnectedOperation”=dword:00000000
[HKEY_LOCAL_MACHINESOFTWAREMicrosoftSoftGrid4.5ClientPermissions]
“ToggleOfflineMode”=dword:00000000

3rd)

other optimizations:

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters]
“EnablePMTUBHDetect”=dword:00000001
“KeepAliveTime”=dword:00007530
“KeepAliveInterval”=dword:00001388
“TcpMaxDataRetransmissions”=dword:00000005
“EnableBcastArpReply”=dword:00000001
“DisableTaskOffload”=dword:00000001

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesLanmanServerParameters]
“TreatHostAsStableStorage”=dword:00000001

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlFileSystem]
“NtfsDisableLastAccessUpdate”=dword:00000001
“DontVerifyRandomDrivers”=dword:00000001
“NtfsDisable8dot3NameCreation”=dword:00000001

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlLsaKerberosParameters]
“MaxTokenSize”=dword:0000FFFF

[HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionPoliciesSystem]
“DelayedDesktopSwitchTimeout”=dword:00000005

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControl]
ServicesPipeTimeout”=dword:0001d4c0

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesBrowserParameters]
“IsDomainMaster”=”FALSE”
“MaintainServerList”=”No”

As always – do NOT use this in a production environment.
Test it in a lab.
Be very careful when touching the autotuning TCP/IP stack. This can backfire.
Talk with the networking gents.

By: Julien Sybille (Julien.sybille)
20 December, 2010
Jonathan, I made some investigation about HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsRDP-TcpUserOverride

And, my result is that your Right about this key, it’s not applied anymore on 2008R2

So it’s meamn that also HKLMSYSTEMCurrentControlSetControlTerminal ServerWinStationsICA-TcpUserOverride is not applied any more.

Looking on the Citrix Policy we found a few settings that replace this options like the DragFullWindows options.

I will test, all the options that you post and that are not already in my tips and make an update soon.

Please, if someone found other tuning Tips for 2008R2 / XenApp 6, let us know.

Julien Sybille

By: Xavier TINON (xaviertinon@yahoo.fr)
20 December, 2010
Thank you very much for all tuning tips.

For my test, I’ve trying to use de three PowerCFG command line, because my servers will going to 100 % CPU, causing by NTDLL.DLL under the svchost without reasons.

With this three command line enter on all Win2K8 R2 servers, this error is not coming back.
For information, we are running all VM on vsphere 4.1.

Very good job,
Best regards,

Xavier TINON

By: hector lorenzo (worldphoenix)
23 January, 2011
Julien

Can you give please give us an idea of what you mean when you say you have validated these performance tweaks?

Was it done in a production environment and if so what were the actual benefits ?

What I am getting at is that just adding them to a server in a test environment doesn’t really mean much….

Thanks

Hector

By: wilson mosquera (wmosquera)
7 February, 2011
This can also be used to xenserver 5.6 ?

Regard best

Wilson

By: Julien Sybille (Julien.sybille)
7 February, 2011
To respond to your question these tweaks have been applied on a production environment 3 months ago now with XenApp 6 patchs in the same time and itâ��s running fine. the tweaks have reduces my logon time from 37 seconds to 18-20 s, reduce my CPU usage and avoid my XenApp server hangs (when’s applying citrix strategy at user logon for example). Please note that Iâ��ve applied this setting only in one particular client environment, let us know your feedback from your personal test?

Julien

By: Julien Sybille (Julien.sybille)
7 February, 2011
Wilson I will say this can be tested on xenserver 5.6 on a lab environnement. I don’t see any reason why it will not working, but I’ve never try…

Please let us now your result if you try it ?

julien

By: Xavier TINON (xaviertinon@yahoo.fr)
15 February, 2011
Hi ,

Sorry for my late reply.

I confirm that the CFG command have correcting my 100 % CPU use for all servers in test and production environment.

I ‘ have seen, that Citrix have developped a hotfix, that fix this 100 CPU by ntdll.dll on “svchost”. But I don’t have enough time for test it.

Xavier

By: Shawn Bass (Shawn Bass)
22 April, 2011
Hello Julien & Pierre!

Was reviewing this list of items a grabbing a few for a project of mine when I noticed an error in your SuppressionPolicy settings for Windows Explorer optimizations. I think the mistake is when you were entering the values in hex you left a 0 out, so instead of entering 100000 which is 1048576 in decimal you have 10000 which is 65536 in decimal.

Thanks.

Shawn

By: Shaun Ritchie (ritchies)
7 July, 2011
Why do all the Explorer tuning tips relate to Windows XP?

All the links I have followed to Microsoft.com refer only to Windows XP or Windows 2000.

By: Orhan Taskin (otaskin@gmail.com)
7 July, 2011
are there any script or tool to backup/set registry values automatically? Thanks.
By: Shaun Ritchie (ritchies)
18 July, 2011
You can use REG EXPORT at command prompt or maybe the REGISTRY cmd-let in PowerShell.

You can also use group policy preference to enforce the registry settings and tick “Remove this item when it is no longer applied”

By: Scott McKenzie (Scott McKenzie)
25 August, 2011
It is recommended to set MaxWorkItems to 4x MaxMpxCt. With MaxMpxCt set to 2048 this would mean MaxWorkItems should be 8192.

I think you might have it back to front because in your article MaxMpxCt is 4x MaxWorkItems.

(Old Post) XP Performance Optimizations for XenDesktop and Provisioning Server vDisks

Performance Optimizations for XenDesktop and Provisioning Server vDisks
By Paul Wilson

Older post on XP xendesktop, but some information still applies.
Introduction
Many environments have already managed to streamline the image building process and already have familiarity with the many Windows XP performance and optimization tips. Both XenDesktop (XD) and Provisioning Server (PVS) support the Windows XP operating system and can benefit from performance enhancements. For those who are familiar with these performance enhancements, this blog may provide little assistance in the way of new information. None of the optimizations below are required, but they are available here for your convenience if they make sense in your environment.

The optimizations are put into three sections: Those that apply to the current user profile, those that apply to all users on the machine, and recommendations before creating the vDisk. The first section deals with the items that can be set in the default user profile. The second section deals with settings that can be set by the administrator for all users that work on the machine. The final section recommends a few things to do before taking the vDisk image. When available, the section will provide links to the URL on Microsoft?s website that explains the setting further.

Settings for the Default User Profile
This section lists a few of the settings that will improve the user experience but are set at the user profile level.  The recommendation is to create a generic user and then set the applicable settings, when completed, replace the default user profile with the generic user profile, the steps for which are found at the end of this section.

Force Offscreen Compositing for Internet Explorer
Turning this setting off removes any of the flickering that may display when using Internet Explorer through XenDesktop, by telling Internet Explore to fully render the page prior to displaying it. This is especially helpful on Internet Explorer 7.

Open Internet Explorer
Select Tools >> Internet Options from the menu
Select the Advanced tab
In the Browsing section, enable the checkbox for ?Force offscreen compositing even under Terminal Services?
Click OK to save the changes
Restart Internet Explorer
More information available at http://support.microsoft.com/kb/271246/en-us

Remove the Menu Delay
The Start menu has a built-in delay of 400 milliseconds. To speed the menu response time, follow these steps to remove the delay:

Start the Registry Editor (Regedit.exe)
Navigate to  HKEY_CURRENT_USERControl PanelDesktop
Set the value of MenuShowDelay to 0
Exit the Registry Editor

Remove Unnecessary Visual Effects
Disabling unnecessary visual effects such as menu animations and shadow effects that generally just slow down the response time of the desktop.

Right-click My Computer
Click Properties
Click Advanced
Click the Settings button under the Performance section
Click ?Adjust for best performance?
If you want to keep the XP Visual Style, scroll to the bottom and check the last box titled ?Use visual styles on windows and buttons?

Disable the desktop cleanup wizard
To stop the wizard from automatically running every 60 days:

Right-click a blank spot on the desktop, and then click Properties to open the Display Properties dialog box
Click the Desktop tab
Click Customize desktop to open the Desktop Items dialog box
Disable the ?Run Desktop Cleanup Wizard every 60 days? setting
Click OK twice to close the dialog boxes
More information available at http://support.microsoft.com/kb/320154

Disable Automatic Searching of Network Printers and Shares
Automatic search periodically polls your network to check for new shared resources and adds relevant icons into My Network Places if anything is found. If you wish to prevent XP from regularly searching your network unnecessarily then follow these steps:

Open the Control Panel
Select Folder Options.  If you use the Control Panel Category View you?ll find Folder Options under Appearance and Themes
Click the View tab
In the Advanced Settings list, disable the ?Automatically Search for Network Folders and Printers? setting
Click OK

Disable the Windows XP Tour Notifier
If you did not turn this off before you logged in as your base user for the default profile, you can manually disable the prompt on a per-user basis by following these steps:

Start Registry Editor (Regedit.exe)
Navigate to HKEY_CURRENT_USERSoftwareMicrosoftWindowsCurrentVersionAppletsTour
On the Edit menu, point to New, click Dword Value, type RunCount
Set the data value to 0 (zero), and then click OK
Quit Registry Editor
More information available at http://support.microsoft.com/kb/311489

Turn off Automatic Updates
Since you are running a read-only image, using automatic updates will cause the operating system to continually download the same updates each time the image is booted. The best course of action is to turn it off. You have three options that can be used to disable the service:

Use the PVS Optimizer tool and leave the ?Disable automatic update service? box checked
~ or ~

In the Services Control Panel, change the Startup Type of the Automatic Updates service to ?Disabled?
~ or ~

Run GPEDIT.MSC and navigate to:  Local Computer Policy > Computer Configuration >  Administrative Templates > Windows Components >  Windows Update. Set the ?Configure Automatic Updates? setting to ?Disabled?

Turn off Language Bar
If there is no need for the language tool bar (the pen icon in the systray) you can disable it using either of these two methods.

Right-click taskbar > Toolbars and uncheck the ?Language Bar? option.
~ or ~

Navigate to Control Panel > Regional and Language Options > Languages (tab) > Details (button) > Language Bar (button at bottom). Disable the ?Show the Language bar on the desktop? and ?show additional Language bar icons in the taskbar?.

Make the User Profile the Default User Profile
When you are done completing all the User Profile Settings (using a generic user) you can copy the profile over to the default user using the process below.

Login as an administrator (Local Administrator is recommended) not as the base user for the profile because you cannot copy a profile that is in use.

Right-click on My Computer
Choose Properties
Select the Advanced tab
Click the Settings button under the ?User Profiles? section
Select your base user profile where the changes above were made and click Copy To
Click the Browse button and browse to C:Documents and SettingsDefault User
Click OK once to save the path
Click the Change button under ?Permitted to use?
Enter Everyone
Click OK to save
Click Yes to confirm overwriting of the default user profile
NOTE: Before copying it over, be sure to remove any user or machine specific data for the ICA Client, the ICA Streaming Client, Password Manager, and EdgeSight. Since the image prep for these items is beyond the scope of this blog, I will save it for a topic another day.

If you would like to know more about user profile management in general, check out David Wagner?s blog on the Citrix User Profile Manager available at http://community.citrix.com/blogs/citrite/davidwag/.
Settings for the Machine
This section provides a list of the optimizations that will affect all users of the image. These settings are usually set after logging in as an administrator.

Power Configuration Settings
Two of the power settings can adversely affect the performance of PVS. One of them is the hard disk power savings. If the PVS server is using a local hard disk for the vDisk cache, you do not want the operating system to power down the local drive. The other setting is the Hibernate setting. The PVS Optimizer tool will disable hibernating, but you can manually do it as well. Here are the steps for disabling the power settings:

Open Control Panel
Select the Power Options applet
Select the Power Schemes  tab
For the default power scheme, set ?Turn off hard disks? setting to Never
Select the Hibernate tab
Disable the ?Enable Hibernation? setting
Click OK to save the settings
Delete the c:hiberfil.sys hidden file

Permanently Remove the Language Bar
If there is no need for the language tool bar to be installed at all, you can permanently remove it by running the following command from a command-prompt:

Regsvr32.exe /u msutb.dll

To reinstall it because you later found out you should not have removed it, you can run this command:

Regsvr32.exe msutb.dll
Disable TCP Checksum Offloading
This performance optimization is highly recommended by both Citrix and Microsoft for all Windows XP workstations that will be communicating over the network with other Microsoft resources. To work around this problem, turn off TCP checksum offloading on the network adapter using these steps:

Start the Registry Editor (Regedit.exe)
Navigate to the registry key HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters
In the right pane, make sure that the DisableTaskOffload registry entry exists. If this entry does not exist, follow these steps to add the entry:
On the Edit menu, point to New, and then click DWORD Value.
Type DisableTaskOffload, and then press Enter
Click DisableTaskOffload.
On the Edit menu, click Modify
Type 1 in the Value data box, and then press Enter
Exit Registry Editor
More information available at http://support.microsoft.com/kb/904946/
Turn off Security Center
To disable the Security Center service, so users are not prompted when the firewall or anti-virus updates are out-of-date, you can disable it by peforming the following steps:

Open the Services Control Panel
Edit the Security Center service properties and set the Startup Type to ?Disabled?

Disable Last Access Time Stamp
Windows XP has a habit of time stamping all the files it reads with the date and time it was last accessed. While this is a nice feature, it is not always necessary in PVS environments where the files are statically supplied from a standard image and no backup software will be used. Putting a timestamp on a recently read file creates a write access every time a read is executed. With Provisioning Server, these writes go to the vDiskCache file increasing network traffic if cached on the PVS server. To disable the last access timestamp behavior, complete the following steps:

Start a command prompt
Type FSUTIL behavior set disablelastaccess 1 and press Enter
Requires a reboot to take effect

Disable the Windows XP Tour Notifier for New Users
Windows XP likes to notify all new users that an XP tour can be taken. While this is a nice feature for new users, it typically is annoying for existing users. To suppress the XP tour prompt for all new users, follow these steps:

Start Registry Editor (Regedit.exe)
Navigate to the registry key HKEY_LOCAL_MACHINESoftwareMicrosoftWindowsCurrentVersionAppletsTour
If the Tour key does not exist, follow these steps to create it:
From the Edit menu choose New
Click Key and type Tour as the key name
On the Edit menu, point to New, and then click Dword Value
Type RunCount as the name for the new value
Set the data value for the RunCount value to 0 (zero), and then click OK
Quit the Registry Editor
More information available at http://support.microsoft.com/kb/311489
Turn off System Restore
System Restore is the feature that allows a computer system to be rolled back, or restored, to a point before certain events took place, for example, prior to specific software or hardware installations. When you are using a standard mode (read-only) vDisk, there is no reason to have the System Restore feature enabled. The PVS Optimizer tool disables the System Restore feature, but if you are not using that tool, you should complete the following steps:

Right-click My Computer, and then click Properties
On the Performance tab, click File System, or press Alt+F
On the Troubleshooting tab, click to select the Disable System Restore check box
Click OK twice, and then click Yes when you are prompted to restart the computer
To re-enable System Restore, follow steps 1-3, but in step 3, click to clear the Disable System Restore check box
More information available at http://support.microsoft.com/kb/264887
Disable Windows Indexing Services
Windows Indexing service adds overhead to the PVS vDisk by reading the files from the vDisk for indexing. Use one of the following three methods to disable Indexing:

Use the PVS Optimizer tool and leave the ?Disable Indexing Services? setting enabled
~ or ~

To turn off indexing at the drive level, perform these steps:
Open My Computer
Right-click on the drive on which you wish to disable the Indexing Service
Select Properties
Under the General tab, disable the ?Allow the Indexing Service to index this disk for fast file searching? setting
~ or ~

To disable the indexing service at the service level, perform these steps:
Click Start, Run, type services.msc then press Enter or click OK
Scroll to the ?Indexing Service? in the right-hand pane and double-click it
Change the Startup type to ?Manual? or ?Disable? and Apply
Click the Stop button and wait for the service to stop then click OK

Modify the Windows Service Timeout
In environments with shift changes and large amounts of virtual machines rebooting some virtual machines may fail to register because the Windows Service timeout may be reached before Citrix Desktop Service starts.  The Windows Service default timeout is 30 seconds which may not be long enough for all the services to when the virtual machines are coming online simultaneously. We recommend changing the 30-second default to 120-seconds to give the services times to start before the Citrix Desktop Service starts.   The timeout value is represented in milliseconds so 60 seconds = 60000 ms.  The following registry change can be made to lengthen the Windows Service timeout period.

Start Registry Editor (Regedit.exe)
Navigate to the registry key HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControl
If the ServicesPipeTimeout value is not present, use the following steps to create it:
Click the Control subkey
On the Edit menu, point to New, and then click DWORD Value
Type ServicesPipeTimeout, and then press Enter
Right-click the ServicesPipeTimeout key and then click Modify
Click Decimal
Type 120000, and then click OK
Quit the Registry Editor
Reboot for the changes to take effect

Disable Remaining Unnecessary Services
You can go through the list of other services that are configured on Windows XP and disable any ones that will not be used in your environment. Two possible services are the Wireless Zero Configuration service and the Themes service.
Enable ClearType
To enable ClearType and make any adjustments to suit your eyes, go to the Microsoft Typography pages and follow the simple instructions. You can adjust ClearType in the Control Panel after installing the software at the link.
Recommendations Before Imaging
Below is a list of recommendations that can be completed right before creating the vDisk image. Most of these are designed to optimize the layout of the files on the disk so that PVS server can operate at maximum efficiency.
Zero Deleted Files
SDelete is a secure file delete utility that can be used to free and cleanup unused space on the image. In short, it zeroes out any files that have been freed up by the operating system and helps the image run faster. For more information about how it works or to download it, visit the URL below. The recommended options are -z and -c. (sdelete -z -c)

Usage: sdelete [-p passes] [-s] [-q] <file or directory>

sdelete [-p passes] [-z|-c] [drive letter]

More information available at http://technet.microsoft.com/en-us/sysinternals/bb897443.aspx
Defragment the Local Disk
Run the some type of disk defragmenter tool to optimize the files on the image. It is best to run the utility after removing the pagefile.sys and hiberfil.sys files. If you will use a page file, just re-enable it after you defragment the disk, that way the page file is contiguous. The Windows Defragmenter can be found at Start >> Programs >> Accessories >> System Tools >> Disk Defragmenter.
Flush the DNS cache
Flush the DNS Cache using the ipconfig /flushdns command. This prevents any IP addresses cached on the read-only disk from interfering with DNS resolution at a later date.
Run ChkDsk
Verify the file system has no missing file links or pieces by running chkdsk –f from a command prompt.

Citrix Image creation

Automated Master Image Creation


//www.facebook.com/plugins/like.php?href=http%3A%2F%2Fxenappblog.com%2F2014%2Fautomated-master-image-creation%2F&send=false&layout=button_count&width=120&show_faces=false&action=like&colorscheme=light&font&height=21“>

WSUS

When deploying a new computer or server image using MDT / SCCM, one of the most time consuming parts is Windows Update. So if the reference image isn’t getting updated on a regular basis, the deployment time will increase each and every month.

To achieve the fastest deployment the reference image should be automatically updated with the recent Windows Updates at least once a month.

Automated Master Image Creation-02

The recommended platform for building references images is the free Microsoft Deployment Toolkit, even if the images is going to be used with SCCM. So let’s get started with Automated Master Image Creation.

Open CustomSettings.ini for your Build Deployment share and add the following to the top:

And at the bottom you add a section for each and every OS type you want to automate with the correct MAC Address:

If you’re using Windows 2008 R2 or Windows 7 you should add the ProductKey= as well to skip the licensing screen.

Automated Master Image Creation-03

To be able to move a captured image to the correct location in a Task Sequence you need to create a script in the scripts catalog, in my case \mdt-01.ctxlab.localMDTBuildLab$Scripts.

Call the script Realocate.cmd and add the following:

And then add a Command Line task at the end with this command:

Automated Master Image Creation-04

So in the example above, if a computer with the MAC Address 00:15:5D:01:FB:32 boots on the ISO it will automatically start the Task Sequence W81, capture the image, move it to the Production Deployment Share and shutdown the computer.

All this without lifting a finger, a real zero touch deployment. Relax and check theAutomated Master Image Creation status now and then from you mobile device.

Automated Master Image Creation-05

Wouldn’t it be cool if we could leverage PowerShell to automatically build the master images as a Schedule Task? The following script will create and start a Hyper-V VM with the correct MAC Address and delete it when the capture is finished.

To further improve the building time of the reference images, the captured images should be copied back to the MDT Build Deployment Share once in a while. This way you will update the latest version instead of the base Microsoft ISO image with hundreds of Windows Updates.

So how long does it take to deploy Citrix XenDesktop 7.1 totally automated which Gunnar Berger referred to above? 28 minutes from start to finish, including GPMC and the SCVMM Console.

To further reduce time and space take a look at this article from fellow CTP Aaron Parker : Cleaning up and Reducing the Size of your Master Image.

To learn more check my free training on Automated Master Image Creation.

Source: http://stealthpuppy.com/cleaning-up-and-reducing-the-size-of-your-master-image/

Cleaning up and Reducing the Size of your Master Image

Compressed Car

There’s typically not too much that you can do to reduce the size of your master image. You might use application virtualization or layering solutions to reduce the number of master images, but once you work out what needs to go into the core image, that’s going to dictate the size of the image.

Reducing the size of your master image can help reduce the capacity required to store master images and virtual machines (dedupe helps, of course) and spend less cycles transmitting an image across the network for physical PCs.

An easy win, is running the Disk Clean-up tool included in Windows (since Windows XP) and fortunately this tool can be automated to run at the end of a build (e.g from MDT or ConfigMgr). For physical PCs or persistent desktops, this tool could even be run as a scheduled task.

Microsoft released an important update for Windows 7 last year that can result in a significant reduction in disk space: Disk Clean-up Wizard addon lets users delete outdated Windows updates on Windows 7 SP1. The same feature was originally delivered with Windows 8. (Windows 8.1 Update 1 is expected to reduce disk space requirements again).

Here’s an example system where I’ve run the Disk Clean-up tool that has resulted in a 3.4 GB reduction in disk usage – on the left is the before image, on the right is after the cleanup. (I’m cheating a bit here, this is a system that has gone from Windows 7 to Windows 7 SP1, hence the reason for such a large change).

Compare Disk Cleanup Before and After

Disk Clean-up can remove a number of interesting items, most of which will actually be applicable for PCs and persistent desktops post-deployment. Here’s the items that Disk Clean-up can manage on Windows 8.1:

Disk Cleanup options

To automate Disk Clean-up, use the following steps:

  1. Elevate a command prompt
  2. Run CLEANMGR /SAGESET:<number> (where <number is any number between 1 and 65535)
  3. Select each of the items to clean up
  4. Click OK

To run Disk Clean-up from a script run CLEANMGR /SAGERUN:<number> (where <number> is the same number use with SAGESET.

To automate the process of running Disk Cleanup, the following script can be used to enable all of the tool’s options in the registry and then execute the cleanup process. This script should work for both Windows 7 and Windows 8 and would be useful to run at the end of a deployment. This example uses 100 as the configuration setting, so you would run CLEANMGR /SAGERUN:100 to run Disk Cleanup with these settings.

It’s important to note that while Disk Clean-up exists on Windows 7 and Windows 8, it does not exist on Windows Server unless the Desktop Experience feature is installed (i.e. a Remote Desktop Session Host).

If your image is Windows 8.1 or Windows Server 2012 R2, then the following command is available to perform an even more in depth cleanup of the WinSXS folder, making some additional space available:

Running Disk Clean-up and the above DISM command in a script to clean up your master image should result in a smaller image. Don’t forget that this approach is also useful for persistent desktops – unless you’re using some type of dedupe solution, then there’s potentially gigabytes per desktop that can be removed.

There is one more method worth for reducing space worth mentioning – the Uninstall-WindowsFeature PowerShell cmdlet in Windows Server 2012 and Windows Server 2012 R2. This can go a long way too to reducing the disk footprint by completely removing features from Windows (making them unavailable for install).

For instance, if you’re deploying a Remote Desktop Session Host, there’s no need for IIS or Hyper-V to be in the component store. See this blog post article for full details: How to Reduce the Size of the Winsxs directory and Free Up Disk Space on Windows Server 2012 Using Features on Demand

[May 14 2014] Microsoft has released this update for Windows Server 2008 R2, which can be on the original KB article here: Disk Cleanup Wizard add-on lets users delete outdated Windows updates on Windows 7 SP1 or Windows Server 2008 R2 SP1. The update is available from Windows Update and WSUS.

XenApp 6.0 tidbits from www.thomaskoetzing.de

source: http://www.thomaskoetzing.de/

Citrix XenApp 6.0/6.5 and Microsoft Windows Server 2008 R2
I highly recommend to install service pack 1 for Server 2008 R2 because it fixes several Remote Desktop Service (Terminal Service) issues that are also affecting XenApp. Read the following KB for Known Issues CTX126711 with SP1 and CTX129241.

I also highly recommend installing the Rollup Pack 1 for XenApp 6.0 and XenApp 6.5. See also Citrix recommended Microsoft and Citrix Hotfixes for XenApp 6 and Windows Server 2008 R2 at CTX129229

Citrix XenApp 6.0/6.5 update script
I modified my existing update script just for XenApp 6.0 on Server 2008 R2. The script will not work with older versions of XenApp. Also the script is not downloading or applying components like the role manager etc.

Download the script here: XA6xUpdate_v1.6.zip

Citrix XenApp 6.0/6.5 things to know:

Citrix XenApp 6.0/6.5 and Server 2008 R2 Service Pack 1 (SP1)

Citrix XenApp 6.0 / 6.5 bug list:


Windows Server 2008 R2 bug list:

  • Shadowing users with multi monitor is not possible right now and therefore also NOT for XenApp 6.0, this is really a pain for support personal
    Error: Shadow failed. Error code 7044
    http://support.citrix.com/article/CTX125693
  • Annoying “Single-click” folder option for users on Server 2008 R2
    The group policy “Classic Shell activate” (Windows-Explorer) should fix that by enabling it but actually you should disable it! Then users have to set the folder option back and forth or you can set a registry value to do it for the users.

XenApp 6.0 Troubleshooting tools

Citrix Troubleshooting Tools

General
Advanced Access Control
Application Isolation Environment
Fatal Errors and Application Crashes

Citrix Access Gateway
Citrix Licensing

Citrix Password Manager

Clients

Content Redirection

CPU Utilization and Memory Optimization

Data Store Issues

Independent Management Architecture

Installation Manager

Installation/Migration

Load Balancing

Login/Authentication

NetScaler
Performance

Ports and Printing

Resource Manager

Seamless

Shadowing

Teros

Virtual IP (VIP)

Web Interface

General

-These tools can be used in most problem types.

What What it does More information
Citrix Utility AppEdit Simple GUI tool to allow quick management of published applications on a Citrix Server farm. Useful to quickly move and change published applications between servers to isolate a test environment or take a suspect server out of the possible causes. Can be used in conjunction with CHFarm to isolate a server.
Centralis Centinel Reboot Monitors connections to a Terminal Server and reboots it once all users are logged off or disconnected. Can also automatically run a script before reboot. This tool can be used to automate the implementation of a fix or registry hack and then reboot the server without causing any data loss for users.
CHFARM

-Available with the Citrix Presentation Server or XenApp 5 installation

Presentation Server Administration Guide – Appendix A

An administrator can use CHFARM to move a server from its current server farm. The server can be moved to an existing server farm or can be moved into a new farm which is created at the same time that the server is moved.

CHFARM can be used to isolate problems and prove whether the data store is involved, or causing the issue.

If the server you want to move provides information for a Resource Manager summary database, update the summary database before using chfarm.

CTX736659

Common Diagnostics Facility (CDF)

– Available with the Citrix Presentation Server/XenApp installation

The Common Diagnostic Facility (CDF) is a utility available as part of the Access Suite Console that allows administrators to collect traces for problem analysis/diagnosis.

Traces can be collected on all retail binaries from MetaFrame Presentation Server 3.0 onwards without the need for stopping and starting services or disrupting users. Traces can be started and stopped remotely through the Access Suite Console, and can be easily packaged and uploaded to Citrix Technical Support to assist in problem diagnosis and resolution.

Usage: Generally this tool is used on the advice of Citrix Technical Support in relation to a specific support case.

There are many articles on support.citrix.com detailing which “modules” to select for tracing particular types of problems.
[Document Not Found] CDF Viewer is a utility that allows you to view the output of a diagnostics log file that is created when a CDF trace session is completed using either the MetaFrame Access Suite Console or the CDF Viewer itself.
CDFControl CDFControl is a software debugging utility that captures Citrix Diagnostic Facility (CDF) trace output messages that are output from the various Citrix product modules. To get this tool you need to authenticate to our support site. Support.citrix.com
User Profile Deletion Utility Delprof is a command-line utility that you can use to delete user profiles on local or remote computers running Windows 2000, Windows XP, and Windows Server 2003. User profiles can grow large and may take up considerable disk space, especially if there are several users using one computer. Can be used in scenarios where there is only one user experiencing the problem or to clean local profiles off a Presentation Server.
WireShark Used to gather and analyze Network in several file formats. This tool can capture data “off the wire” or can read data saved in files from previous captures. Displayed data can be filtered to narrow down your search to traffic to and from a particular IP address. Understanding the protocol you are analyzing (such as TCP/IP) is a prerequisite to understanding the results of this tool. Example case studies: CTX110007
ExamDiff This tool is for comparing text files. Very useful for comparing good CDF traces with bad CDF traces. Capture two separate traces of any technical issue experience. Parse your output pointing to TMF files. Save the parsed output from CdfViewer to CSV format and use Examdiff to speed up the comparison of the output. Could also be used for comparing XML files for Web Interface setups.
fc.exe

– Native to Windows 2000, Windows 2003 and Windows 2008

Can be used to compare ntuser.dat files. fc /?
Process Monitor Permissions issues, such as user can not perform a task but an administrator can. Look for “File Not Found” or ”Permission Denied” on specific registry keys or values or search for .wav files for system beeps.

Example case studies: CTX109109

ICAPORT

-Installed with Presentation Server

Use icaport to query or change the TCP/IP port number used by the ICA protocol on the server. The default port number is 1494. The port number must be in the range of 0–65535 and must not conflict with other well-known port numbers. CTX101594
MessageHistory Simple GUI tool that traces window messages, like Spy++ or Winspector, but easier to use.

Consists of two executable files (32- and 64-bit). Simply start the exe, select via tick boxes which messages should be traced and click start.

Used to troubleshoot issues with GUI, Seamless, Clipboard, client, and so on.
MSGHOOK.EXE V4.0.1.0

– Presentation Server 4.x Server CD – Support – Debug: Windows 2003

MsgHook.exe (Message Hook) is an executable from Citrix Independent Management Architecture. MsgHook is a command line utility that displays all IMA traffic on a member server – Note: Execute msghook only if information is requested by a Citrix Technical Support representative or a Citrix engineer. When invoked, this command significantly reduces Presentation Server performance.
[Document Not Found] The PDBFinder Deluxe utility simplifies finding rare program database (PDB) or debug (DBG) files (symbols) or determine a hotfix corresponding to a given module name and date.
Process Explorer v15.23 Process Explorer shows information about which handles and DLLs processes have opened or loaded. Can be used for tracking down DLL-version problems or handle leaks, and provide insight into the way Windows and applications work. For Application Isolation Environment you can see if DLLs and Named Objects are virtualized.

For Virtual Memory Optimization issues, you can use Process explorer to highlight relocated DLLs.

PsExec v1.98 PsExec is a light-weight telnet-replacement that lets you execute processes on other systems, complete with full interactivity for console applications, without having to manually install client software. Can be used in combination with SystemDump.exe to execute a system dump remotely.
Qfarm (query farm)

– Installed with Presentation Server

Use this to display information about servers within an IMA based server farm. For Example: display the current load on each server with qfarm /load, list all currently active servers in the farm with qfarm /online. Presentation Server Administrator’s Guide – Appendix A
QRYClientIP Command line tool to query a session’s client IP. Used to allow one login script to process specific commands for specific IP addresses or IP segments.
QUERY PROCESS Displays information about processes running on a terminal server. You can use this command to find out which programs a specific user is running, and also which users are running a specific program.
QUERY SESSION Displays information about sessions on a terminal server. The list includes information not only about active sessions but about other sessions that the server runs.
QUERY TERMSERVER Displays a list of all terminal servers on the network.
QUERY USER Displays information about user sessions on a terminal server.
QueryDC

– Presentation Server installation CD – Support – Debug

The QueryDC utility is used to determine the data collector for a given zone. Without any parameters, Querydc defaults to the host server’s zone and returns the zone name and name of the current zone data collector.
[Document Not Found]

– Presentation Server installation CD – Support – Debug

Use this utility to display information about member servers in the farm. Executing queryhr with no parameters lists all servers in the farm. Queryhr obtains information from the local host cache.
Regshot This tool compares two registry snapshots. This tool can be used to troubleshoot issues with server configuration/custom registry settings
SHADOW Enables you to remotely control an active session of another user.
wbemtest.exe

– Native to Windows 2000, Windows 2003, and Windows 2008

Troubleshooting Issues that Involve WMI Data. This would mainly include issues with licensing, the Access Suite Console, Suite Monitoring and Alerting or the Presentation Server Management pack for MOM.

The tool can be used to test basic functionality of WMI providers.

BareTail(Pro), BareGrep(Pro) log file monitoring tool

file finding and text searching tool

Fiddler Fiddler is a HTTP Debugging Proxy which logs all HTTP traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP Traffic, set breakpoints, and “fiddle” with incoming or outgoing data. Fiddler is designed to be much simpler than using NetMon or Achilles, and includes a simple but powerful JScript.NET event-based scripting subsystem.
MemProfiler .NET Memory Profiler is a powerful tool for finding memory leaks and optimizing the memory usage in programs written in C#, VB.NET or any other .NET Language.

It allows you to retrieve information about all instance allocations performed on the garbage collected heap (GC heap) and all instances that reside on the GC heap. The retrieved information is presented in real-time, both numerically and graphically. Using the new unmanaged resources tracker, information about unmanaged resources such as HBITMAP, HWND and unmanaged memory is also presented.

Microsoft Network Monitor (NetMon) A network protocol analyzer that captures network traffic for display. Useful when troubleshooting network issues related to performance, disconnects, etc.

Back to top

Application Isolation Environments

What What it does More information
AIERUN

– Available with the Citrix Presentation Server/XenApp installation

This is the command to launch an isolated application. Primarily for use in scripting environments. Aierun is an internal launcher used by Presentation Server during application launching.

Can be used to modify shortcuts in a desktop to launch an application in Isolation.

For troubleshooting purposes this could be used to test running the application in Application Isolation Environment without publishing it (remove a layer of complexity)Syntax Aierun AieName c:pathapp.exe

Presentation Server Administration Guide – Appendix A + Errata

Dependency Walker Dependency Walker (Depends) is a useful GUI tool for pointing to an executable on a system and listing all the DLLS required by that DLL, it will highlight any missing DLLs or conflicts. Dependency Walker (Depends) is a useful tool for trouble-shooting application errors, file registration errors, memory access violations, and invalid page faults.

Also useful for troubleshooting Application Isolation Environment issues.

Back to top

Fatal Errors and Application Crashes

– any fatal errors on servers, Dr Watson userdump

What What it does More information
How to Use the Citrix Symbol Server to Obtain Debug Symbols The Citrix Symbol Server can be used to obtain the symbol files necessary to debug Citrix products. The symbol server currently contains symbols for the CPS3.0 to XenApp5.0 products. You must use a debugging tool to connect to the Citrix Symbol Server
[Document Not Found] The Citrix DumpCheck tool helps to diagnose corrupt dumps. This tool is easier and quicker to use than launching WinDbg.exe and then loading the dump, and so on. Before sending dump files to another party, check that the file is not corrupt
[Document Not Found] The Citrix DumpCheck Utility helps diagnose corrupt dumps. This tool is easier and quicker to use than launching WinDbg.exe and then loading the dump, and so on. Unlike the Explorer extension described in CTX108825 – Citrix DumpCheck Explorer Extension version 1.4, it doesn’t require changing the environment. CTX108890
Dependency Walker Dependency Walker (Depends) is a useful GUI tool for pointing to an executable on a system and listing all the DLLS required by that DLL, it will highlight any missing DLLs or conflicts Dependency Walker (Depends) is a useful tool for trouble-shooting application errors, file registration errors, memory access violations, and invalid page faults.

Also useful for troubleshooting Application Isolation Environment issues.

Dr. Watson

– Available with the Windows 2003 installation

Collect the dump file when applications crash or suddenly exit. Enable Dr Watson by issuing the DrWtsn32.exe –I command at the command prompt. Further configuration options are available by issuing the command DrWtsn32.exe.

When enabled, Dr Watson generates a dump file and a log file in the Documents and SettingsAll UsersDocumentsDrWatson directory (by default). The log file can be used to determine basic information about the error, and the dump file can be analyzed with WinDbg.

Example case studies: CTX108312

LiveKD LiveKD allows you to run the KD and Windbg Microsoft kernel debuggers, which are part of the Debugging Tools for Windows package, locally on a live system. Execute all the debugger commands that work on crash dump files to look deep inside the system. See the Debugging Tools for Windows documentation and our book for information on how to explore a system with the kernel debuggers. Citrix Technical Support may request you to use LiveKD in certain circumstances
LoadOrder If you need to see the order in which the system loads device driver, this tool is for you! Can be useful in resolving device driver conflicts, exp when multiple drivers for the same device exist on the system.
[Document Not Found] The PDBFinder Deluxe utility simplifies finding rare program database (PDB) or debug (DBG) files (symbols) or determine a hotfix corresponding to a given module name and date.
[Document Not Found] The SystemDump GUI utility can force a server to generate a memory dump from the session or when a keyboard is not available or non-standard (fatal error).
UMDH UMDH dumps information about the heap allocation of a process and can be used to determine memory leaks. CTX104201
User Mode Process Dumper Version 8.1 You can use the Userdump.exe tool to generate a user dump of a process that shuts down with an exception or that stops responding (hangs). Before dumping a hanging process, it is helpful to take a screenshot of the Qslice.exe window. This output will assist in dump analysis. Microsoft Knowledge Base article 241215
WinDBG Use WinDbg to open crash dumps obtained from Dr. Watson, manually generated userdumps, or Windows crashes. Once the crash dump has finished loading, many commands can be used to provide helpful information about the error.

The command !analyze –v performs basic exception analysis, reporting the error that occurred and the module in which it occurred.

The command lmv can be used to show all of the modules loaded, including their full path and version information.

Search support.citrix.com for “WinDBG” to get a list of How To guides.
WinDBG 64-bit This is the Windows debugger for 64-bit platforms.

Back to top

Citrix Access Gateway

What What it does More information
[Document Not Found] Secure Access Client Remover is used to detect if any Secure Access Client modules or registry entries are left after uninstalling it. If the installation of the Secure Access client is suspected to cause a conflict with another application and has to be removed for testing.

CTX108092

Back to top

Citrix Licensing

What What it does More information
LMNEWLOG

– Installed on Citrix License Server

The lmnewlog utility switches the report log file by moving the existing report log information to a new file, then starting a new report log with the original report log file name. Advanced Concepts Guide
LMSWITCH

– Installed on Citrix License Server

The lmswitch utility switches the debug log file written by the Citrix vendor daemon by closing the existing debug log for that vendor daemon and starting a new debug log for that vendor daemon with a new file name. Advanced Concepts Guide

Back to top

Citrix Password Manager

What What it does More information
CTXDOMAINPREP.EXE

– Password Manager CD – Tools folder: Password Manager 4.0

A domain administrator can use the Active Directory Setup for the MetaFrame Password Manager utility to set the correct permissions for these objects. This tool is a utility, run from a command line, that sets the correct permissions for these objects on
CTXFILESYNCCLEAN.EXE

– Password Manager CD – Tools folder: Password Manager 4.0

CtxFileSyncClean.exe removes any folders in your central store that point to objects that were deleted. This ensures that your environment contains the most up-to-date information.
CTXFILESYNCPREP.EXE

– Password Manager CD – Tools folder: Password Manager 4.0

Delegating administration of a Citrix Password Manager central store to a group or user account that is not a domain administrator. By default, no permissions are allowed to propagate from root share to the child folders CentralStoreRoot and People.
CTXNWFILESYNCPREP.EXE

– Password Manager CD – Tools folder: Password Manager 4.0

If you plan to use a Novell NetWare folder as your central store, you must create (while logged on with supervisor rights) the shared folder and a folder named People inside the shared folder.
CTXSCHEMAPREP.EXE

– Password Manager CD – Tools folder: Password Manager 4.0

Delegating administration of a Citrix Password Manager central store to a group or user account that is not a domain administrator. The schema preparation tool must be run by a member of the Schema Administrators group for the target forest.

Back to top

Clients

What What it does More information
CHANGE CLIENT

– Available with the Citrix Presentation Server/XenApp installation.

Presentation Server Administration Guide – Appendix A

“Change client” changes the current disk drive, COM port and LPT port mapping settings for a client device.

“Change Client” or “CHGCDM” command

Can be used to query or refresh client setting or printer settings. May correct settings for a timing issue at connect time or clients reconnecting from a new device.
FTACLN The FTACLN utility can be used to clean up the file type associations in the Windows Registry on the device running the Program Neighborhood Agent.

After uninstalling the Program Neighborhood Agent Version 6.3x from a client device and restarting, some of the file types associated with the PNAgent can remain on the client device. This is evident, for example, when you double-click a .txt file in Explorer and the “Executable Cannot be Found” Windows message appears.

CTX101594

Can also be found on:

Presentation Server 4.x Server CD – Support – Debug

MessageHistory Simple GUI tool that traces window messages, like Spy++ or Winspector, but easier to use.

Consists of two executable files (32- and 64-bit). Simply start the exe, select via tick boxes which messages should be traced and click start.

Used to troubleshoot issues with GUI, Seamless, Clipboard, client, and so on.
MSI Installation Log

– Available with the Windows installation

The Microsoft Windows Installer Tool can generate a log of an MSI-based installation.

To enable this the MSI file must be installed using the MSEXEC command from the command prompt.

Microsoft Knowledge Base article227091
Repair Clipboard Chain 2.0.1 The RepairCBDChain utility temporarily restores clipboard functionality.

Run the RepairCBDChain utility on your workstation and/or inside the session desktop

[Document Not Found] The RepairCBDChain64 utility for x64 platforms.
TWCONFIG

– Installed with Presentation Server

Use twconfig to configure ICA display settings that affect graphics performance for clients. Presentation Server Administration Guide – Appendix A
WindowHistory 4.0 for 32-bit platforms The WindowHistory utility complements Spy++ and other tools. It constantly monitors windows in the system (session) and records all changes made to it. Information about windows can then be saved for further analysis.
[Document Not Found] This is the version of the WindowHistory tool that works on ARM processors.
WindowHistory64 4.0 for x64 Platforms This is the version of the WindowHistory tool that works on 64-bit platforms.
Winspector Winspector is a freeware that can be used instead of Spy++ with mostly the same functionality.

It captures all messages (keyboard input, mouse movements/click, changes in the window size and location, and so on) sent to a particular application for identifying issues with ICA clients, compare results inside a session and outside a session, keyboard mapping issues, and so on.

Useful for troubleshooting some Seamless Windows issues and allows distinguishing of the Window Class Name, which is then used to set an exception flag.

CTX101644

Back to top

Content Redirection

What What it does More information
FTACLN The FTACLN utility can be used to clean up the file type associations in the Windows Registry on the device running the Program Neighborhood Agent.

After uninstalling the Program Neighborhood Agent Version 6.3x from a client device and restarting, some of the file types associated with the Program Neighborhood Agent can remain on the client device. This is evident, for example, when you double-click a .txt file in Explorer and the “Executable Cannot be Found” Windows message appears.

CTX101594

Can also be found on:

Presentation Server 4.x Server CD – Support – Debug

Back to top

CPU Utilization and Memory Optimization

What What it does More information
ADSCleaner 2.0 Citrix Memory Optimization works by creating Alternate Data Streams on each of the optimized files. Disabling the feature or adding a file to the exclusion list does not remove the Alternate Data Stream. The ADSCleaner.exe utility lists and optionally deletes alternative data streams created by Citrix memory optimization code.
[Document Not Found] Citrix Memory Optimization works by creating Alternate Data Streams on each of the optimized files. Disabling the feature or adding a file to the exclusion list does not remove the Alternate Data Stream. The ADSCleaner64.exe utility lists and optionally deletes alternative data streams created by Citrix memory optimization code on x64 platforms.

Back to top

Data Store Issues

– database corrupted, database migration issue, database connection issues

What What it does More information
DSCHECK – XenApp Data Store Checker Tool Commands DSCheck provides a way to perform validation checks on a server farm’s data store. Results are displayed on the console and written to the event log. The /clean option deletes inconsistent records. Citrix recommends that you back up the data store before using the /clean option. Further information on using this tool can be found in Citrix Knowledge Base article CTX124406
DSMaint

– Available with the Citrix Presentation Server/XenApp installation

DSMAINT is used to configure the IMA data store database for a server farm. When using this command, user names and passwords may be case-sensitive, depending on the database product being used and the operating system it runs on.
DSView

– Available on the Support folder of the Server CD

DSView can be used when you need to look at the value or find a specific record in the data store or the local host cache. DSView works by capturing the data from the Binary Large Object (BLOB) and placing it in an order so it can be interpreted. CTX106232
fc.exe

– Native to Windows 2000, Windows 2003 and Windows 2008

Can be used to compare ntuser.dat files fc /?
[Document Not Found] This script will terminate all IMA connections to an SQL 2000 database. CTX106684
[Document Not Found]

– Presentation Server/XenApp installation CD – Support – Debug

Use this utility to display information about member servers in the farm. Executing queryhr with no parameters lists all servers in the farm. Queryhr obtains information from the local host cache.

Back to top

Independent Management Architecture (IMA)

What What it does More information
[Document Not Found] This script will terminate all IMA connections to an SQL 2000 database. CTX106684
IMAPORT

– Installed with Presentation Server

Use imaport to query, set or reset the IMA – TCP/IP port. Important: If you change port number 2513 on the first server in the farm on which you install MetaFrame Presentation Server, you cannot join additional servers to the server farm. Administrators Guide – Appendix A

Back to top

Installation/Migration

– any issue related to installation or migration of Presentation Server, unable to install/uninstall

What What it does More information
MSI Installation Log

– Available with the Windows installation

The Microsoft Windows Installer Tool can generate a log of an MSI-based installation.

To enable this the MSI file must be installed using the MSEXEC command from the command prompt.

Microsoft Knowledge Base article 227091
Wilogutl.exe

– Available with Microsoft Platform SDK

Wilogutl.exe assists the analysis of log files from a Windows Installer installation, and it displays suggested solutions to errors that are found in a log file.

Back to top

Load Balancing

What What it does More information
[Document Not Found] Because all of the dynamic information is stored in tables in the data collector’s physical RAM, this command-line utility is provided to query the current information on the local zone data collector. You can use QueryDS to determine which servers are currently available in a farm. It retrieves all information from the tables stored on the local zone data collector. For example, the PN_Table contains information about all available servers that are accepting Program Neighborhood connections.

Back to top

Login/Authentication

– any authentication issue such as Single Sign On, smart card authentication, Novell authentication

What What it does More information
ACRCFG

– Available with the Citrix Presentation Server/XenApp installation

Use ACRCFG to configure or enable automatic client reconnection settings for a server or a server farm.

acrcfg [/server:servername | /farm][/query | /q]

acrcfg /?

Troubleshoot client disconnects or not getting automatically reconnected.
AUDITLOG

– Available with the Citrix Presentation Server/XenApp installation

Auditlog generates reports of logon/logoff activity for a server based on the Windows NT Server security event log. To use auditlog, you must first enable logon/logoff accounting. You can direct the auditlog output to a file. Presentation Server Administration Guide – Appendix A
Limit Login The application adds the ability to limit concurrent user logins and to keep track of all login information in an Active Directory domain. LimitLogin capabilities include limiting the number of logins per user from any machine in the domain (including Terminal Server sessions), displaying the login information of any user in the domain according to specific criteria.

While the main purpose of LimitLogin is to enforce concurrent login quotas, it can also be used purely as a login data capture solution that lets you manage your Active Directory environment more effectively.

LogonSessions v1.21 When you want to see who is authenticated and is running a session on a server. It lists the currently active logon sessions and the processes running in each session. It lists the session ID, name of the user, the authentication type, SID, Logon Server and Domain of the user.
Netdiag Netdiag.exe is a command-line tool that you can use to test the network connectivity of the computer. Netdiag.exe performs a series of tests to determine the state and functionality of your network client computer. You can use the results of these tests, and the network status information that is provided by Netdiag.exe, to isolate network and connectivity problems on your Windows based workstation or server. Can be used for troubleshooting multihomed servers or client latency.
Nltest.exe Nltest.exe is a very powerful command-line utility that can be used to test trust relationships and the state of domain controller replication in a Windows NT domain. Can be used for multi domain farms where you suspect trusts to be an issue, when publishing applications to users or using Advanced Access Control.
MedEvac 2.5 The MedEvac tool is utilized to run checks against a Presentation Server farm to verify environment health. CTX108311
QueryDC The QueryDC utility is used to determine the data collector for a given zone. Without any parameters, Querydc defaults to the host server’s zone and returns the zone name and name of the current zone data collector.
[Document Not Found] Because all of the dynamic information is stored in tables in the data collector’s physical RAM, this command-line utility is provided to query the current information on the local zone data collector. You can use QueryDS to determine which servers are currently available in a farm. It retrieves all information from the tables stored on the local zone data collector. For example, the PN_Table contains information about all available servers that are accepting Program Neighborhood connections.
User Profile Hive Cleanup Service UPH Clean fixes issues with “stuck” user profiles. It kills any processes that have hooks open to the user’s profile after the user initiates the logoff process.

Issues with this are commonly encountered with roaming user profiles.

Microsoft Knowledge Base article 837115
Userenv.log

– Available with the Windows installation

The Userenv.log file is used for capturing logging and debugging information regarding the user profile and the Windows system policy processes, including registry accesses, during the logon process. This can be helpful for troubleshooting issues that occur during the logon to a Citrix server.

Because of the large amount of data logged, it should be enabled on an isolated server when possible.

Microsoft Knowledge Base article 221833

Back to top

NetScaler

What What it does More information
NSCONMSG

– Located in /netscaler/ on all NetScaler systems

Executed from SHELL in a NetScaler SSH session. Examples of the command are: (Note the capital K as a command switch!!)

nsconmsg -K /var/nslog/newnslog -d consmsg gives you all messages sent to the console during the timeline covered by the newnslog

nsconmsg -K /var/nslog/newnslog -d ? lists other arguments which can be used to gather more information.
SHOW

– Part of the NetScaler CLI, all versions of the NetScaler operating system

Run this command to view current configuration information. It extracts a snapshot of information from the NetScaler Kernel. Use this command if you are not familiar with the configuration or environment in which the NetScaler sits.

Back to top

Performance

– ICA session slow, server slow down, ICA session hangs, application hangs, server hangs

What What it does More information
[Document Not Found] The Automatic Farm Tuner utility is designed to tweak the advanced registry setting on the data collectors in a large farm and tune the data collectors to adapt to a large farm environment. With this utility in place, all manual performance tweaking on the data collectors can be automatically set. CTX108456
[Document Not Found] “Citrix Connection Test Tool” provides various connection methods for scalability test and other tests which require a number of sessions to be established. The tool enables testers to establish as many numbers of ICA sessions as they want by creating or duplicating session settings on Citrix Presentation Server Client. Useful for loading the server for trying to identify performance issues.
[Document Not Found] The Citrix Server Test Kit (CSTK) version 2.1 is an automated tool that can be used by administrators to configure and run various user load combinations. Can be used for loading servers for performance testing.

CTX091649

Netdiag Netdiag.exe is a command-line tool that you can use to test the network connectivity of the computer. Netdiag.exe performs a series of tests to determine the state and functionality of your network client computer. You can use the results of these tests, and the network status information that is provided by Netdiag.exe, to isolate network and connectivity problems on your Windows based workstation or server. Can be used for troubleshooting multihomed servers or client latency.
TWCONFIG

– Installed with Presentation Server/XenApp

Use twconfig to configure ICA display settings that affect graphics performance for clients.
Userdump You can use the Userdump.exe tool to generate a user dump of a process that shuts down with an exception or that stops responding (hangs). Before dumping a hanging process, it is helpful to take a screenshot of the Qslice.exe window. This output will assist in dump analysis. Microsoft Knowledge Base article 241215
WinDBG Use WinDbg to open crash dumps obtained from Dr. Watson, manually generated userdumps, or Windows crashes. Once the crash dump has finished loading, many commands can be used to provide helpful information about the error.

The command !analyze –v performs basic exception analysis, reporting the error that occurred and the module in which it occurred.

The command lmv can be used to show all of the modules loaded, including their full path and version information.

Search support.citrix.com for “WinDBG” to get a list of How To guides.
WinDBG 64-bit This is the Windows debugger for 64-bit platforms.

Back to top

Ports and Printing

– any cases related to port mapping and printing, printer autocreation, printer drivers, replications, and spooler issues/crashes

What What it does More information
StressPrinters 1.3.2 for 32-bit and 64-bit Platforms Use to test if a driver is suitable for multi-user environments. It should be configured with multiple instances simultaneously, simulating multiple sessions autocreating printers using the same print driver. This allows an admin to ensure a driver does not cause a fatal error on their server and autocreates successfully.
Print Detective Enumerates print drivers on local and remote machines. Displays driver details, provider, supporting files, and so on. Great for determining non-native print drivers and quickly removing them.
PRINTUI Built-in utility to perform many printing related tasks including pushing print drivers, adding and deleting printers, and more. PRINTUI Reference
CHANGE CLIENT

– Available with the Citrix Presentation Server/XenApp installation.

Change client changes the current disk drive, COM port, and LPT port mapping settings for a client device.

“Change Client” or “CHGCDM” command

Can be used to query or refresh client setting or printer settings. May correct settings for a timing issue at connect time or clients reconnecting from a new device.
Driver Deleter A utility for cleaning the registry after removing printer drivers.

The Driver Deleter utility is used to remove old driver files and registry entries before installing new drivers. After running this utility you must reboot your server for the effects to take place.

[Document Not Found] The ListPrinterDrivers tool allows you to list all printer drivers in a farm.
Portmon for Windows v3.03 Displays serial and parallel port activity on a system with the capability to filter the output for better overview. Useful to troubleshoot port problems, for example, using a device on the client’s serial port from a published application. Run the tool in the session and on the client and see which actual commands are being sent to the port.

Back to top

Resource Manager

What What it does More information
TSSHUTDN Enables an administrator to remotely shut down or reboot a terminal server. You can also choose to power off the server if the computer supports software control of AC power.

Back to top

Seamless

– any issue related to applications in Seamless only

What What it does More information
MessageHistory Simple GUI tool that traces window messages, like Spy++, but easier to use.

Consists of two executable files (32- and 64-bit). Simply start the exe, select via tick boxes which messages should be traced and click start.

Used to troubleshoot issues with GUI, Seamless, Clipboard, client, and so on.
WindowHistory 4.0 for 32-bit platforms The WindowHistory utility complements Spy++ and other tools. It constantly monitors windows in the system (session) and records all changes made to it. Information about windows can then be saved for further analysis.
[Document Not Found] This is the version of the WindowHistory tool that works on ARM processors.
WindowHistory64 4.0 for x64 Platforms This is the version of the WindowHistory tool that works on 64-bit platforms.
Winspector Winspector is a freeware that can be used instead of Spy++ with pretty much the same functionality.

It captures all messages (keyboard input, mouse movements/click, changes in the window size and location, and so on) sent to a particular application for identifying issues with ICA clients, compare results inside a session and outside a session, keyboard mapping issues, and so on.

Useful for troubleshooting some Seamless Windows issues and allows distinguishing of the Window Class Name, which is then used to set an exception flag.

CTX101644

Back to top

Shadowing

What What it does More information
SHADOW Enables you to remotely control an active session of another user.

Back to top

Virtual IP (VIP)

What What it does More information
QRYClientIP Command line tool to query a session’s client IP. Used to allow one login script to process specific commands for specific IP addresses or IP segments.

Back to top

Web Interface

What What it does More information
CTXXMLSS

– Available with the Citrix Presentation Server/XenApp installation

By default, the Citrix XML Service shares a port with IIS. The CTXXMLSS command can be used to change to a different port, or revert back to sharing with IIS CTX104063

Can be used to ensure that port conflicts are not seen on Presentation Servers with IIS. Try changing the XML port to a different port to see if problem persists.

Back to top

Clone XenAPP 6.0 VM

One of the many enhancements Citrix made in XenApp v6 is that cloning a server is now much easier that it was in previous versions. Here’s a step-by-step guide, with lots of screen caps:

  1. Install the updated XenApp Server Configuration Tool.
  2. Run the XenApp Server Role Manager (Start – All Programs – Citrix – XenApp Server Role Manager – XenApp Server Role Manager):
  3. Select “Edit Configuration:”
  4. Select “Prepare this server for imaging and provisioning:”
  5. On the next screen, check “Remove this current server instance from the farm,” as shown below, then click “Next.” As the pop-up tip indicates, this will save you from having to do it manually later. The server will automatically join the farm when you bring it back on-line.
  6. On the next screen, click “Apply:”
  7. The server runs through the items that are needed to prepare XenApp for cloning. Note the informational warning that the settings will be applied when you clone or reboot the server. This means that once your new server comes on-line, it will automatically join the farm that the original server was in (before you removed it in Step 5).
  8. Back at the XenApp Server Role Manager screen, you can choose to reboot the server (which you probably don’t want to do just yet), or simply close the window and proceed with any additional tasks you may need to perform before cloning, such as Sysprep.
  9. After you’ve finished any additional tasks, you can shut the server down, and clone it to your heart’s content. When your clones come back on-line, if they have a network connection on the correct IP subnet, they will automatically join the farm. However(“gotcha” alert), if you didn’t Sysprep them, they will all try to join the farm under the same machine name – the one your original server had. So if you didn’t change the name of the server, it’s best to disconnect it from the network, change the name and IP address, reconnect to the network, join it to the AD Domain, and then reboot it so it can join the XenApp farm using the correct name.

If you’re a Citrix “old-timer,” you’ve got to agree that it doesn’t get much easier that this!

WHY USE HP MOONSHOT FROM THE CUSTOMER’S VIEW POINT (Repost of Carl Webster article)

December 22, 2014

Written by

Carl Webster

Source: WHY USE HP MOONSHOT FROM THE CUSTOMER’S VIEW POINT

Between August and November 2014, I have worked on three HP Moonshot Proof of Concepts (two in the USA and one in the Middle East). While Moonshot is not the solution to every desktop problem in the world, it has some very strong advantages. I decided to reach out to a couple of the Citrix leads on these PoCs and have them tell you why they selected to use Moonshot. Before I give you their comments, a couple of items:

  • Don’t ask for the company names, you will not get them
  • Don’t ask for the individual’s names, you will not get them
  • Don’t ask for their contact information, you will not get it
  • Don’t ask me to put you in contact with them, not going to happen
  • Don’t ask me to put them in contact with you, not going to happen

Both of these individuals read my blog and I will send them the link to this article. If you have any questions for them or me, leave a comment and if they are allowed to, they will respond to the comments.

I only edited their responses to correct typos, line formatting and wrapping in their responses and had them correct a couple of sentences that were not clear to me.

The questions I asked them to respond to: “Why did you select HP Moonshot over traditional VDI?  Because doesn’t Moonshot cost way more per user than traditional VDI?”

Medical Related Field

Note: This customer will deliver around 2,000 desktops with Moonshot.

Costs comparison: Moonshot HDI vs VDI

My friend Carl ask me last week: “Why did you select HP Moonshot over traditional VDI?  Because doesn’t Moonshot cost way more per user than traditional VDI?”

During my lifetime of layovers in the world’s most “cozy” terminal (much love EWR), I teetered away from basically disagreeing with the question, but I’m feeling more accommodating since then. Comparing the two methods is a tough one.

On one hand we have user dedicated PCs, and on the other we have a populated blade chassis, shared via your virtualization of choice. Totally an apples and oranges situation. So the difference maybe be jarring, in that the Moonshot m700 carts do not require any hypervisor. Every m700 cart has 4 self-contained systems and supports booting directly from Citrix PVS 7.1.

For those that have done HDI on the past with other solutions, this one is much smaller, at around 5u’s….get ready for this, 180 PCs in 5u’s. Maybe I’m easy to impress, but that is kind of amazing. 180 PCs with a fully dedicated 8gb, four core APU, and onboard SSD. If this PC was sitting on your desktop all by its lonesome, that would be a pretty great business-class system.

You could get the same specs from traditional VDI, but you would need a system that supported almost 1.5tb of memory and 720 cores, and then would require a virtualization layer to share it all out.

What end users want is an experience like they are using a dedicated computer, and what a better solution than one which is exactly that?! So this is why I almost disagree with the question. The cost difference as little as it may be is now negligible because the experience is head and shoulders above any traditional VDI experience I have encountered.

It is all about experience and alignment, the end-user knows what they want from their experience. It is up to us “techies” to get them to a solution that is in alignment with the business case.

Retail Related Field

Note: This customer will deliver around 750 desktops with Moonshot.

My answer would be:

As a long time Citrix admin, I knew the advantages of centralized compute environments. For many years I was at a community bank and we used terminals connecting to Citrix published desktops on MetaFrame XP.  This in essence was the first type of VDI.  We were able to support nearly 400 users with an IT staff of 4 full time employees. There was only a user side Linux device, and all user compute was load balanced across the farm.  Roaming profiles and redirected folders stayed on shares in the data center.  This gave a measure of security for PII data, knowing it was not on remote hard drives that could be lost or stolen.  Also there is an economic benefit to this model as terminals usually cost less than PCs and have a far longer useful life than PCs.  Using terminals also gives a centralized management framework that allows for minimal daily maintenance for the user end points.

So the concepts of VDI have strong advantages for organizations concerned with user data security, user hardware life cycles, and IT management with a small staff.

I am now at a larger organization with multiple corporate sites and several hundred retail stores. I had been trying for a year or more to raise interest in traditional VDI at my current company. We have a very robust VMware environment and SAN.  We also use XenApp to provide multiple user apps across the country to our retail stores and other corporate sites.

Additionally, we have a large number of onsite consultants working on multiple projects. My suggestion was to use VDI to provide all the advantages above on a new project. The retail store managers needed a way to have more robust applications and other access that could not be accommodated on a POS sales register.  Also, each consultant was issued a company laptop. The motivation was to keep data assets safe as possible and under company control.

My suggestion was to use VDI and terminals for a new store user system and for consultants. Including the consultants could enforce the traditional controls but allow for BYOD to reduce hardware expense.

But there was a lot of resistance because of the general understanding that VDI could go very badly. There is another problem with IOPS when it comes to VDI. All IOPS coming out of virtual desktops are typically treated as “equal” by the hypervisor. This causes a lack of consistent user experience (as user workloads vary). Imagine a user running a zip file compression or running an on-demand virus scan on the same host as the CEO who needs his desktop to work on his board meeting presentation. I researched several hybrid and flash based storage systems aligned with VDI deployments. My conclusion was that the total VDI solution was viable now because of the new storage options.

But that was not the only barrier.  The organization is very committed to vendor standardization and not enabling a sprawl of siloes of independent solutions.  So the addition of new VDI-centric storage was not agreeable.  And without that enhancement, the usual VDI IOPs concern remained.

Another hurdle turned out to be the business side.  As they came to understand the shared nature of VDI resources, there was growing resistance.  No one wanted a system that was not completely “theirs”. Even after explaining the IT benefits and small probabilities of user bottlenecks, it was still not well thought of. So traditional VDI was not seen as a safe and reliable solution to match the company culture and expectations.

Then I discovered the HP Moonshot platform and the Converged System 100. Immediately I knew that it had great potential.  Hosted Desktop Infrastructure solves all the concerns I encountered.  It matched our existing hardware vendor. It provides substantial dedicated CPU, GPU, and RAM for every user. And because of the nature of Citrix Provisioning and its ability to cache in memory, the user IOPs to disk are greatly reduced.  Plus Citrix Provisioning frees the onboard 64GB SSD for other uses.  It could hold persistent data, or business apps.  We use it as the page file location.

The use of XenDesktop and Receiver also creates a user system that can be available anytime on multiple devices.

I will say there is one caveat.  We decided to segregate the CS100 server components on dedicated VMware hosts. We also used a new HP 3PAR system as the underlying storage for all of the design. This was mainly because it started as a POC.  But because of its success, and vendor match, the additional hosts and storage was something that was accepted.

Another motivation for making that “giant leap” to Moonshot was the vision behind it. Having that chassis in your Data Center does more than enable HDI. Other server cartridges are available and more will be available in the future. I think it’s the beginning of a new phase of hardware consolidation and server computing. Also, the power consumption is impressive.  It only requires 33 watts typical for a cartridge running 4 Windows systems with a Quad core AMD APU, 8GB RAM, and an SSD.

Another plus is each Windows node has 2 x 1GB NICs.  This may not be meaningful when you think of an end user station.  But having it there gives you more options. We use 1 NIC as a normal LAN link.  The second is used as a direct link to a dedicated iSCSI LUN on the 3PAR.  Having a permanent storage partition per system has enabled us to add business data that is unique to each store location.

I am a big fan of HP HDI and Moonshot in general.  I know our particular situation will not match a lot of businesses.  But people should sit down and think about the potential it offers in terms of consolidation, energy savings, flexibility of architectures, end user mobility and user computing resources.  I believe it is a game changer on several levels.


There you go.

If you have any questions or comments, leave them in the comments section.

Thanks

Webster

CITRIX XENDESTOP AND PVS: A WRITE CACHE PERFORMANCE STUDY

Thursday, July 10, 2014   , , , , , , , , , , , ,   Source: Exit | the | Fast | Lane

image

If you’re unfamiliar, PVS (Citrix Provisioning Server) is a vDisk deployment mechanism available for use within a XenDesktop or XenApp environment that uses streaming for image delivery. Shared read-only vDisks are streamed to virtual or physical targets in which users can access random pooled or static desktop sessions. Random desktops are reset to a pristine state between logoffs while users requiring static desktops have their changes persisted within a Personal vDisk pinned to their own desktop VM. Any changes that occur within the duration of a user session are captured in a write cache. This is where the performance demanding write IOs occur and where PVS offers a great deal of flexibility as to where those writes can occur. Write cache destination options are defined via PVS vDisk access modes which can dramatically change the performance characteristics of your VDI deployment. While PVS does add a degree of complexity to the overall architecture, since its own infrastructure is required, it is worth considering since it can reduce the amount of physical computing horsepower required for your VDI desktop hosts. The following diagram illustrates the relationship of PVS to Machine Creation Services (MCS) in the larger architectural context of XenDesktop. Keep in mind also that PVS is frequently used to deploy XenApp servers as well.

image

PVS 7.1 supports the following write cache destination options (from Link):

  • Cache on device hard drive – Write cache can exist as a file in NTFS format, located on the target-device’s hard drive. This write cache option frees up the Provisioning Server since it does not have to process write requests and does not have the finite limitation of RAM.
  • Cache on device hard drive persisted (experimental phase only) – The same as Cache on device hard drive, except cache persists. At this time, this write cache method is an experimental feature only, and is only supported for NT6.1 or later (Windows 7 and Windows 2008 R2 and later).
  • Cache in device RAM – Write cache can exist as a temporary file in the target device’s RAM. This provides the fastest method of disk access since memory access is always faster than disk access.
  • Cache in device RAM with overflow on hard disk – When RAM is zero, the target device write cache is only written to the local disk. When RAM is not zero, the target device write cache is written to RAM first.
  • Cache on a server – Write cache can exist as a temporary file on a Provisioning Server. In this configuration, all writes are handled by the Provisioning Server, which can increase disk IO and network traffic.
  • Cache on server persistent – This cache option allows for the saving of changes between reboots. Using this option, after rebooting, a target device is able to retrieve changes made from previous sessions that differ from the read only vDisk image.

Many of these were available in previous versions of PVS, including cache to RAM, but what makes v7.1 more interesting is the ability to cache to RAM with the ability to overflow to HDD. This provides the best of both worlds: extreme RAM-based IO performance without the risk since you can now overflow to HDD if the RAM cache fills. Previously you had to be very careful to ensure your RAM cache didn’t fill completely as that could result in catastrophe. Granted, if the need to overflow does occur, affected user VMs will be at the mercy of your available HDD performance capabilities, but this is still better than the alternative (BSOD).

Results

Even when caching directly to HDD, PVS shows lower IOPS/ user numbers than MCS does on the same hardware. We decided to take things a step further by testing a number of different caching options. We ran tests on both Hyper-V and ESXi using our standard 3 user VM profiles against LoginVSI’s low, medium, high workloads. For reference, below are the standard user VM profiles we use in all Dell Wyse Datacenter enterprise solutions:

Profile Name Number of vCPUs per Virtual Desktop Nominal RAM (GB) per Virtual Desktop Use Case
Standard 1 2 Task Worker
Enhanced 2 3 Knowledge Worker
Professional 2 4 Power User

We tested three write caching options across all user and workload types: cache on device HDD, RAM + Overflow (256MB) and RAM + Overflow (512MB). Doubling the amount of RAM cache on more intensive workloads paid off big netting a near host IOPS reduction to 0. That’s almost 100% of user generated IO absorbed completely by RAM. We didn’t capture the IOPS generated in RAM here using PVS, but as the fastest medium available in the server and from previous work done with other in-RAM technologies, I can tell you that 1600MHz RAM is capable of tens of thousands of IOPS, per host. We also tested thin vs thick provisioning using our high end profile when caching to HDD just for grins. Ironically, thin provisioning outperformed thick for ESXi, the opposite proved true for Hyper-V. Toachieve these impressive IOPS number on ESXi it is important to enable intermediate buffering (see links at the bottom). I’ve highlighted the more impressive RAM + overflow results in red below. Note: IOPS per user below indicates IOPS generation as observed at the disk layer of the compute host. This does not mean these sessions generated close to no IOPS.

Hyper-visor PVS Cache Type Workload Density Avg CPU % Avg Mem Usage GB Avg IOPS/User Avg Net KBps/User
ESXi Device HDD only Standard 170 95% 1.2 5 109
ESXi 256MB RAM + Overflow Standard 170 76% 1.5 0.4 113
ESXi 512MB RAM + Overflow Standard 170 77% 1.5 0.3 124
ESXi Device HDD only Enhanced 110 86% 2.1 8 275
ESXi 256MB RAM + Overflow Enhanced 110 72% 2.2 1.2 284
ESXi 512MB RAM + Overflow Enhanced 110 73% 2.2 0.2 286
ESXi HDD only, thin provisioned Professional 90 75% 2.5 9.1 250
ESXi HDD only thick provisioned Professional 90 79% 2.6 11.7 272
ESXi 256MB RAM + Overflow Professional 90 61% 2.6 1.9 255
ESXi 512MB RAM + Overflow Professional 90 64% 2.7 0.3 272

For Hyper-V we observed a similar story and did not enabled intermediate buffering at the recommendation of Citrix. This is important! Citrix strongly recommends to not use intermediate buffering on Hyper-V as it degrades performance. Most other numbers are well inline with the ESXi results, save for the cache to HDD numbers being slightly higher.

Hyper-visor PVS Cache Type Workload Density Avg CPU % Avg Mem Usage GB Avg IOPS/User Avg Net KBps/User
Hyper-V Device HDD only Standard 170 92% 1.3 5.2 121
Hyper-V 256MB RAM + Overflow Standard 170 78% 1.5 0.3 104
Hyper-V 512MB RAM + Overflow Standard 170 78% 1.5 0.2 110
Hyper-V Device HDD only Enhanced 110 85% 1.7 9.3 323
Hyper-V 256MB RAM + Overflow Enhanced 110 80% 2 0.8 275
Hyper-V 512MB RAM + Overflow Enhanced 110 81% 2.1 0.4 273
Hyper-V HDD only, thin provisioned Professional 90 80% 2.2 12.3 306
Hyper-V HDD only thick provisioned Professional 90 80% 2.2 10.5 308
Hyper-V 256MB RAM + Overflow Professional 90 80% 2.5 2.0 294
Hyper-V 512MB RAM + Overflow Professional 90 79% 2.7 1.4 294

Implications

So what does it all mean? If you’re already a PVS customer this is a no brainer, upgrade to v7.1 and turn on “cache in device RAM with overflow to hard disk” now. Your storage subsystems will thank you. The benefits are clear in both ESXi and Hyper-V alike. If you’re deploying XenDesktop soon and debating MCS vs PVS, this is a very strong mark in the “pro” column for PVS. The fact of life in VDI is that we always run out of CPU first, but that doesn’t mean we get to ignore or undersize for IO performance as that’s important too. Enabling RAM to absorb the vast majority of user write cache IO allows us to stretch our HDD subsystems even further, since their burdens are diminished. Cut your local disk costs by 2/3 or stretch those shared arrays 2 or 3x. PVS cache in RAM + overflow allows you to design your storage around capacity requirements with less need to overprovision spindles just to meet IO demands (resulting in wasted capacity).

References:

DWD Enterprise Reference Architecture

http://support.citrix.com/proddocs/topic/provisioning-7/pvs-technology-overview-write-cache-intro.html

When to Enable Intermediate Buffering for Local Hard Drive Cache

XENAPP 7.X ARCHITECTURE AND SIZING

XenApp 7.x Architecture and Sizing

Source: http://weestro.blogspot.com/2014/05/xenapp-7x-architecture-and-sizing.html

image

Peter Fine here from Dell CCC Solution Engineering, where we just finished an extensive refresh for our XenApp recommendation within the Dell Wyse Datacenter for Citrix solution architecture.  Although not called “XenApp” in XenDesktop versions 7 and 7.1 of the product, the name has returned officially for version 7.5. XenApp is still architecturally linked to XenDesktop from a management infrastructure perspective but can also be deployed as a standalone architecture from a compute perspective. The best part of all now is flexibility. Start with XenApp or start with XenDesktop then seamlessly integrate the other at a later time with no disruption to your environment. All XenApp really is now, is a Windows Server OS running the Citrix Virtual Delivery Agent (VDA). That’s it! XenDesktop on the other hand = a Windows desktop OS running the VDA.

Architecture

The logical architecture depicted below displays the relationship with the two use cases outlined in red. All of the infrastructure that controls the brokering, licensing, etc is the same between them. This simplification of architecture comes as a result of XenApp shifting from the legacy Independent Management Architecture (IMA) to XenDesktop’s Flexcast Management Architecture (FMA). It just makes sense and we are very happy to see Citrix make this move. You can read more about the individual service components of XenDesktop/ XenApp here.

image

Expanding the architectural view to include the physical and communication elements, XenApp fits quite nicely with XenDesktop and compliments any VDI deployment. For simplicity, I recommend using compute hosts dedicated to XenApp and XenDesktop, respectively, for simpler scaling and sizing. Below you can see the physical management and compute hosts on the far left side with each of their respective components considered within. Management will remain the same regardless of what type of compute host you ultimately deploy but there are several different deployment options. Tier 1 and tier 2 storage are comprehended the same way when XenApp is in play, which can make use of local or shared disk depending on your requirements. XenApp also integrates nicely with PVS which can be used for deployment and easy scale out scenarios.  I have another post queued up for PVS sizing in XenDesktop.

image

From a stack view perspective, XenApp fits seamlessly into an existing XenDesktop architecture or can be deployed into a dedicated stack. Below is a view of a Dell Wyse Datacenter stack tailored for XenApp running on either vSphere or Hyper-v using local disks for Tier 1. XenDesktop slips easily into the compute layer here with our optimized host configuration. Be mindful of the upper scale utilizing a single management stack as 10K users and above is generally considered very large for a single farm. The important point to note is that the network, mgmt and storage layers are completely interchangeable between XenDesktop and XenApp. Only the host config in the compute layer changes slightly for XenApp enabled hosts based on our optimized configuration.

image

Use Cases

There are a number of use cases for XenApp which ultimately relies on Windows Server’s RDSH role (terminal services). The age-old and most obvious use case is for hosted shared sessions, i.e. many users logging into and sharing the same Windows Server instance via RDP. This is useful for managing access to legacy apps, providing a remote access/ VPN alternative, or controlling access to an environment through which can only be accessed via the XenApp servers. The next step up naturally extends to application virtualization where instead of multiple users being presented with and working from a full desktop, they simply launch the applications they need to use from another device. These virtualized apps, of course, consume a full shared session on the backend even though the user only interacts with a single application. Either scenario can now be deployed easily via Delivery Groups in Citrix Studio.

image

XenApp also compliments full XenDesktop VDI through the use of application off-load. It is entirely viable to load every application a user might need within their desktop VM, but this comes at a performance and management cost. Every VDI user on a given compute host will have a percentage of allocated resources consumed by running these applications which all have to be kept up to date and patched unless part of the base image. Leveraging XenApp with XenDesktop provides the ability to off-load applications and their loads from the VDI sessions to the XenApp hosts. Let XenApp absorb those burdens for the applications that make sense. Now instead of running MS Office in every VM, run it from XenApp and publish it to your VDI users. Patch it in one place, shrink your gold images for XenDesktop and free up resources for other more intensive non-XenApp friendly apps you really need to run locally. Best of all, your users won’t be able to tell the difference!

image

Optimization

We performed a number of tests to identify the optimal configuration for XenApp. There are a number of ways to go here: physical, virtual, or PVS streamed to physical/ virtual using a variety of caching options. There are also a number of ways in which XenApp can be optimized. Citrix wrote a very good blog article covering many of these optimization options, of which most we confirmed. The one outlier turned out to be NUMA where we really didn’t see much difference with it turned on or off. We ran through the following test scenarios using the core DWD architecture with LoginVSI light and medium workloads for both vSphere and Hyper-V:

  • Virtual XenApp server optimization on both vSphere and Hyper-V to discover the right mix of vCPUs, oversubscription, RAM and total number of VMs per host
  • Physical Windows 2012 R2 host running XenApp
  • The performance impact and benefit of NUMA enabled to keep the RAM accessed by a CPU local to its adjacent DIMM bank.
  • The performance impact of various provisioning mechanisms for VMs: MCS, PVS write cache to disk, PVS write cache to RAM
  • The performance impact of an increased user idle time to simulate a less than 80+% concurrency of user activity on any given host.

To identify the best XenApp VM config we tried a number of configurations including a mix of 1.5x CPU core oversubscription, fewer very beefy VMs and many less beefy VMs. Important to note here that we based on this on the 10-core Ivy Bridge part E5-2690v2 that features hyperthreading and Turbo boost. These things matter! The highest density and best user experience came with 6 x VMs each outfitted with 5 x vCPUs and 16GB RAM.  Of the delivery methods we tried (outlined in the table below), Hyper-V netted the best results regardless of provisioning methodology. We did not get a better density between PVS caching methods but PVS cache in RAM completely removed any IOPS generated against the local disk. I’ll got more into PVS caching methods and results in another post.

Interestingly, of all the scenarios we tested, the native Server 2012 R2 + XenApp combination performed the poorest. PVS streamed to a physical host is another matter entirely, but unfortunately we did not test that scenario. We also saw no benefit from enabling NUMA. There was a time when a CPU accessing an adjacent CPU’s remote memory banks across the interconnect paths hampered performance, but given the current architecture in Ivy Bridge and its fat QPIs, this doesn’t appear to be a problem any longer.

The “Dell Light” workload below was adjusted to account for less than 80% user concurrency where we typically plan for in traditional VDI. Citrix observed that XenApp users in the real world tend to not work all at the same time. Less users working concurrently means freed resources and opportunity to run more total users on a given compute host.

The net of this study shows that the hypervisor and XenApp VM configuration matter more than the delivery mechanism. MCS and PVS ultimately netted the same performance results but PVS can be used to solve other specific problems if you have them (IOPS).

image

* CPU % for ESX Hosts was adjusted to account for the fact that Intel E5-2600v2 series processors with the Turbo Boost feature enabled will exceed the ESXi host CPU metrics of 100% utilization. With E5-2690v2 CPUs the rated 100% in ESXi is 60000 MHz of usage, while actual usage with Turbo has been seen to reach 67000 MHz in some cases. The Adjusted CPU % Usage is based on 100% = 66000 MHz usage and is used in all charts for ESXi to account for Turbo Boost. Windows Hyper-V metrics by comparison do not report usage in MHz, so only the reported CPU % usage is used in those cases.

** The “Dell Light” workload is a modified VSI workload to represent a significantly lighter type of user. In this case the workload was modified to produce about 50% idle time.

†Avg IOPS observed on disk is 0 because it is offloaded to RAM.

Summary of configuration recommendations:

  • Enable Hyper-Threading and Turbo for oversubscribed performance gains.
  • NUMA did not show to have a tremendous impact enabled or disabled.
  • 1.5x CPU oversubscription per host produced excellent results. 20 physical cores x 1.5 oversubscription netting 30 logical vCPUs assigned to VMs.
  • Virtual XenApp servers outperform dedicated physical hosts with no hypervisor so we recommend virtualized XenApp instances.
  • Using 10-Core Ivy Bridge CPUs, we recommend running 6 x XenApp VMs per host, each VM assigned 5 x vCPUs and 16GB RAM.
  • PVS cache in RAM (with HD overflow) will reduce the user IO generated to disk almost nothing but may require greater RAM densities on the compute hosts. 256GB is a safe high water mark using PVS cache in RAM based on a 21GB cache per XenApp VM.

Resources:

Dell Wyse Datacenter for Citrix – Reference Architecture

XenApp/ XenDesktop Core Concepts

Citrix Blogs – XenApp Scalability

  1. Do you have anything on XenApp 7.5 + HDX 3D? This is super helpful, but there is even less information on sizing for XenApp when GPUs are involved.

    Reply

  2. Unfortunately we don’t yet have any concrete sizing data for XenApp with graphics but this is Tee’d up for us to tackle next. I’ll add some of the architectural considerations which will hopefully help.

    Reply

  3. Two questions:
    1. Did you include antivirus in your XenApp scalability considerations? If not, physical box overhead with Win 2012 R2 and 1 AV instance is minimal, when compared to 6 PVS streamed VMs outfitted with 6 AV instances respectively (I am not recommending to go physical though).
    2. When suggesting PVS cache in RAM to improve scalability of XenApp workloads, do you consider CPU, not the IO, to be the main culprit? After all, you only have 20 cores in a 2 socket box, while there are numerous options to fix storage IO.

    PS. Some of your pictures are not visible

    Reply

  4. Hi Alex,

    1) Yes, we always use antivirus in all testing that we do at Dell. Real world simulation is paramount. Antivirus used here is still our standard McAfee product, not VDI-optimized.

    2) Yes, CPU is almost always the limiting factor and exhausts first, ultimately dictating the limits of compute scale. You can see here that PVS cache in RAM didn’t change the scale equation, even though it did use slightly less CPU, but it all but eliminates the disk IO problem. We didn’t go too deep on the higher IO use cases with cache in RAM but this can obviously be considered a poor man’s Atlantis ILIO.

    Thanks for stopping by!

    Reply

CITRIX XENSERVER – SETTING MORE THAN ONE VCPU PER VM TO IMPROVE APPLICATION PERFORMANCE AND SERVER CONSOLIDATION E.G. FOR CAD/3-D GRAPHICAL APPLICATIONS

source: Citrix Blog

Many rich graphical applications such as CAD applications may benefit from allocating more than 1 vCPU per VM (1 vCPU is the default) via XenCenter or the command line. This will be of interest to many of those evaluating XenDesktop and XenApp HDX 3D Pro GPU pass-through and GPU sharing technologiesand can lead to noticeable performance enhancements. The performance gains possible though are highly application specific and users should evaluate and benchmark for their particular application and user workflows.

Overprovisioning

Virtualisation allows users to define more virtual vCPUs than there are physical CPUs (pCPUs) available.  A key driver for server consolidation is to virtualise workloads which spend much of their time idle. When virtualising, the expectation is often that when one of the VMs requires increased processing resources, several other of the VMs on the same host will be idle. This type of statistical multiplexing enables the total number of vCPUs across all VMs on a host to be potentially much greater than the number of physical cores.

In the context of virtualising graphical workloads (such as those that benefit from GPU-accelerations) this means you really should look at the demographics and usage patterns of your user base – I blogged about this a few months ago. If you are streaming video or similar constantly to all users there may be little scope to overprovision but if your users use applications in ways that use the CPUs in bursts there is often significant scope to consolidate resources.

Reservation of Capacity for Dom0

Before proceeding, it is worth noting that the XenServer scheduler ensures that dom0 cannot be starved of processing resources, given that it is crucial for normal operation of the platform. The scheduler ensures that dom0′s CPUs are allocated at least as much real CPU cycles as they would receive if they had a dedicated physical core. In other words, today, we ensure that dom0 uses the entirety of 4 physical cores, if it needs them. The maximum amount of CPU resource that dom0 can use is the equivalent of half the number of physical cores on the system.

This means that on a heavily loaded host, capacity for VMs will be the number of physical cores, minus 4.

vCPU Consolidation Ratio

As a rule of thumb, the ideal consolidation ratio will mean that ~80% of physical CPU utilisation is achieved under normal operation, to leave a 20% margin for bursts of activity. Therefore, the consolidation ratio that can be achieved is highly variable. It is conceivable that a 10:1 consolidation ratio is not _theoretically_ unreasonable.

Having said this, very high consolidation ratios can result in unexpected performance impacts. For example, when attempting to use 150 vCPUs on a 54 physical core box, 4 cores will go to dom0. This then means that we have a consolidation ratio of 3:1 (quite low). However, if all the CPUs are reasonably heavily loaded, this means that each vCPU will receive a 30 millisecond time slice, then have to wait for 90 milliseconds before its next time slice. The higher the consolidation ratio, the longer the interval between the time slices.

This waiting can impact aspects such as TCP network throughput, because packets appear to take far longer than expected to arrive at the VM. In the worst case, with very high consolidation ratios, if a VM receives a time slice extremely infrequently (once every few seconds) it will appear to freeze.

There is no absolute limit/number for the vCPU consolidation ratio. Clearly the greater the ratio, the greater the expectation is that the VMs are going to spend a greater fraction of their lives idle. The way to understand whether a consolidation ratio is appropriate on a running system is to examine the overall physical utilisation of the host.

cores-per-socket

Windows client operating systems support a maximum of two CPU sockets e.g. Windows 7 can only use 2 sockets. XenServer’s default behaviour is to present each vCPU with 1 core per socket; i.e. 4 vCPUs would appear as a 4 socket system.  Within XenServer you can set the number of cores to present in a socket. This setting is applied per guest VM.  With this setting you can tell XenServer to present the vCPUs as a 4 core socket.

[root@xenserver ~]# xe vm-param-set uuid=<vm-uuid> platform:cores-per-socket=4

VCPUs-max=4 VCPUs-at-startup=4

Further background can be found in http://support.citrix.com/article/CTX126524

 

Licensing

Users will need to assert whether any additional constraints are imposed by the licensing conditions of their graphical software and operating systems.

Maximum number of vCPUs per VM

The underlying Xen Hypervisor ultimately limits the number of vCPUs per guest VM, see here. Currently the Citrix XenServer limit is lower and defined by what is auto and regression tested, this limit is currently 16 vCPUs, you should reference the XenServer Configuration Guide for the version of XenServer you are interested in, e.g. for XS6.2, see here. It is therefore likely that higher numbers of vCPUs will work which maybe of interested to unsupported users, however for those with support we advise you stick to the supported limits. If there was sufficient demand raising the supported limit is something we would evaluate.

The XenCenter GUI currently allows users to set up to 8 vCPUs, for higher numbers users must use the command line CLI.

 

Anecdotal information

I’ve collected some of the feedback I’ve had from those involved with vGPU deployments and although it cannot be considered official Citrix endorsed best practice, I felt it was useful to share:

  • Most of the major CAE analysis apps will probably benefit from >2 cores although in some cases it depends on licensing. So Ansys, Abaqus, SolidWorks analysis module etc. all usually take as many cores as they find. Some applications will limit you to 2 cores unless you license an HPC pack or similar though.
  • Similarly ray tracing apps even when they are GPU accelerated will often use multiple cores (in addition to the CPU) although in current versions they may not take all cores since they want to leave some for interactivity.
  • We’ve had some feedback from customers doing analysis they expect applications use all cores so it runs faster, and they have noticed in a virtualized environments when CPU cores get over subscribed (please be aware of the need to understand overprovisioning above).
  • Our general rule is that to do good 3D interactive graphics you need a minimum of 4 vCPU cores, and for workstation cases 8 vCPU cores not because 8 will be used during graphics but because most high-end workstations are 8 cores and users and apps expect it.
  • Likewise during graphics all 4 vCPU cores may not be 100% busy but it’s likely that 2 probably are between the various needs of application code, graphics driver executions, operating system, virus checkers etc. So if you only have 2 vCPU cores we frequently observe a noticeable impact on performance.

How to investigate vCPU usage and server consolidation

One useful tool for investigating the performance of a system is xentop,details of xentop and other useful tools are available here. This allows you to understand the overall load on the system, and hence whether you are below the 80% load recommended above.

Over the last few versions of XenServer we’ve been increasing the number of metrics available and also improving the documentation and alerting mechanisms around these metrics. These new metrics complement those comprehensively documented in Chapter 9 of the XenServer 6.2 Administrators Guide.If you are interested in measuring the vcpu overprovisioning from the point of view of the host, you can use the host’s cpu_avg metric to see if it’s too close to 1.0 (rather than 0.8, i.e. 80%): If you are interested in measuring the vcpu overprovisioning from the point of view of a specific VM, you can use the VM’s runstate_*  metrics, especially the ones measuring runnable, which should be less than 0.01 or so. These metrics can be investigated via the command line or XenCenter.

 

Investigating cores per socket and thread per core

The Xen hypervisor utility xl can be of some use for debug and diagnosis in XenServer environments, particularly the inquiry options such as info [-n, --numa], by which you can query hardware information such as cores_per_socket and threads_per_core and similar data that you might want to log or keep in benchmarking assessments. Further details and links to this utility are detailed on this page, alongside other useful tools for use in XenServer environments.

Fine tuning

This type of fine tuning is indeed fiddly, and we are open to user feedback on how this could be improved, including the documentation of this type of information. In fact our documentation team are specifically watching this post to see what comments it solicits. A large number of you have already written blogs and articles of your experiences of best practice in your own “real life” XenServer deployments and they are of interest to us and many customers, see:

Further Improvements for graphical applications – CPU pinning and GPU pinning

For many of these applications a pinning strategy that considers the NUMA architecture of the server can also lead to performance enhancements. It is highly application and server specific and you really do need to investigate this with your particular applications, user demographic profile and usage patterns. There is some information linked to at the bottom of this page, but really it’s one best addressed in a future blog….. in the meantime I’d be interested to hear user anecdotes if anyone has tried this though.

One Comment

  1. Tobias Kreidl

    Very nice article, Rachel! Looking forward to more.

    Indeed it should also be noted that for many thin clients, one of the VCPUs may be used pretty much entirely just for reformatting and dealing with the video output, so we — in fact — always assign two VCPUs for all our VMs at a minimum to provide extra CPU cycles where they are needed. With as many as 80 or more VMs running on a 32-VCPU server youd think this would be inefficient, but with the environment we have, the dom0 load on a server is typically just 20-40%. Even with 20 or so pretty active users, it rarely climbs above 60%. Note that we allocate eight dom0 instances, each with 4 GB of memory, to be able to accommodate heavy use or boot storms on the XenDesktop environment and stress tests have shown that four dom0 instances at times were just not quite enough. With 256 GB of memory, the extra 16 GB were worth allocating for that purpose.

    The use of a GPU/vGPU can cut VCPU load on a VM by 30 to 50%. We have not had the chance to see how well this scales with many users, but the impact is clearly not negligible. Current testing underway to see if allocating a portion of a GPU to a VM is worse or better with many users than, say, creating a XenApp VM running on a XenServer and leveraging GPU passthrough for the entire XenApp server and having the VMs leverage the GPU that way. And of course, you can do both! The recent articles on PVH are, of course, tantalizing.

    We have not tried CPU pinning and in an environment with this many VMs and a parade of users coming and going, it probably wouldnt make that much sense. The CPU multiplexing certainly doesnt seem to incur a big penalty in this environment, at least from qualitative observations.

    We have also experimented with various XenServer parameters, such as modifying the txqueuelen and some of the network and TCP parameters with apparently positive effects. Linux has such a plethora of tuning options, which of course should be modified with caution.

MyXenApp

A blog dedicated to Citrix technology

There's More to the Story: a blog about LIFE, chronic illness, and Mental Health

I’m the loud and relentless "patient" voice and advocate they warned you about. I happen to have type 1 diabetes, ADHD, anxiety, OCD, PCOS, endometriosis, thyroid issues, asthma, allergies, lactose intolerance (and more), but there’s more to story.

DeployWindows

Learn Troubleshoot and Manage Windows

Dirk & Brad's Windows Blog

Microsoft Platform How To's, Best Practices, and other Shenanigans from Highly-qualified Windows Dorks.

Ingmar Verheij

About Citrix, Remote Desktop, Performance, Workspace, Monitoring and more...

Virtual to the Core

Virtualization blog, the Italian way.

CloudPundit: Massive-Scale Computing

the business of Internet infrastructure, cloud computing, and data centers

UCSguru.com

Every Cloud Has a Tin Lining.

speakvirtual

See no physical, hear no physical, speak no physical - speakvirtual.com

Ask the Architect

My workspace journey

blog.scottlowe.org

The weblog of an IT pro specializing in virtualization, storage, and servers

akosijesyang

a place under control of his big head

this is... The Neighborhood

the Story within the Story

Yellow Bricks

by Duncan Epping

THE SAN GUY

Enterprise Storage Engineer

My Virtual Vision

My thoughts on application delivery