Sunday, May 9, 2010

Using Perfmon for Web application based issue

USING PERFMON TO MONITORE ASP.NET AND IIS SERVER.

Purpose of the document:-

Perfmon is the monitoring tool which we can use to monitor different subsystem of web application such as IIS, SQL Server, ASP.NET, Services...
This document details about gathering more knowledge on how to performance monitor Web application.
This will be further extended to SQL Server monitoring and using WMI queries extensively.

What is perfmon?
The performance monitor is a utility which is used to track a range of processes and give a real time graphical display of the results. This tool can be used to assist with the tracking of processes that are needed to be optimized, monitoring results of tuning and configuration scenarios, and the understanding of resource usage to identify bottlenecks.
Perfmon help us to identify the bottlenecks and allow us to take action.

It can be opened by navigating to the performance icon in the administrative tools folder in the control panel or by typing perfmon.msc in the run box through start menu.





Once the program opens we see two sections under the console root:-
1. System Monitor
2. Performance Logs and alerts

System Monitor.
Through system monitor one can view the current activity for different performance counter, through Graphs, Histogram and reports.

Adding a counter.

Right click anywhere on the graph and choose Add Counter as shown below.

The Add Counter box consists of the following options:
Computer: The source system of the object. You can choose to select the local computer or another computer on your network – type.
Object: This refers to the virtual part of the computer that you want to monitor. Memory, Processor or Network Interface, for example.
Counter: The aspect of performance of interest. This refers to what parts of the object you want to monitor - they differ depending on the object.
Instance: The specific object to be measured when multiple objects of the same type exist on a single system.

For example, if you go to the Process performance object, the instances list will display all the active processes on the specified computer.

The Add Counter box consists of the following dialog box.


System monitor properties
Right click anywhere on the graph and choose Properties. This brings up the System Monitor Properties window that will allow you to customize the appearance and settings. You can change the view to graph, report or histogram style, the monitoring time interval and the colour of the counter lines, amongst others.
  
Using the monitor for network related performance.
The performance monitor can be a great tool to help with investigating the performance of your network. You are able to monitor things such as the Network Interface, TCP, UDP packet flow, terminal services sessions, and ICMP, amongst others. You can then compare the collected data and keep it as a record or use it for problem analysis.
In my example I have chosen to use the Network Interface as the performance object.
The following counters were added:
Current Bandwidth – to display the amount of bandwidth the network interface has.
Packets/Sec – to display the amount of packets transferred per second.
Bytes Total/Sec – to display the total amount of bytes per second.


NOTE:-In the screen shot above the purple line represents the number of packets per second, the yellow line represents the total bytes per second and the light green line shows how much bandwidth is available.

Performance Logs and Alerts
With the use of logs you are able to capture data that you can analyze later. Logged counter data information can be exported to spreadsheets and databases for future review and reporting. Alerts allow you to set an action that will be performed when specified counters reach a given value. These actions include sending a network message, executing a batch file, recording an item in the application log of the event viewer, and to start logging performance data.

You can use Alerts to send out warnings when disk space is running low or when network or level of CPU utilization poses a risk.

Logs
There are two types of logging features:
Counter Logs: - it is used to record the measurements of specific counters

Trace Logs:-  it is used to record memory and resource events.


NOTE:-The above image displays the counter log window that allows you to specify which counters should be monitored. The schedule permits you to set the start and stop time of logging. One can go to the log files tab if he wants to customize the name, size and location of the log file.
The above displays the trace log window which allows you to change what events will be logged by the system provider. Click ‘Provider Status’ to bring up a window that will show what system trace log providers and available and their current status. If you wish to add non system providers then select that option and press Add. You can run the this process as a different user, type the username in the Run As box and press the Set Password box to enter the password of the user.

Alerts
Right click anywhere on the white screen and choose “New Alert setting” to bring up the properties window for a new alert. You can set it to monitor the resultant of different type of counters and if they exceed the set limit then an alert will be triggered. The schedule tab gives you the option to set the start and stop times of the scan.

 Using the performance monitor on your Windows Server will help you identify where the problem is coming from. If this tool is used with correct configuration and planning to suit your network environment then the administrator can benefit from being able to tackle problems in less time, therefore making the situation more efficient.
How to use perfmon remotely:-
Those with read access to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurePipeServers\Winreg will have ablility to run performance monitor remotely.

Using the System Monitor (PerfMon) with ASP.NET Applications (IIS 6.0)
Unlike traditional Active Server Pages (ASP), which exposed performance counters globally for the entire server computer, most of the ASP.NET performance counters are exposed per application. In the System Monitor tool (PerfMon), per-application counters are available under the ASP.NET Applications performance object. If there are multiple applications on the server, you need to specify a particular application instance when selecting a counter to monitor. However, there is a special __Total__ application instance in PerfMon that aggregates the counter values for all applications on a server. The value of __Total__ is usually a sum of the counter values.
ASP.NET also exposes global-only counters that are not bound to a particular application instance. These counters are located under the ASP.NET performance object.

Procedures.

To view the available ASP.NET counters on a computer running Windows Server 2003.
1. From the Start menu, point to Administrative Tools, and then click Performance.
2. Right-click the System Monitor details pane and click Add Counters.
3. To monitor any computer on which the monitoring console is run, click Use localcomputer counters. Or, to monitor a specific computer, regardless of where themonitoring console is run, click Select counters from computer and specify a computer name or IP address.
4. In Performance object, click an object to monitor.
5 .To monitor all counters, click all counters.Or, to monitor only selected counters, click Select counters from list and select the counters you want to monitor.
6. To monitor all instances of the selected counters, click All instances.Or, to monitor only selected instances, click Select instances from list and select the Instances you want to monitor.
7. Click Add, and then click OK.

For information about each of the ASP.NET system and ASP.NET

Overview of ASP.NET Performance Monitoring (IIS 6.0).

There are a number of tools available to help you test and monitor the performance of your Web application. Microsoft Visual Studio .NET provides Application Center Test 1.0 (ACT), which simulates a large group of users by opening multiple connections to a Web server and rapidly sending HTTP requests. ASP.NET includes a number of performance counters that you can use to track the execution of your application. You can also use the built-in ASP.NET tracing feature to track code execution for a page or an application.

ACT is designed to stress-test Web servers and to analyze performance and scalability problems with Web applications. For more information about ACT, either install Visual Studio .NET and open ACT Help, or see information located in the MSDN Library.
When you are doing steady-state running analysis, it is often best to ignore the first request and any one-time initialization costs for objects. For example, when requesting an ASP.NET page for the first time, there is always the cost of compiling an instance of then Page class. Once that has occurred, performance will improve for subsequent requests.
 This section includes the following information:

System and Application Performance Counters for ASP.NET (IIS 6.0).
ASP.NET supports two groups of performance counters: system and application. The former is exposed in PerfMon in the ASP.NET performance counter object; the latter is exposed in the ASP.NET Applications performance object. There is a significant difference between the State Server Sessions counters found in the ASP.NET performance object, which apply only to the server computer on which the state server is running, and the Sessions counters found in the ASP.NET Applications performance object, which apply only to user sessions that occur in process. The value associated with each performance counter is updated every 400 milliseconds. When monitoring the performance of your ASP.NET Web applications.

you should always track the performance counters listed in the following table.

Performance object                          Performance counter

ASP.NET                                            Application Restarts

ASP.NET                                            Requests Queued

ASP.NET                                            Worker Process Restarts

ASP.NET Applications                         Errors Total

ASP.NET Applications                         Requests/Sec

Processor                                           % CPU Utilization

NOTE:-The % CPU Utilization counter monitors CPU utilization on the Web server computer. Low CPU utilization or the inability to maximize CPU utilization regardless of client load can signify contention for resource or a lock in your Web application.

In addition, the performance counters listed in the following table can be valuable in determining problems with your Web applications' performance.

Performance object                               Performance counter

ASP.NET Applications                             Pipeline Instance Count

.NET CLR Exceptions                             # Of Exceptions Thrown

System                                                   Context Switches/sec

NOTE:-The # of Exceptions Thrown counter displays the number of exceptions thrown in an application, since these can have performance implications.
Some code paths, however, rely on exceptions for their proper functioning. For example, the HttpResponse.Redirect method always throws an uncatchable exception, ThreadAbortException.

Therefore, it can be more useful to track the number of exceptions thrown using the Errors Total counter to see if the exception generated an error on the application.
The Context Switches/sec counter measures the rate at which thread contexts are switched by all CPUs in the Web server computer.
A high number for this counter usually indicates either high contention in locks or many switches between user and kernel mode by the thread.
Further investigation with sampling profilers and other tools might be warranted.

The counters in the ASP.NET and ASP.NET Applications performance objects are detailed in the following lists.

ASP.NET System Performance Counters.
ASP.NET supports the following ASP.NET system performance counters. This aggregate information for all ASP.NET applications on a Web server computer.
Application Restarts.
The number of times that an application has been restarted during the Web server's lifetime. Application restarts are incremented with each Application_OnEnd event. An application restart can occur because of changes to the Web.config file, changes to assemblies stored in the application's \Bin directory, or too many changes in Web Forms pages. Unexpected increases in this counter can mean that unforeseen problems are causing your Web application to shut down. In such circumstances you should investigate as soon as possible.

Note:- This value is reset every time the Internet Information Services (IIS) host is restarted.

Application Running.
1. The number of applications running on the server computer.
2. Requests Disconnected
3. The number of requests that have been disconnected due to a communication failure.

Requests Queued.
The number of requests waiting for service from the queue. When this number starts to increment linearly with respect to client load, the Web server computer has reached the limit of concurrent requests that it can process.

Requests Rejected.
The total number of requests not executed because of insufficient server resources to process them. This counter represents the number of requests that return a 503 HTTP status code, indicating that the server is too busy.

Request Wait Time.
The number of milliseconds that the most recent request waited for processing in the queue.

Session State Server Connections.
The total number of session-state connections to the computer on which out-of-process session-state data is stored.


Session SQL Server Connections.
The total number of session-state connections to the Microsoft SQL Server-TM- database in which session-state data is stored.

State Server Sessions Abandoned.
The number of user sessions that have been explicitly abandoned. These are sessions that are ended by specific user actions, such as closing the browser or navigating to another site. This counter is available only on the computer where the state server service (aspnet_state) is running.

State Server Sessions Active.
The number of currently active user sessions. This counter is available only on the computer where the state server service (aspnet_state) is running.

State Server Sessions Timed Out.
The number of user sessions that have become inactive through user inaction. This counter is available only on the computer where the state server service (aspnet_state) is running.

State Server Sessions.
The number of sessions created during the lifetime of the process. This counter is the cumulative value of State Server Sessions Active, State Server Sessions Abandoned, and State Server Sessions Timed Out. This counter is available only on the computer where the state server service (aspnet_state) is running.

Worker Process Restarts.
The number of times a worker process has been restarted on the server computer. A worker process can be restarted if it fails unexpectedly or when it is intentionally recycled. In the case of unexpected increases in this counter you should investigate as soon as possible.

Worker Process Running.
The number of worker processes running on the server computer.

ASP.NET Application Performance Counters.
ASP.NET supports the following application performance counters, which you can use to monitor the performance of a single instance of an ASP.NET application. A unique instance appears for these counters, named __Total__, which aggregates counters for all applications on a Web server (similar to the global counters described in the first section of this topic). The __Total__ instance is always available. The counters will display zero when no applications are present on the server.

Anonymous Requests.
The number of requests that are using anonymous authentication.

Anonymous Requests/Sec.
The number of requests per second that are using anonymous authentication.

Cache Total Entries.
The total number of entries in the cache. This counter includes both internal use of the cache by the ASP.NET page framework and external use of the cache through exposed APIs.

Cache Total Hits.
The total number of hits from the cache. This counter includes both internal use of the cache by the ASP.NET page framework and external use of the cache through exposed APIs.

Cache Total Misses.
The number of failed cache requests per application. This counter includes both internal use of the cache by ASP.NET and external use of the cache through exposed APIs.

Cache Total Hit Ratio.
The ratio of hits to misses for the cache. This counter includes both internal use of the cache by ASP.NET and external use of the cache through exposed APIs.

Cache Total Turnover Rate.
The number of additions and removals to the total cache per second. It is useful in helping determine how effectively the cache is being used. If the turnover is large, the cache is not being used efficiently.

Cache API Entries.
The total number of entries in the application cache.

Cache API Hits.
The total number of hits from the cache when it is accessed through only the external cache APIs. This counter does not track any use of the cache internally by ASP.NET.

Cache API Misses.
The total number of failed requests to the cache when accessed through the external cache APIs. This counter does not track any use of the cache internally by ASP.NET.

Cache API Hit Ratio.
The cache hit-to-miss ratio when accessed through the external cache APIs. This counter does not track any use of the cache internally by ASP.NET.

Cache API Turnover Rate.
The number of additions and removals to the cache per second, when used through the external APIs (excluding internal use by the ASP.NET page framework). It is useful in helping determine how effectively the cache is being used. If the turnover is large, then the cache is not being used effectively.

Compilations Total.
The total number of compilations that have taken place during the lifetime of the current Web server process. This occurs when a file with an .aspx, .asmx, .ascx, or .ashx extension, or a code-behind source file, is dynamically compiled on the server.

Debugging Requests.
The number of requests that occur while debugging is enabled.

Errors During Preprocessing.
The number of errors that occurred during parsing. Excludes compilation and run-time errors.

Errors during Compilation.
The number of errors that occur during dynamic compilation. Excludes parser and run-time errors.

Errors during Execution.
The total number of errors that occur during the execution of an HTTP request. Excludes parser and compilation errors.

Errors Unhandled During Execution.
The total number of unhandled errors that occur during the execution of HTTP requests.

Errors Unhandled During Execution/Sec.
The number of unhandled exceptions per second that occur during the execution of HTTP requests.

Errors Total.
The total number of errors that occur during the execution of HTTP requests. Includes any parser, compilation, or run-time errors. This counter is the sum of the Errors During Compilation, Errors During Preprocessing, and Errors During Execution counters. A well-functioning Web server should not generate errors. If errors occur in your ASP.NET Web application, they may skew any throughput results because of very different code paths for error recovery. Investigate and fix any bugs in your application before performance testing.

Errors Total/Sec.
The number of errors per second that occur during the execution of HTTP requests. Includes any parser, compilation, or run-time errors.

Output Cache Entries.
The total number of entries in the output cache.

Output Cache Hits.
The total number of requests serviced from the output cache.

Output Cache.
Number of failed output-cache requests per application.

Output Cache Hit Ratio.
The percentage of total requests serviced from the output cache.

Output Cache Turnover Rate.
The number of additions and removals to the output cache per second. If the turnover is large, the cache is not being used effectively.

Pipeline Instance Count.
The number of active request pipeline instances for the specified ASP.NET application. Since only one execution thread can run within a pipeline instance, this number gives the maximum number of concurrent requests that are being processed for a given application. In most circumstances it is better for this number be low when under load, which signifies that the CPU is well utilized.

Request Bytes in Total.
The total size, in bytes, of all requests.

Request Bytes out Total.
The total size, in bytes, of responses sent to a client. This does not include standard HTTP response headers.

Requests Executing.
The number of requests currently executing.

Requests Failed.
The total number of failed requests. Any and all status codes greater than or equal to 400 will increment this counter.
The equivalent ASP counter would also increment on requests rejected, which cannot be done (because the rejection is done by IIS and not the process model).

Requests Not Found.
The number of requests that failed because resources were not found (status code 404, 414).

Requests Not Authorized.
The number of requests that failed due to no authorization (status code 401).

Requests Succeeded.
The number of requests that executed successfully (status code 200).

Requests Timed Out.
The number of requests that timed out (status code 500).

Requests Total.
The total number of requests since the service was started.

Requests/Sec.
The number of requests executed per second. This represents the current throughput of the application. Under constant load, this number should remain within a certain range, barring other server work (such as garbage collection, cache cleanup thread, external server tools, and so on).

Sessions Active.
The number of sessions currently active. This counter is supported only with in-memory session state.

Sessions Abandoned.
The number of sessions that have been explicitly abandoned. This counter is supported only with in-memory session state.

Sessions Timed Out.
The number of sessions that timed out. This counter is supported only with in-memory session state.

Sessions Total.
The number of sessions timed out. This counter is supported only with in-memory session state.

Transactions Aborted.
The number of transactions aborted.

Transactions Committed.
The number of transactions committed.

Transactions Pending.
The number of transactions in progress.

Transactions Total.
The total number of transactions since the service was started.

Transactions/Sec.
The number of transactions started per second.

System and Application Performance Counters for ASP.NET.
  • NET CLR Exceptions.
# of Exceptions Thrown/Sec.
The total number of managed exceptions thrown per second. As this number increases, performance degrades.

  • NET CLR Loading.
Bytes in loader Heap.
The number of bytes committed by the class loader across all application domains.

Current App Domains.
The current number of application domains loaded in the process. The value of this counter should be the same as the number of virtual directories, plus 1 (to account for the default application domain).

Current Assemblies.
The current number of assemblies loaded in the process. Note that assemblies cannot be unloaded from an application domain; the entire application domain must be unloaded. This is an issue for sites with a large number of pages.

  • NET CLR Memory
# Bytes in all heaps.
Displays the sum of the Gen 0 Heap Size, Gen 1 Heap Size, Gen 2 Heap Size, and the Large Object Heap Size counters. This counter indicates, in bytes, the current memory allocated on the garbage collection heaps.

# Gen 0 Collections.
Displays the number of times that generation 0 objects (that is, the youngest, most recently allocated objects) have been reclaimed by garbage collection since the application started.
Generation 0 garbage collection occurs when the available memory in generation 0 is not sufficient to satisfy an allocation request. This counter is incremented at the end of a generation 0 garbage collection. Higher-generation garbage collections include all lower-generation collections. This counter is explicitly incremented when a higher-generation (generation 1 or 2) garbage collection occurs.

This counter displays the last observed value. The _Global_ counter value is not accurate and should be ignored.

# Gen 1 Collections.
Displays the number of times that generation 1 objects have been reclaimed by garbage collection since the application started.
The counter is incremented at the end of a generation 1 garbage collection. Higher-generation garbage collections include all lower generation collections. This counter is explicitly incremented when a higher-generation (generation 2) garbage collection occurs.

This counter displays the last observed value. The _Global_ counter value is not accurate and should be ignored.

# Gen 2 Collections.
Displays the number of times that generation 2 objects have been reclaimed by garbage collection since the application started. The counter is incremented at the end of a generation 2 garbage collection (also called a full garbage collection).

This counter displays the last observed value. The _Global_ counter value is not accurate and should be ignored.

Memory.

• Available Mbytes.
The amount of physical memory available to processes running on the computer, in megabytes. It is calculated by adding the amount of space on the zeroed, free, and standby memory lists. Free memory is ready for use; zeroed memory is filled with zeros to prevent later processes from seeing data used by a previous process; standby memory is removed from a process's working set (its physical memory) to be stored on the server's hard drive but is still available to be recalled. This counter displays the last observed value only; it is not an average.

Process.

% Processor Time.
The percentage of the elapsed time on the processor that it took all threads associated with a single process to perform execution instructions. An instruction is the basic unit of execution in a computer; a thread is the object that executes instructions; and a process is the object created when a program is run. Code executed to handle some hardware interrupts and trap conditions are included in this count.

Handle Count.
The total number of handles currently open that are associated with this process. This number is equal to the sum of the handles currently open for each thread in this process.

Private Bytes.
The current amount of memory, in bytes, that this process has allocated that cannot be shared with other processes.

Thread Count.
The number of threads currently active in this process. An instruction is the basic unit of execution in a processor, and a thread is the object that executes instructions. Every running process has at least one thread.

Processor.
% Processor Time.
The percentage of sampled time that the processor spends to execute a non-Idle thread. It is calculated by measuring how long the idle thread is active in the sample interval and subtracting that time from the interval duration. (Each processor has an idle thread that consumes cycles when no other threads are ready to run). This counter is the primary indicator of processor activity, and displays the average percentage of busy time observed during the sample interval.

Web Service.
Bytes Received/Sec.
The number of bytes per second received by the Web service.

Bytes Sent/Sec.
The number of bytes per second sent by the Web service.

Current Connections.
The current number of connections established with the Web service.

ISAPI Extension Requests/Sec.
The number of ISAPI Extension requests per second that are being processed by the Web service.


Total Method Requests/Sec.
The number of HTTP requests per second received by the Web service.


..

2 comments:

Anonymous said...

Please comment

Anonymous said...

Can yo please elaborate on Anonymous Requests object in permon

This is win server 2003 standard and iis6

Is this a real time count or a total?
What does it represent? individual Request from say a browser to the web app? Is this somehow related to "Current Anonymous Users" in the "Web Service" perfmon object?

I am trying to understand why it keeps growing with time in a web application.

Recent Posts