Detecting applications which cause Page faults

Over the weekend, I ran the Bellman Ford algorithm (All Pair Shortest Path). The algorithm has a running time of O(n²m). There were 20,000 nodes in the graph. And it was pretty dense or more inter-connected. The program ran for four hours with a CPU load of 100% on a quad-core processor.

I noticed a significant degradation in performance due to Page faults within the application. Page faults degrade performance, especially the hard fault. There are two types of Page faults: Hard fault and Soft fault.

Soft fault occurs within an application when the application requests for a large amount of memory. And the memory is not available as a continuous block. Either the process requests for more physical memory or does garbage collection to free up memory. Soft faults happen quite frequently and are not that expensive as hard faults.

Hard fault occurs when an application requests a resource from memory which is paged into the physical disk. Hard fault is more expensive. This is because Windows usually swaps stale physical memory into the Paging file (stored in the disk). Disk I/O is always expensive.

My computer has 6 GB of physical memory (RAM), which is fairly adequate. Normally, there is less Page faults. Page faults are measured using Performance counters. Page Faults / sec counter in Memory category is the sum of both soft and hard faults happening every second.  Page Reads / sec counter in Memory category measures the hard faults happening ever second. If Page Reads / sec is lower, performance is better.

For each application, there is another performance counter that measures the faults. It is within the Process category. And the counter is Page Faults / sec. Below is a simple utility to identify the top 5 applications causing the maximum Page faults.

static bool CheckPerm()
{
    var watch = new Stopwatch();
    watch.Start();
    Console.WriteLine("Getting perf counters");
    var category = new PerformanceCounterCategory("Process");
    var counters = new List<PerformanceCounterValue>();

    var readCounter = new PerformanceCounter("Memory", 
"Page Reads/sec", true);
    var readCounterValue = new PerformanceCounterValue(readCounter);

    foreach (var instanceName in category.GetInstanceNames())
    {
        var counter = new PerformanceCounter("Process", 
"Page Faults/sec", instanceName, true);
        var counterValue = new PerformanceCounterValue(counter);
        counters.Add(counterValue);
    }


    for (int i = 0; i < 600; i++)
    {
        double startTime = watch.Elapsed.TotalSeconds;

        float readValue = readCounter.NextValue();
        if (i >= 10)
        {
            readCounterValue.Value += readValue;
        }

        Console.WriteLine("Getting {0} value", i);
        foreach (var counter in counters)
        {
            try
            {
                if (!counter.Remove)
                {
                    float counterValue = counter.Counter.NextValue();
                    if (i >= 10)
                        counter.Value += counterValue;
                }
            }
            catch (Exception)
            {
                counter.Remove = true;
                Console.WriteLine("{0} counter is removed", 
counter.Counter.InstanceName);
            }
        }

        Thread.Sleep(1000);
    }

    var orderedCounters = counters.OrderByDescending(c => c.Value).Take(8).ToList();
    Console.WriteLine("{0} - {1}", readCounter.InstanceName, 
readCounterValue.Value / 590);
    foreach (var counter in orderedCounters)
    {
        Console.WriteLine("{0} - {1}", counter.Counter.InstanceName, 
counter.Value / 590);
    }

    watch.Stop();
    Console.WriteLine("Time taken: {0}", watch.Elapsed.TotalSeconds);

    Console.ReadKey();
    return true;
}

The above application gets the Page Reads /sec counter from the Memory category. It then iterates through each instance of the Process category. And gets the Page Faults /sec counter. It orders the list of counter values by decreasing value and displays it on the screen.

Here is a sample snapshot of the output from the utility:

The Page Reads /sec has a value of 3.446 (quite high). The total faults per second is 522. The Bellman Ford app has about 207 faults per second. It is followed by Chrome and McAfee with about 100 faults per second. To fine-tune the system, I rewrote my app to use lesser memory. This reduced the Page Reads /sec and my system returned back to normal.

Hope this helps to detect performance issues in your system.