LoadRunner

Here is a very common question posted on Stack Overflow. I am just bringing it together here-
(Thanks to original author and for all the responses)

Question-

We are load testing a web application. The application is deployed on two VM servers with a a hardware load balancer distributing the load. There are tow tools used here 1. HP Load Runner (an expensive tool). 2. JMeter - free

JMeter was used by development team to test for a huge number of users. It also does not have any licensing limit like Load Runner.

How the tests are run ? A URL is invoked with some parameters and web application reads the parameter , process results and generates a pdf file.

When running the test we found that for a load of 1000 users spread over period of 60 seconds, our application took 4 minutes to generate 1000 files. Now when we pass the same url through JMeter, 1000 users with a ramp up time of 60 seconds, application takes 1 minutes and 15 seconds to generate 1000 files.
I am baffled here as to why this huge difference in performance.
Load runner has rstat daemon installed on both servers.

Any clues ?

 Possible Answer-

1. You are measuring two different things. Check your timing record structure.
2. Your request and response information is different between the two tools. Check with Fiddler or Wireshark.
3. Your test environment initial conditions are different yielding different results. Test 101 stuff, but quite often overlooked in tracking down issues like this.
4. You have an overloaded load generator in your loadrunner environment which is causing all virtual users to slow. For example you may be logging everything resulting in your file system becoming a bottleneck for the test. Deliberately underload your generators, reduce your logging levels and watch how you are using memory for correlations so you don't create a physical memory oversubscribed condition which results in high swap activity.

It should be noted that LoadRunner tracks its internal API time to directly address accusations of the tool influencing the test results. If you open the results set database set (.mdb or Microsoft SQL server instance as appropriate) and take a look at the [event meter] table you will find a reference for "Wasted Time." The definition for wasted time can be found in the LoadRunner documentation.


 Few more points-

• Think / wait time: When recording, Jmeter does not automatically put in waits.
• Items being requested: Is Jmeter ONLY requesting/downloading HTML pages while Load runner gets all embedded files/Pics?
• Invalid Responses: are all 1000 Jmeter responses valid? If you have 1000 threads from a single desktop, I would suspect you killed Jmeter and not all your responses were valid