tag:blogger.com,1999:blog-9238405.post111351844598726729..comments2024-03-18T02:04:50.380-07:00Comments on Agile Testing: More on performance vs. load testingGrig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.comBlogger30125tag:blogger.com,1999:blog-9238405.post-80448407557118185722011-07-18T08:22:22.878-07:002011-07-18T08:22:22.878-07:00Henry -- what your dev team is asking makes sense ...Henry -- what your dev team is asking makes sense to me. I usually have what is called a staging environment for that type of test. It's a smaller-scale replica of production.Grig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-87351875363693568162011-07-15T10:14:26.688-07:002011-07-15T10:14:26.688-07:00I was researching for application performance test...I was researching for application performance testing and came across this blog. very useful and informative. <br /><br />The question that I have is not so much about what performance, load testing is (yes, it is still good to understand what they are) but when to do it and what type of environment to do it in. <br /><br />I am struggling with the development team right now, they wanted to do "application performance testing" (they also refer to as the non-functional testing)as part of their iterative development cycle on code that are "stable" but might not be a release candidate (did not pass user acceptance testing). The reason is to eliminate any surprises and make sure the code is clean and "perform" as expected when they release for "performance and load testing". If they did not do it this way, it will be a bit too late or too close to production deployment to have enough time to fix things that the "perform and load testing" unearth.<br /><br />To perform the test, they do not want to do it in the development environments. They request for a separate, production like environment in order to get some meaningful results.<br /><br />So is what they said and wanted to do make sense?<br /><br />Is this "application performance testing"?Henryhttps://www.blogger.com/profile/08181805561392308068noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-55639484723473669722011-02-22T06:39:45.766-08:002011-02-22T06:39:45.766-08:00Hello,
I am trying to find the extrapolation tech...Hello,<br /><br />I am trying to find the extrapolation techniques that can be used after load testing on a scaled down environment.<br />I have compled a bench mark testing on a scaled down version of the LIVE environment(from hardware /infra perspective) . Now, I am trying to extrapolate the results to suggest the performance of the LIVE environment.<br />Any suggesstions?Pankajnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-87884256506625453442009-04-09T00:19:00.000-07:002009-04-09T00:19:00.000-07:00If something is interesting in drawing charts from...If something is interesting in <BR/>drawing charts from vmstat log,<BR/>you could try vmstax at<BR/>http://www.michenux.net/blog/?p=1Unknownhttps://www.blogger.com/profile/10158836696229552210noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-88882146156203759072008-11-15T01:30:00.000-08:002008-11-15T01:30:00.000-08:00Very Informative. Could you please let me know wha...Very Informative. Could you please let me know what does concurrent user actually means?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-17571765880107212352008-09-16T15:44:00.000-07:002008-09-16T15:44:00.000-07:00Anonymous -- when I said 1,000 connection per user...Anonymous -- when I said 1,000 connection per user, I was referring to the total number of connections that the simulated user will make to the server during the time of the test run. They are not concurrent connections necessarily. If you read the httperf blog post I wrote, it would refer to this parameter:<BR/><BR/>num-conns: specifies how many total HTTP connections will be made during the test run -- this is a cumulative number, so the higher the number of connections, the longer the test run<BR/><BR/>Hope this helps.<BR/><BR/>GrigGrig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-69223571594105719882008-09-16T14:31:00.000-07:002008-09-16T14:31:00.000-07:00I'm trying to decipher info between your several a...I'm trying to decipher info between your several articles on this subject. You said 1000 requests per concurrent user in this article, and in another article, you said 1 or 10 requests per concurrent user. 1000 requests per concurrent user sounds kinda high. Was that a typo?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-51913637662025944142008-03-27T00:40:00.000-07:002008-03-27T00:40:00.000-07:00Very Useful Information. Previously I taught Perfo...Very Useful Information. Previously I taught Performance = Load + Stress, then on reading different topics in your blog I came to know the difference.Thejesh Giri Chikmagalurhttps://www.blogger.com/profile/16142936437235268117noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-20397515065600622032007-12-21T02:40:00.000-08:002007-12-21T02:40:00.000-08:00You can use soapui for doing load testing on web s...You can use soapui for doing load testing on web services. It is a open source tool and one of the best tools i have seen for web services.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-49259053289582100812007-12-09T20:33:00.000-08:002007-12-09T20:33:00.000-08:00Question:I'm new to the subject of Load testing an...Question:<BR/><BR/>I'm new to the subject of Load testing and am planning on doing a grad project using some tool against some web site - since I can't get a LoadRunner license I was thinking of using SilkPerformer, but I've not chosen any tool yet - any suggestions? <BR/><BR/>But the other concern is just as great - do you know where I could find an e-commerce web site to test?<BR/><BR/>Thanks,<BR/><BR/>ChadUnknownhttps://www.blogger.com/profile/14181116257145912921noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-88910285790841310772007-10-29T10:03:00.000-07:002007-10-29T10:03:00.000-07:00P1 -- yes, you're right, sysadmins and developers ...P1 -- yes, you're right, sysadmins and developers should be the main parties responsible for the performance tuning aspect. The testers' role is to measure performance and point out issues that they uncover. However, in an 'agile' environment there is close collaboration between all roles. So ideally sysadmins, developers and testers would collaborate in closing the feedback loop as tightly as possible. So the testers give feedback, the sysadmins and the developers tune the systems and the app, then the testers give more feedback, etc.<BR/><BR/>GrigGrig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-74770837855050336122007-10-29T02:16:00.000-07:002007-10-29T02:16:00.000-07:00Hi Greg ,, thnx for the informative articles.I ha...Hi Greg ,, thnx for the informative articles.I have one queery here . Consider these 2 statements <BR/><BR/>"1]At this point, the developers need to take back the profiling measurements and tune the code and the database queries. The system administrators can also increase server performance simply by throwing more hardware at the servers -- especially more RAM at the Web/App server in my experience, the more so if it's Java-based.<BR/><BR/>2. Let's say the application/database code, as well as the hardware/OS environment have been tuned to the best of everybody's abilities. You re-run the load test from step 2 and now you're at 75 concurrent users before performance starts to degrade."<BR/><BR/> The things that were done in step one , is perf testing really req to find out that .... isn't the developer , admins etc suppose to optimize things right in the first place.Only input we gave them is that there is a problem at 50 user load. and after this input they ll do changes [follow industry best practices etc which they were suppose to do anyways] and then it works fine on 75 users .<BR/><BR/> Can you please elaborate these stepsP1https://www.blogger.com/profile/00549397021677722146noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-7859750673623524842007-09-26T11:07:00.000-07:002007-09-26T11:07:00.000-07:00Swathi -- as a general technique, I'd use profilin...Swathi -- as a general technique, I'd use profiling in your case. There are tons of java profilers that will time each of your methods and will show you the hotspots. Say you identify a database-bound method which is a hotspot. Then you can issue that particular SQL query directly against the database, time it, and try optimizing it. Rinse and repeat :-)<BR/><BR/>GrigGrig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-71492263261158822582007-09-25T14:41:00.000-07:002007-09-25T14:41:00.000-07:00Thanks for the ingo Grig!..Sorry, for not mentioni...Thanks for the ingo Grig!..Sorry, for not mentioning the type of application being put under test...its a java standalone application and I am in search of some useful tools(java oriented) that give us the statistical info. on the usage of CPU and Memory.<BR/><BR/>I found JConsole pretty useful wrt. memory, but not many tools wrt. CPU or Database. Say,I wanted to know whether/not a DB (be it Oracle, MySql or Derby) is overwhelmed and I wanted to test how fast the DB can receive inputs; to identify if a DB is blocking any messages and etc., are there any tools that could answer such questions?...Besides,while doing performance testing with some available java tools,like JProbe, I found that it identifies bottlenecks in some methods(hotspots)..but how do I now the user-response time using tools likt that? (I don't expect precise answers wrt. the tool that i use,but, in general, how does one analyse the tool results?)Unknownhttps://www.blogger.com/profile/00321791680207244843noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-54233456747460784972007-09-25T12:42:00.000-07:002007-09-25T12:42:00.000-07:00Swathi -- if your app runs on Linux, then there's ...Swathi -- if your app runs on Linux, then there's top, vmstat, iostat, mpstat, and others. For database monitoring, if you're using MySQL, there's a tool called mytop (google for it).<BR/><BR/>On Windows, there's the PerfMon tool that allows you to monitor all kinds of OS-related counters.<BR/><BR/>GrigGrig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-6604649505147649812007-09-25T12:32:00.000-07:002007-09-25T12:32:00.000-07:00A highly informative and precise article...can you...A highly informative and precise article...can you also let me know what are the simple, open source tools available to profile and monitor the CPU, MEMORY, DATABASE upon running an application?. As you've mentioned, some toolkits should serve the purpose, but is there any way to interpret the results given by the tools in an understandable fasion? If yes, can you please suggest some of such inexpensive lightweight toolkits that help us with the statistics and analysis w.r.t. Database queries, CPU & Memory?Unknownhttps://www.blogger.com/profile/00321791680207244843noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-63654626201338787882007-09-17T07:56:00.000-07:002007-09-17T07:56:00.000-07:00Both of your articles were very informative and ev...Both of your articles were very informative and even though I had a fairly good grasp of the topic, your clear explanations assisted me greatly in explaining the differences to less technical business staff. Cheers, Darren.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-21559977755305681752007-08-27T15:36:00.000-07:002007-08-27T15:36:00.000-07:00Hi, How do you do performance testing for a clien...Hi,<BR/> How do you do performance testing for a client server application?In other words what are all the things we need to take into account?We do not want to use any tool for this...How can we acheive performance testing then?Unknownhttps://www.blogger.com/profile/07582164796190852538noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-34241827813515533852007-07-23T23:09:00.000-07:002007-07-23T23:09:00.000-07:00HI Grig,Thanks for very nice articles :)Can you pl...HI Grig,<BR/>Thanks for very nice articles :)<BR/>Can you please explain these three terms in only 3 lines.<BR/><BR/>Thanks,<BR/>RajivRajiv Waliahttps://www.blogger.com/profile/09636065749808085354noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-30666260316352176242007-05-27T21:20:00.000-07:002007-05-27T21:20:00.000-07:00Hi Grig ,The article really helped me in understan...Hi Grig ,<BR/>The article really helped me in understanding the diff between perfromance and load testing.<BR/>Basically im a black box tester and im doing load testing with a tool called webload.<BR/>In my case im facing a problem whent the load has been increased beyond 40 users.<BR/>And also the results which were given by the tool in QA environment does not help me more while the application has been launched in production.<BR/>My application performance is degrading day by day and im not able to identify the key reason for this.<BR/>Can u suggest me some other tools which will help me find the root cause fot this degradation like some n/w performance identifying tools and data base optimizer tools.It will be great if these tools are open ware.Unknownhttps://www.blogger.com/profile/10571005137916059033noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-53888980832835289582007-02-07T15:39:00.000-08:002007-02-07T15:39:00.000-08:00phoenix -- you may be able to differentiate betwee...phoenix -- you may be able to differentiate between performance and benchmark testing by the fact that benchmark testing is usually done with industry-standard tools and it usually follows industry-standard procedures. As an example, check out the TPC benchmarks for database transaction processing at http://www.tpc.org/<br /><br />GrigGrig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-260201964170909902007-02-06T20:56:00.000-08:002007-02-06T20:56:00.000-08:00Nice article.
But I'm confused with one more aspec...Nice article.<br />But I'm confused with one more aspect of testing which is Benchmark testing. In Benchmark testing we need to findout a reference point which can be use for further analysis, and in Performance testing we need to analyse the performance of product. <br />If I have a web application and wanna to do performace testing on that application then how can I differenciate it with Benchmark testing?Unknownhttps://www.blogger.com/profile/03751343000064130823noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-63741434156087539602007-01-08T02:22:00.000-08:002007-01-08T02:22:00.000-08:00Grig,
Nice to see a comparative study on the "Perf...Grig,<br />Nice to see a comparative study on the "Performance" jargon.<br />An applaudable effort.<br /><br />PrabhakarPrabhakarhttps://www.blogger.com/profile/09782001859230257590noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-1141163758615804092006-02-28T13:55:00.000-08:002006-02-28T13:55:00.000-08:00I have been using Microsoft Web Application Stress...I have been using Microsoft Web Application Stress Tool. It is very easy to use, allows you to record scripts by navigating around your web site, has settings for number of concurrent connections, fixed or random delay, etc. and can coordinate running the scripts across multiple client machines if needed in order to increase the load on the server. <BR/><BR/>Although it is not an open source tool, it is available for free download from Microsoft.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-1135100667035048142005-12-20T09:44:00.000-08:002005-12-20T09:44:00.000-08:00Barakino,Take a look at some of the tools I mentio...Barakino,<BR/><BR/>Take a look at some of the tools I mentioned in my <A HREF="http://agiletesting.blogspot.com/2005/04/http-performance-testing-with-httperf.html" REL="nofollow"> HTTP performance testing with httperf, autobench and openload</A> post.Grig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.com