Performance Testing - Starting Out
We recently started implementing performance testing procedures for our products in my company. The trigger wasn’t an execution of some thought-out plan but a desperate measure following a service disruption unfortunately. I always thought that we had more time. I have to admit, it was very naive and irresponsible thinking. Regardless of the precursor, here are some initial observations after I started looking for performance testing practices.
First of all, in comparison to all resources dedicated to scalability and NoSQL, the web felt semi abandoned when it came to performance testing. The reason for this comparison is not arbitrary. My line of thinking was that before jumping on scalability initiatives, and before moving from relational databases to document ones to address performance concerns, for example, people would first need to show that those are, in fact, concerns. I thought I wouldn’t have any problems locating first hand accounts on how teams approached performance testing and found pressing needs to tackle scalability and try out NoSQL. No such luck. I hope folks choose NoSQL because it’s a better tool for certain jobs, not because it solves imaginary performance issues.
Search results for performance testing tools was a list of well known and old suspects. Some of them weren’t updated in years. By itself it’s not such a bad thing. The tools could have matured and reached a level where frequent updates weren’t necessary any longer. The biggest surprise was a discovery of almost perfect vacuum in place of how-tos and tutorials on integration performance testing as a part of an application lifecycle management. No set-ups shared, no words of wisdom, no proverbial best practices.
There are two reasons that I can think of to explain this:
1) Performance testing is bordering art and magic, and it’s very dependent on the application or system being tested. There may be hundreds of companies in my city only, that do this every day, but their set-ups are so custom that they don’t feel sharing any details would benefit anybody else.
2) Performance testing is not really common. Applications go down generating wake up calls to development and operation teams, who them scramble in haste to fix the problem.
What is your experience?