Following are the few factors which every performance engineer should know:
- Which type of performance testing is needed?
- Knowledge of known issues in application
- Non-functional requirements (NFR)
- Important Data Collection
- Network and Infrastructure knowledge
- Performance Testing Methodology
- Result Analysis and recommendations
- Which type of performance testing is needed?
There are around seven type of Performance Testing which helps us to understand the behavior of the application:
1. Load Testing
Load Testing help us to study the behavior of the application under various loads. The main parameter to focus is Response Time. This understanding revels how the application handles concurrent user effectively.
2. Endurance Testing
Endurance Testing reveals memory utilization when you load your test for prolonged execution. Usually you execute your test for 1 hour for load testing. But in endurance testing, you execute your test for at least 8 hours with significant load. Main parameter to focus is Memory.
3. Stress Testing
Stress Testing help us to observe the stability of the application. The main intention is to identify the breaking point of the server. The main parameter is to focus is Response Time and throughput (how much data is being transferred between clients and severs).
4. Volume Testing
Volume Testing helps to identify whether the server can handle more number of high complex queries under significant load. The main intention is to observe how the database acts under various circumstances. Main parameter to focus is Response time of Queries.
5. Spike Testing
Spike Testing is performed by sudden increase/decrease in load and observing the behavior of the application. This helps us to determine whether the application can handle significant changes in load or not. Main parameter to focus is Response Time.
6. Availability Testing
This testing ensures whether the application is available 24×7. If you would’ve noticed any website hosting providers, they advertise as 99.99% up time. Availability testing helps us to determine whether the application is up and running fine without any issue. Main parameter to focus is Response Time.
- Scalability Testing
Say, your website has 1000 visitors per month with 10% increase in unique visitors. By forecasting the number of visitors by expected business growth and performing the load test is called scalability testing. Main parameter to focus is Response Time and Maximum Transactions per second.
- Knowledge of known issues in application
Prepare a list of current and past known issues from clients/developers. This will help performance engineer to analyze the application for performance testing.
E.g. any memory locks issue faced in live environment, application being hacked due to security reasons, unexpected growth in daily visitors, more response time which leads to time out error etc. Different issue can be fixed by different type of performance testing.
- Non-functional requirements (NFR)
Crystal clear Non-functional requirements i.e. NFR elicitation is mandatory task for performance testing. It defines the inception phase. This can be Microsoft Word / Excel document which consist of Non-functional requirements’ detailed/overview of application under test (AUT).
E.g. Application purpose, Number of daily/monthly visitors, risk associated with the project, performance monitoring etc. It could seek the technical data from the clients such as technology of AUT, types of servers deployed, infrastructure model, protocol details etc. If the business analysts are not able to respond, seek the response from the developers. This will help performance engineer to plan the performance testing.
- Important Data Collection
It is important to collect all critical data like expected response time, throughput, transactions per second, etc. This data can be collected from server logs. Following are the important parameters must collect from clients.
- Expected response time
- Throughput
- Transactions per second
- Concurrent Users
- CPU Utilization
- Memory Utilization
- Disk Utilization
If no data available for expected response time, you can refer industry standard which is 5 seconds.
- Network and Infrastructure Knowledge
It is important to gather details of network and infrastructure from Developers. LAN/WAN details, terminal servers, bandwidth link, load balancing techniques, batch transactions and disaster recovery details are the important details must to know before you start performance testing process.
- Performance Testing Strategy
Performance Testing Strategy defines how to approach performance testing and accomplish the mission to identify the performance issues. It defines the project in-scope, out-scope, risk and mitigations, tools licenses, protocol licenses, number of engineers to work etc.
- Result Analysis and Recommendations
Performance engineer must be able to reading the graphs between various parameters and provide his inferences and recommendations. Also correlating the graphs revels more details which helps to identify the bottlenecks. Experience makes a man perfect. This proverb fits for performance testing analysis as well.