Blog post series about testing development projects, part 10/11
Articles about planning software testing and managing it.
Point of views:
- Business critical development projects, where IT plays a strong role.
- One or more outside vendors.
- Client’s business personnel as testers. They are business professionals, not software testing professionals.
- Goals for testing:
- Make sure that the implemented solution supports the needs of the business.
- Make sure the implementation doesn’t endanger the business.
Why it’s useful to make predictions
As more and more projects become agile, reporting has to support agile work environment. I use predictions in project reporting. Before each project phase I make a progress prediction and compare it to a real time progress percentage.
Why I do this:
- I get a report that is easily interpreted
- Project is more freely executable
- I include unplanned activities to the report to get a more realistic status overview
- I get an objective for every week –> Tool for management and incentive programs
An example of definition phase progress prediction and readiness
Include unplanned activities in the progress percentage
Usually the progress percentage is calculated from activity progress. All change requests are reported separately.
I include all unplanned activities like open issues in my calculations as well.
Why I do this: Going through every issue takes time. I use an estimate of 2.5hrs per issue –> If there are 50 open issues in a project, work hours come up to around 125hrs.
When making a prediction I always estimate the work load for open issues. I make sure that the project has the capacity to solve problems.
How to calculate the prediction graph:
- Plan the upcoming project phase.
- Plan processes: e.g. How to write and accept the use case. Define progress percentage for every process phase.
- Plan who does what and when.
- Make an estimate of open issues and the time to process them.
- Draw a graph of the progress – ergo the prediction
- Communicate, accept plan = commit
- Compare real time progress to your prediction
How to use predictions, an example
Below is an example of how to use predictions to your benefit in testing. This is how I made the prediction:
- I made testing and resource plan.
- With the help of vendors and past data, we estimated the number of defects.
- I made a graph based on the plan. I estimate the test readiness daily.
When the testing started, we followed the realisation compared to the prediction. Testing started as planned. During the first few weeks the number of defects started to grow.
-> Getting tests ready was getting slow. Going through defects and retesting was taking a lot of time. Defects were preventing many test cases from being tested.
So what was the benefit of making predictions?
In this case, the larger number of defects than what was estimated, was because two areas were not as ready as reported. Well, what was the benefit of making predictions then?
- I could clearly point out the reason, why testing is progressing slower.
- When we made actions, we could track their effects in real time.
- The vendor made strong effort to fix the situation.
- Testing resources and schedule was re-planned.
- The work put in this testing was approx.172 hours more than estimated, because there were more defects found.
- We had a tool that helped us react to problems on time -> Because of the things we fixed, we were eventually able to finish the implementation successfully and on time.
Well communicated prediction steers people to work according to plan. If you announce that we are 5% late from our goal that week, people start to think what tasks could be done to make that goal.
Another clear benefit is that making predictions requires careful planning. Before you publish and communicate your graphs, you actually have to have your plan well thought out and monitoring set to place.
