Poor quality of the project estimation is often attributed as the main reason for missing deadlines in most of the software projects. Slipping deadlines is most common and even management teams anticipates this event and accordingly negotiate pricing with their customers. As this happens to be "known Risk" and corrective actions are taken upfront, the people involved in the wrong estimates are not penalised. Even if they do, it is only going to impact their "annual" appraisal. They might slip just "one" grade. Most people try their best to arrive at right estimates but there are few scenarios I will try to discuss later in this post that may impact the success. On the otherhand there are certain cases where there is higher propabiliy of failures.
1) Manager/Project Lead, who is estimating the project, divides the task into various functional tasks, allocates tasks to individual developers to provide estimates. He/She uses their prior experience to adjust the figures. Most often it is fixed percentage +/-. (Usually on the negative. 5 days estimate may be cut short to 3 days). After adjustments, adds all the individual estimates each developer and arrive at a final estimate. He again applies his experience to adjust a fixed percentange (+/-). (Usually on the positive. 40 days overall estimate becomes 55?)
2)Manager/Project Lead, who is estimating the project, uses standard estimation methodologies but donot have hands-on experience.
There are definitely number of scenarios that effect the project timelines attributed to the customer such as
- Requirement changes
- Unavailability of key resources for testing
- Deployment environment availability for UAT and Production site
- etc
Project teams breathe a sigh of relief, when any of the said events occur. Sometimes, we may hope for this type of oppurtunity to excuse overselves to revise the estimates.
Unfortunately, if we work in a project where all the requirments are crytal clear and there is no scope to blame on others except for yourselves, what are the excuses for slipping deadlines?
1. Usability requirements
Many of the requirement specifications may not state explicitly certain features that customer/end-user expects to be "obvious" or "assumed", but they take significant time of development. I could quote an example- usual happens in custom development projects. "Requirment spec" states a need for a "contact list" with features to add, modify, delete, search. If I pose this question to a developer I may get a response from anywhere between 2 person days to 5 person days, depending on an enthusiastic developer or seasoned professional who has seen several of this type of requests earlier in his career. what is assumed by the developer?
a) A data entry screen with 4 or 5 attributes.
b) A "List" screen that shows list of contacts in an alphabetical order (or in the order of entry-date).
c) About 3 queries to "Insert", "Modify", "Delete" into persistance storage (Sqlserver/Oracle/Mysql etc)
d) On the "List" page - 2 or 3 important fields to filter the data.
e) Menus and Navigational buttons to integrate this feautre along with other application features. "Actions" to be performed on the "List" page. (Add/Modify/Delete)
The requirements appears clear and solution is in the developers mind as soon as he is told about the requirement.
However, in the implementation phase, he notices that among 4 or 5 attributes there fields such as "country", "state" and "city". Fortunately in the same project, if these fields are defined elsewhere and if he has to reuse the same, partially the problem is solved. If it is not the case, he has to estimate for additional scenarios. For each of the three additional attributes a repeat of steps a) to e) planned for "contact list". We may argue that these are simple attributes and may take half-day each. Even then, 1.5 person days effort is added.
The nightmare donot end here. When he successful creates the above additional screens/objects/tables, and start using them in data-entry screen, he uncovers more issues.
He notices that he has to create drop-down fields for these attributes and they are inter-dependent. The "state" is dependend on the "country" and "city" is dependend on the "state". The end-user expects the fields to filter automatically depending on the selection. (Unstated usability requirement?). Also, all the 3 drop-downs tend to be long. End-user finds it difficult to select from the list. (Usability requirement). We may need a custom drop-down with auto-suggest feature (like google auto-suggest search) Or a tiny button next to the corresponding "input" field to open a popup that has facility to search by a keyword. Another issue with this type of auto-suggest or popup searches - what user sees is a "text string" in readable format is different from what program needs, usually "a code or an id". We have to add additional attributes to hold these extra fields.
Similarly for point b) a sort feature on the column heading may be required (Usability requirement). Paging may be required showing Pages (1,2, 3 .. , prev, next, first, last etc).
For point d) the dependent drop-down issues has to be handled if the filter is needed on fields like "country", "state" or "city".
We may derive more usability features from this requirement but at the end, the original estimate is definitely way beyond the actual.
Another important point to note is that it is not required by the end-user to explicity spec the requirement for countries, states and cities as stated in the above example. A smart developer may introduce just "text fields" to input these attributes at the expense of usability.
2. Non-functional requirements
Performance requirements, Data audit requirements, security requirements(Access control and authorization) are usually specified in the project requirements. However, the additional effort may not be estimated when an individual task is given to a team or developer. He assumes that it is taken at the overall project level and he is estimating only for a sub-task. However, we need to explicitly allocate time for performance tuning of the new changes related to his sub-task, persisting audit data, and checking user permissions before allowing to access features like "view","add", "modify" or "delete". Sometimes, architecuture and design of the project accomplishes this task automatically. If not additional effort needs to be attributed. There are other tasks such as documentation, and component reuse. In the example quoted in the previous section, if the "contact list" is supposed to be reused across several projects, considerable effort has to be estimated to make the component reusable and pluggable.
3. System architecture overheads
Most application designs introduce multi-layer architecture where data flows from Front-end to the persistance layers. There is data conversion requirement between layer to layer. For example in a typical web-based java project, there is a need to convert Html form elements to Java objects or beans. In other application layer there is need to convert Java objects to SQL statements. Fortunately, there exists today, a number of frameworks that does the conversions seamlessly such as Struts, Spring, Hibernate. In the absence of the above frameworks and utilities in the project - due care has to be taken in estimation to factor the conversion efforts.
4. Quality Process
Majority of the software development companies have defined set of quality processess that adds certain man-hours to the project. Most often it is a fixed percentage of the estimates but people tend to forget this fact at the time of estimation. Testing, Peer reviews, QA documentation etc takes significant time and resources.
5. Other factors leading to missed deadlines
- Unplanned leaves
- Sick leaves
- Team communication. As number of team-members associated in accomplishing a task increases more inter-dependencies are introduced and may delay the overall delivery.
- Build, deploy cycles in Testing, UAT and production environments.
- Multiple project versions when more than one project team is working on a large project. In the version control terminology - need to branch, merge, label etc are significant time killers.
There could be more such issues that impacts project timelines. I'm fortunate to work in companies where there are multiple stages of estimations - Level 0, Level 1, Level 2 and Level 3. However, in Agile developement and Iterative development project engineering scenarios, the luxury of doing multi stage estimations is lost. In the previous case we would have missed deadlines only 4 times in a project (Level 0 ...Level 3). In case of Agile and Iterative development process - project teams skip deadlines every week.
1) Manager/Project Lead, who is estimating the project, divides the task into various functional tasks, allocates tasks to individual developers to provide estimates. He/She uses their prior experience to adjust the figures. Most often it is fixed percentage +/-. (Usually on the negative. 5 days estimate may be cut short to 3 days). After adjustments, adds all the individual estimates each developer and arrive at a final estimate. He again applies his experience to adjust a fixed percentange (+/-). (Usually on the positive. 40 days overall estimate becomes 55?)
2)Manager/Project Lead, who is estimating the project, uses standard estimation methodologies but donot have hands-on experience.
There are definitely number of scenarios that effect the project timelines attributed to the customer such as
- Requirement changes
- Unavailability of key resources for testing
- Deployment environment availability for UAT and Production site
- etc
Project teams breathe a sigh of relief, when any of the said events occur. Sometimes, we may hope for this type of oppurtunity to excuse overselves to revise the estimates.
Unfortunately, if we work in a project where all the requirments are crytal clear and there is no scope to blame on others except for yourselves, what are the excuses for slipping deadlines?
1. Usability requirements
Many of the requirement specifications may not state explicitly certain features that customer/end-user expects to be "obvious" or "assumed", but they take significant time of development. I could quote an example- usual happens in custom development projects. "Requirment spec" states a need for a "contact list" with features to add, modify, delete, search. If I pose this question to a developer I may get a response from anywhere between 2 person days to 5 person days, depending on an enthusiastic developer or seasoned professional who has seen several of this type of requests earlier in his career. what is assumed by the developer?
a) A data entry screen with 4 or 5 attributes.
b) A "List" screen that shows list of contacts in an alphabetical order (or in the order of entry-date).
c) About 3 queries to "Insert", "Modify", "Delete" into persistance storage (Sqlserver/Oracle/Mysql etc)
d) On the "List" page - 2 or 3 important fields to filter the data.
e) Menus and Navigational buttons to integrate this feautre along with other application features. "Actions" to be performed on the "List" page. (Add/Modify/Delete)
The requirements appears clear and solution is in the developers mind as soon as he is told about the requirement.
However, in the implementation phase, he notices that among 4 or 5 attributes there fields such as "country", "state" and "city". Fortunately in the same project, if these fields are defined elsewhere and if he has to reuse the same, partially the problem is solved. If it is not the case, he has to estimate for additional scenarios. For each of the three additional attributes a repeat of steps a) to e) planned for "contact list". We may argue that these are simple attributes and may take half-day each. Even then, 1.5 person days effort is added.
The nightmare donot end here. When he successful creates the above additional screens/objects/tables, and start using them in data-entry screen, he uncovers more issues.
He notices that he has to create drop-down fields for these attributes and they are inter-dependent. The "state" is dependend on the "country" and "city" is dependend on the "state". The end-user expects the fields to filter automatically depending on the selection. (Unstated usability requirement?). Also, all the 3 drop-downs tend to be long. End-user finds it difficult to select from the list. (Usability requirement). We may need a custom drop-down with auto-suggest feature (like google auto-suggest search) Or a tiny button next to the corresponding "input" field to open a popup that has facility to search by a keyword. Another issue with this type of auto-suggest or popup searches - what user sees is a "text string" in readable format is different from what program needs, usually "a code or an id". We have to add additional attributes to hold these extra fields.
Similarly for point b) a sort feature on the column heading may be required (Usability requirement). Paging may be required showing Pages (1,2, 3 .. , prev, next, first, last etc).
For point d) the dependent drop-down issues has to be handled if the filter is needed on fields like "country", "state" or "city".
We may derive more usability features from this requirement but at the end, the original estimate is definitely way beyond the actual.
Another important point to note is that it is not required by the end-user to explicity spec the requirement for countries, states and cities as stated in the above example. A smart developer may introduce just "text fields" to input these attributes at the expense of usability.
2. Non-functional requirements
Performance requirements, Data audit requirements, security requirements(Access control and authorization) are usually specified in the project requirements. However, the additional effort may not be estimated when an individual task is given to a team or developer. He assumes that it is taken at the overall project level and he is estimating only for a sub-task. However, we need to explicitly allocate time for performance tuning of the new changes related to his sub-task, persisting audit data, and checking user permissions before allowing to access features like "view","add", "modify" or "delete". Sometimes, architecuture and design of the project accomplishes this task automatically. If not additional effort needs to be attributed. There are other tasks such as documentation, and component reuse. In the example quoted in the previous section, if the "contact list" is supposed to be reused across several projects, considerable effort has to be estimated to make the component reusable and pluggable.
3. System architecture overheads
Most application designs introduce multi-layer architecture where data flows from Front-end to the persistance layers. There is data conversion requirement between layer to layer. For example in a typical web-based java project, there is a need to convert Html form elements to Java objects or beans. In other application layer there is need to convert Java objects to SQL statements. Fortunately, there exists today, a number of frameworks that does the conversions seamlessly such as Struts, Spring, Hibernate. In the absence of the above frameworks and utilities in the project - due care has to be taken in estimation to factor the conversion efforts.
4. Quality Process
Majority of the software development companies have defined set of quality processess that adds certain man-hours to the project. Most often it is a fixed percentage of the estimates but people tend to forget this fact at the time of estimation. Testing, Peer reviews, QA documentation etc takes significant time and resources.
5. Other factors leading to missed deadlines
- Unplanned leaves
- Sick leaves
- Team communication. As number of team-members associated in accomplishing a task increases more inter-dependencies are introduced and may delay the overall delivery.
- Build, deploy cycles in Testing, UAT and production environments.
- Multiple project versions when more than one project team is working on a large project. In the version control terminology - need to branch, merge, label etc are significant time killers.
There could be more such issues that impacts project timelines. I'm fortunate to work in companies where there are multiple stages of estimations - Level 0, Level 1, Level 2 and Level 3. However, in Agile developement and Iterative development project engineering scenarios, the luxury of doing multi stage estimations is lost. In the previous case we would have missed deadlines only 4 times in a project (Level 0 ...Level 3). In case of Agile and Iterative development process - project teams skip deadlines every week.
Comments
Post a Comment