Factors and Issues that Influence the Behaviour of Software Engineering Groups

Most presentations on software engineering highlight the historically high failure rates of software projects, of up to eighty percent. Failure under the guise of budget overruns, delivery of solutions not compliant with specifications, late delivery and the like. More often than not, these failure rates are used to motivate the use of software engineering practices. The premise being that if adequate engineering practises were utilised, failure would become more of an exception rather than a rule. Best practise and lifecycles have been proposed and tailored to the various paradigms that the computer and information sciences throw up in rapid succession. There is extensive debate on what works and what does not within academia and without. The consensus being that what is best depends on the problem at hand and the expertise of those working on the problem. A few software engineering group models have been popular in the history of software development. Earlier groups tended to be hierarchical, along the lines of traditional management teams. The project manager in charge did not necessarily contribute in a non-managerial capacity and was responsible for putting together teams, had the last word on accepting recommendations and delegation to team members. Later groups worked around one or more chief-programmers or specialists.

The specialists took charge of core components themselves and were assisted by other group members in testing, producing documentation and deployment. More recently, collegial groups have become common. Here, people with varied specialisations form groups wherein they organise themselves informally by assuming roles as needs arise. The advantage of a particular model over the others becomes evident only in the context of specific projects. The hierarchical model is best suited to relatively large projects that are decomposable into sub-goals that can be addressed by near independent teams. This is usually possible for software tasks that are very well defined, that need reliable and quality controlled solutions, particularly those that are mission critical. A large project may inherently require many people working on it to successfully complete it, if it were to be deployed in multiple sites, for instance. Alternatively, a large group may be assembled to expedite delivery. In either case, structured organisation and well-defined roles facilitate coordination at a high level. A central problem with adding people to expedite delivery, or otherwise, is that the effectiveness of a group does not scale linearly. One person joining another does not mean that they are collectively twice as productive.

More importantly, the contribution of the seventh person in a seven-person group is a fraction of the contribution of the second person in a two-person group. This is due to additional overheads in communication and coordination as group size increases and to the dilution of tasks assigned to individual member. As is evident, this is a problem for any group; however, in very large groups the problem is exacerbated. In hierarchical settings, group members do not have a sense of ownership of the bigger solution. This may be reflected in their productivity. Because of the concentration of decision-making powers to particular individuals according to some hierarchy, the success of processes ultimately lies with them. A lot rides on their ability to pick the best practises and recommendations, delegate effectively and keep track of the bigger picture. In quality-controlled or mission-critical settings, there are not many alternatives to having large hierarchical groups with redundant contributors. Primarily in non-commercial settings, a single specialist engineers a complete software solution. Invariably, the solution being a prototype is accessible only to other specialists.

In addition, it is not designed for general consumption and is put together without going through most recommended processes in software engineering lifecycles. Single programmers tend to practise evolutionary programming. This involves producing a quick working solution followed by repeated reworking of the solution to make it more accessible to the programmer for future review, incremental development and peer review or development. If demand for such a software solution gains momentum, for either its general utility or its commercial viability, the core solution would most likely be adopted for further development by a larger software engineering group. It stands to reason that the core developer, who is most familiar with the solution, retains the last word on further technical development. Other members organise themselves around the chief programmer. In general, some form of incremental development and periodic redevelopment from scratch of software solutions are common regardless of group models. The first incrementally developed solution tends to be the least well-engineered solution and is a patchwork of poorly designed and tightly coupled components. This is a reflection of the difficulty involved in producing quick solutions using new tools and techniques and inexperienced software engineers. Supported by a high immediate cost barrier to reworking solutions, incumbents from pervious software development cycles, spend a lot of their post deployment time in supporting and patching what they produced.

In collegial groups formed in smaller organisations or departments, software engineers assume roles as needs arise. Brainstorming may be carried out by all members and design approved by consensus but development may be carried out by a few individual members, while the others gain feedback from end-users, keep track of competitor solutions and the like. In the initial phases of a software development life cycle, the problem definition, feasibility study and system analysis phases, end users of the system and independent specialists may form part of the group. During the design and implementation phases, a disjoint group of outsiders could merge with the team. The external members may then be invited for their feedback post implementation during the quality assurance and maintenance phases. Generally, best practise suggests that groups should be adaptive or loosely structured during the creative phases and become more structured as the design becomes clearer. Groups with loosely defined structures are the most flexible in adapting to changing user needs. However, the greatest risk to project cancellations and overruns are ill-defined and changing requirements.

Adaptiveness to an extent is crucial. Given that users change requirements so compulsively, lacking adaptiveness completely would make an engineering group not viable. If group size is variable, the learning curve of new entrants must be kept in mind. A project manager hiring additional developers late in the software development cycle, after not meeting some deadline say, must factor in delayed contributions from the newcomers as a result of time taken by them to familiarise themselves with the project and time lost in coordinating their joining the group. Following this, the next most common cause of failure is due to poor planning or management. If the person taking on the role of project manager has poor management or planning skills, the likelihood of which is heightened by the fact that each group member is called upon to serve in diverse capacities, projects are destined to fall over. A number of reasonable software engineering guidelines are ignored by software engineers commonly. When programming, using descriptive names for variables is a good example. A section of program code will immediately make sense to its author for a reasonably long period, when reviewed.

However, if the code were not documented sufficiently, which includes using descriptive variable names, and with the correct intended audience in mind, it would take a considerable amount of time for another programmer to understand what the other had implemented. In the extreme, some programmers obfuscate because they can or to ensure that only they will ever understand what they have written thereby making them indispensable. The potential for doing a half-hearted job of writing code is obvious in that poorly structured and poorly designed code is functionally indistinct from well-structured code and is less demanding a task. If software projects were evaluated only on their functionality, this would not pose a problem but upgrades and patches require someone to review the code and add to it or repair it in the future. The long term cost of maintaining software that is not well designed and documented may rise exponentially as older technologies are phased out and finding people competent to carry out repair and review shrink. In essence, this is an instance of a quality control problem. Uncontrolled quality problems are the third most common cause of cancellations and overruns of software projects. It is convenient to group documentation along with quality control as they should be reviewed in tandem in a software development lifecycle.

The first casualties of a late running project are quality control and documentation. The long-term costs of skimping on either have been illustrated by example above but there are short-term costs as well. In both evolutionary engineering common among specialist-centred groups and component engineering commonly employed by hierarchical groups, the quality of each revision or component affects the quality of subsequent revisions or combined components. The next most common causes of failure are unrealistic or inaccurate estimates and naAve adoption of emerging technologies. The blame for the former rests with both users and planners or project managers. Most engineering groups are unrealistically optimistic about the speed with which they can deliver solutions. Their estimates may be accurate for prototypes.

In actual deployment, conformance to specifications, human-computer interfaces, quality control, training and change management are essential and take time. Users have a poor understanding of how descriptive their specifications are and much too often assume that implementers are contextually familiar with the environments in which they work and intend to use the system. Project managers and implementers have an affinity to emerging technologies ignoring their core competencies that are more likely to be established proven technologies. Success among software engineering groups is a function of planning and execution. The responsibility of planning falls on a project manager. A manager must draw on the best a group has to offer, appreciate software and technical concerns, facilitate communication and coordinate a groups effort.

Enforcing quality standards from the beginning by adopting design and programming guidelines, for example, helps formalise expectations. A project manager with a technical background has the advantage of understanding the position of other technical members and is likely to communicate more effectively with them and has the opportunity of leading by example. Given the emphasis on planning, it is worthwhile noting that it can be overdone. Over-engineering is not ideal engineering. It is often convenient for a single developer to take the lead for coding. Other developers and end-users should concurrently test the developing solution for functionality, usability and quality. Execution in isolation is likely to result in solutions that developers are comfortable with and even proud of but that end-users find lacking. The various stakeholders of the project must be simultaneously and consistently involved throughout the development cycle of software projects.

The greater the communication between specialist designers and specialist implementers, the more successful the group would be in terms of quality and ease of use of solutions. The technical crowd in a software engineering group sees the problem uniquely in terms of simplifying or making more elegant their contribution. The design crowd balances out this perspective by offering an alternative view, which is more likely to be aligned with that held by end-users, uncurtailed by technical considerations. Ultimately, end-users must be given an opportunity to have their say. The solution is theirs. Changing requirements and specifications may be an acceptable excuse from the user's perspective for delays in final solution delivery. Many projects are twenty percent complete after eighty percent of the initially estimated time. More people are brought in to expedite the process, budget overruns follow and sub-par solutions are delivered, albeit, late. Given the historical frequency, project managers should factor in possible requirement changes to arrive at estimates that are more realistic before commencing projects.

Cite this page