Software projects fail because many current business models are at odds with the way we developers build software. Businesses who like to command and control or encourage internal competitiveness will fail at producing software which requires permission to fail, collaboration and freedom to innovate. For a development team to produce great software they must combine the creativity and innovation of young artists with discipline and collaboration skills. Organisations that are unable to create environments where these skills can grow face the slow and painful reality of failing software projects.
Command and Control doesn’t work. You can’t tell a craftsman what to do, he learns through years of practice. Developers are the same, being poorly managed can send them to the hills/coast leaving teams in tatters. Given the right goals and boundaries good developers will self organise and hone their own process. They just want to develop great software and with the right culture and mentoring will, through practice, succeed. Interference, enforced practice and process will only hinder this.
Teams must be trusted and given the space to learn about the problem they are solving through continual experimentation and rework. Without rework we are left with an unmaintainable code and the longer that sticks around the harder it is to remove. Working with experienced developers who are able to collaborate and share their knowledge speeds the process up. Learning on the job and rework go against many modern businesses instincts where short term thinking is king and if you don’t have the knowledge you simply buy it in. Every problem is unique and although you can buy in experience and talent there is always something more to learn. Learn this lesson early.
Agile processes bring us closer to the business, we need to understand how their often crazy world works. That doesn’t mean we have to become part of that world, and it doesn’t mean we have to follow their rules.
The prevalent business culture is unlikely to change quickly. As companies who fail in allowing their teams to be productive outsource their development making their smartest employees redundant and sacrificing their domain knowledge, the software houses in the East grow stronger and smarter. To save your team you need to do whatever it takes to gain the trust, freedom and independence that you need to be productive.