Software engineering management is a complex topic. Finding good managers isn’t easy. Finding good software engineers isn’t easy. Taking a good software engineer and turning them into a manager can sometimes work, but in my experience, that’s very rarely successful. The two roles require very different skill sets. Engineers need attention to detail, computer science fundamentals, and specific technical skills. Managers need leadership skills, communication skills, collaboration skills, and project management skills. Even if you were to find an outstanding individual with both skill sets you don’t want to try and force their career growth and direction – you want to allow them to take their career where they want to.
This impacts how to structure a software development organization. The organization needs to understand how to reward technically inclined staff. The organization needs alternate sources of talent for front-line management. The organization may need to develop ways to integrate nontechnical managers to technical teams.
I’ve seen several ways to reward and develop technical staff over the years, each with their own pros and cons. A separate technical title track with things like Principal Engineer or other superlatives (lead, chief, etc.) can be used to reward someone. The different technical titles are visible to others, which can be both a pro and a con since it says to everyone that the organization considers you more talented/experienced than others, but it could also cause jealousy if someone felt they were in the wrong place in the hierarchy. Also, giving out a new title is generally expected to be accompanied with a raise, so an excess of title-based differentiation gets expensive fast. Another option is a specialized assignment or group; like the surgical team from Fred Brooks, or a specialized research group. This can work well on a short-term basis, but the surgical team isn’t always needed, and it is difficult to keep the research group focused and relevant day-to-day in a smaller organization. The last option I’ve seen is perks of various sorts, e.g., bonuses, special conference travel, extra vacation, raises, or allowing someone to work on a pet project. Most of these can be public or private depending on what the perception would be, however, these are all essentially short-term rewards. None of these options is very good, when you consider motivation theory.
All of those options above are extrinsic motivation, but extrinsic motivation has been found to be less effective than intrinsic motivation. Intrinsic motivation drives the best of human endeavours – it is your internal drive to do well because it is satisfying. I highly recommend this video by Dan Pink on the subject, since it covers it better than I can here. He notes that you need to create an environment that will allow people to apply their internal drive to do well, not because they want a title or recognition.
Creating this environment comes strongly from the low-level management you have. This makes finding alternate sources for where to get front-line managers important to building a resilient organization if you aren’t going to be drafting developers into management. I don’t have real answers as to where to find some magical source of leaders you can tap into. There are a couple of other options you can use to try to minimize the impact of moving into management while using your existing staff. You can try to encourage the leadership tendencies that people do show, so that each member of the team does a part of the role that would otherwise be performed by a single “manager.” You can try to build management roles that have more leadership and less management, essentially structuring lower-level management roles to have time to continue doing technical work. This causes issues with making sure they are doing both sides of the role consistently. I’ve seen people in these types of roles eschew one portion of their role in favor of the other and either way can cause considerable friction and limit the managed team’s ability to achieve its objectives.
Integrating nontechnical management to technical teams isn’t easy. You need a non-technical manager who can accept the technical focus your staff brings and integrate their technical focus into the rest of your organization. They should be someone that understands how technical staff needs to have large blocks of time to do the technical work. Someone who can help clear out impediments and let the technical staff keep their focus. You also want your technical staff to have a say in who they work with. You wouldn’t have Team A do the interviews for Team B only to find out that the candidate didn’t get along with Team B. There needs to be mutual trust between the team and leadership so that everyone can focus on doing the work rather than various CYA actions.
This nontechnical management shouldn’t be directly running technical teams, that should be handled by technical leadership (under whatever model for that you prefer). At some point product/project management needs to feed information about what they want into the technical organization. They will want status updates back too. Technical management should be responsible for dealing with nontechnical management directly and generally shielding the engineers from potentially confusing interference.
Software engineers (or similar jobs requiring very specific skills) shouldn’t feel that the only way to progress their career is going into management. I’ve been trying to think of other industries that don’t have career growth exist exclusively by moving into management. Actors don’t move into directing or producing or casting because they feel they need to progress their careers. Many will never make the move, and those that do transition because they are interested in making the change. As far as I can tell while most coaches were athletes in the sport they coach, they generally weren’t great performers at their sport (and many star players were horrible coaches). Doctors, lawyers, accountants, and academics seem to have opportunities for leadership but even as they advance to department head, partner, etc., they continue to do the technical work as opposed to strictly “management” tasks. In fact, continuing to perform technical work is considered critical in these professions to maintain the skills necessary to perform effectively.
With managers who were induced into the role and therefore avoid fully committing to their management responsibilities because their incentive is to still do technical work, it can produce a leadership and mentoring gap that harms the organization. It can also allow the organization’s nontechnical management to run roughshod over the team and negatively affect morale; a stronger manager would be protecting them from this type of issue. There is a recent post by Michael O. Church that covers some of this. While he has his own strong viewpoint on some agile things that I disagree with, his view on management is mostly on point with how I see things. The push towards management of your best and brightest robs your technical staff of its best assets because the allure of power and money can sometimes have people make decisions for reasons other than passion for the work. This issue is on my mind because I see symptoms of it wherever I go and it’s sad. People hopefully get into industries because they find it interesting and want to do the work, and it’s a shame to see them lose their passion because they are forced to choose between roles counter to that interest or career stagnation.