Propinquity

Propinquity can be described as the rather obvious idea that you interact more with people near you. Not just physical nearness – it also encompases mental nearness like shared beliefs, common experiences, or a shared ubiquitous language. This comes up in software development in a number of different ways. Propinquity is teams that are more efficient because they understand each other and the domain at a deeper level. It is a team that intimately understands each other and the domain. It has negative expressions too. You get software ecosystems that become insular and don’t integrate new ideas. It is even sexism and racism. It is an unconscious bias that influences the way you think about things.

Recognizing propinquity can help you overcome your negative biases, build stronger teams, create solutions to your hardest problems from outside the box. Team culture is propinquity – it can be good forming bonds between coworkers, it can be bad when the team looks for people like themselves and ends up rejecting the best candidate because they didn’t have a strong enough resemblance. This can also be seen as a kind of implicit bias; when you start to think of the discussion of diversity in the workplace in terms of propinquity, it starts to mean something different.

I recently heard an interesting discussion with Leslie Miley of Twitter on a podcast (starting about 12 minutes in) about how the lack of diversity at Twitter could be negatively impacting their ability to solve problems. His thesis is that if you hire everyone with very similar backgrounds that they will all look at the same problem the same way and come up with the same solutions. They would then miss the same things, and get stumped by the same problems. This is why you often get great solutions from people outside of the problem, because they are looking at the problem from a different perspective.

I don’t have much insight into how other professions interview people, but in software it seems like the interview is more to determine if you can do the job than if the applicant is the person you actually want to be working with. But the way we go about determining if you can do the job is to ask questions that you don’t truly need to be able to answer to do the job, but implicitly reflect schools of thought on how the job should be done. If you ask a question about red-black trees you are looking for someone who had a computer science education; you aren’t expecting anyone to build a red-black tree but you are using that as a proxy for having been exposed to certain ideas. If instead you ask questions about TDD or design patterns it says you are the kind of people who care about those ideas instead and have less of an interest in computer science fundamentals and optimizations.

I went on an interview previously where they asked lots of low-level questions. The job had nothing particular to do with those skills, but they felt that those skills were what differentiated good programmers from bad ones. Their understanding of what makes a good programmer is that you can twiddle bits and will micro-optimize solutions. The interviews we gave at a place I used to work were focused on design patterns, which  was the language everyone there shared. At that job we didn’t have the low-level optimization skills to find a particular optimization that would have been a big win for an ongoing performance problem.

This ties back into the post I had on evolution of the javascript ecosystem. The ecosystem has propinquity in terms of javascript language itself, but has diversity in that most of the developers came from different stacks with different backgrounds. This level of creativity is what can be achieved if you balance propinquity and diversity against each other well.

Propinquity looks like culture superficially, but it really is a double-edged sword: a team that intuitively understands each other, but blocks out other perspectives that could be valuable to solve problems better. The values espoused by your organization are all about creating a culture and a shared experience to build the positive aspects of propinquity, but the negative aspects can appear too if not kept in mind. The different aspects of what an engineer looks like can’t be too rigid, otherwise you reject great candidates for lack of a degree, or interview for markers that imply competence but aren’t competence themselves. Conversely diversity is more than ethnic diversity – it’s diversity of thought. You need people who think in design patterns and people who think functionally and people who buck the trends of what you are doing right now.

Ideally you could get people who can do all of those things. That’s obviously not a realistic way to do recruitment. You want people who fill in your weak areas, ideally creating something like this.

overlapping

It represents the whole organization and how each individual’s competencies overlap but are each valuable. Each individual is close enough to the others to have some propinquity, but the whole group embodies the diversity you would want to get different viewpoints on problems. Conceptually with smaller units you can also look at this as a radar chart, if you map out where your team is currently strong and where it is currently weak, you see where you need help and where you don’t need as much, but it’s still hard to find the right way to find that person when they look different, and may not be in your normal talent pipelines.

Even just considering these sorts of issues points you in the right direction. If you consider that the person you are looking for is different than those you have, you might be willing to ask different questions or judge the responses differently.

Advertisements

Fix vs Replace

I was thinking about when to fix a piece of software versus replace it as part of our normal software lifecycle, prompted by a discussion at work of a particular piece of software. The software in question works great, but nobody is confident that they are successful when changes do need to be made. Part of the lack of confidence is because changes are only made once or twice a year, so getting it set up and running locally and testing it is a chore, plus it turns into a complex process with many chances of failure. The other part is it runs on top of another library that nobody is really familiar with; the library isn’t used anywhere else in our stack, so nobody develops any familiarity with it. This particular piece of software is also critical to expanding our business, as it’s the primary way new client data is loaded into the system.

If we wanted to fix it, we could take the existing solid software and enhance it to make the setup and testing easier, or we could clean up the usage of the underlying library so it’s more intuitive. However, the decision was instead made to replace it with something totally new. I wasn’t involved in the decision, but became involved with the original piece of software after the fact to make a change while the replacement is still being developed. It seems like this software could be rehabilitated at first glance, but clearly someone else thought otherwise.

I know in general I’m biased towards fixing existing software. I’ve spent most of my career working on brownfield applications and building oddly shaped pegs to fit back into the oddly shaped holes of those applications. I think that I’ve done this because I enjoy it; building everything from scratch is almost too easy since there are so many fewer constraints involved. I get a different sort of satisfaction from it. I know I’m not the only one who has this particular tendency; the folks at Corgibytes are specializing in this sort of work. I’ve even been nostalgic for a codebase I’ve worked on – not the application but the codebase itself.

I feel like most organizations are biased towards replacing software because it lets you just say the entire thing is bad and try again instead of having to pick a particular thing to do or change. You don’t have to agree on what’s wrong with it, or get into details of what to change. This flexibility of scope leads to the quid pro quo rewrite where a piece of software is replaced with a new version that also contains a major new feature; this concept was introduced to me by Re-Engineering Legacy Software. Re-Engineering Legacy Software describes this as a bargaining tool to enable you to gain acceptance of a plan to do the rewrite, but I’ve always seen the business bring up the rewrite with the idea that they’re unhappy with the team’s ability to change a piece of software and this would clean up the underlying causes of the problems.

That’s the big problem: the current software has problems due to something. And unless you deal with whatever that “something” is, the new software probably is not going to be significantly better than the old. It may not have had the chance to become crufty yet, so it seems better when it’s new, but given a few years goes back to the same sorts of problems you had the last time. You need normal software processes that enable you to create and maintain quality software even as requirements change.

You need feedback into your initial processes of what caused the cruftiness to accumulate. This can be seen as a form of double-loop learning, where the feedback of what happened impacts how you see the world and thereby influences your decision-making process, not just the decisions themselves. If you are accumulating cruft because you put schedule pressure on the initial development resulting in a less modular design, the feedback to the decision-making process would be different than if you are accumulating cruft because the requirements changed radically. To make true long-term improvements, that’s the step you need to take, which sometimes might lead you to fix, and sometimes might lead you to replace.

Modern Agile

I heard some references to Modern Agile and went to investigate further. To me, it’s just a reimagining of values espoused in the Agile Manifesto in a response to how the original manifesto was misused. The four values of the modern agile movement are:

  • Make People Awesome
    • The “people”: here is those making, using, or otherwise impacted by software
  • Make Safety a Prerequisite
    • Mostly psychological safety, but includes physical safety too if that is a concern
  • Experiment and Learn Rapidly
    • Try to find ways to do whatever they do better
  • Deliver Value Continuously
    • This, to me, means the technical practices around continuous delivery. But could mean other things outside of the web application space.
    • This is also breaking work into smaller individually valuable pieces.

At first I wasn’t that enthusiastic about the idea, and it seemed to be splintering the agile community. Then I started thinking about the recent resignations at the Scrum Alliance, and consider that maybe the community was already splintering and this is just people writing down what had already happened. The people for whom orthodox scrum is agile had already disregarded “individuals and interactions over process and tools.” Both LeSS and SAFe give up on “responding to change over following a plan” to varying degrees.  

The “focus on people” is, to me, the only right way to build systems and products, but the diffusion of who you are supposed to be making “awesome” makes this focus less impactful. If you were trying to empower the makers to do awesome you would do things differently than a single-minded focus on the customer, and some of those things are mutually exclusive. Modern agile examples of focus on the customer from Amazon and Apple come with horror stories on the engineering side of the late nights and insane demands – can you really make all the parties named “awesome” at the same time?.

“Safety as a prerequisite” is the opposing force to the focus on the customer and the negatives it can cause. This can be protecting work-life balance and everything else that makes people able to engage with the work. Another aspect of this safety is an openness to discuss problems in a candid way. I don’t know if I’ve ever seen the sort of openness to admitting problems that this is designed to inspire. I’ve been places where small mistakes were admitted to openly, but it’s unclear what would happen with larger mistakes since as far as I know none happened while I was there. I don’t think I’ve ever seen the sort of transparency from above which would make this level of openness either which makes it difficult to reciprocate.

I think experimentation is the key insight of modern agile. The original manifesto implied that if external stimuli did not change, the team did not need to try to change. This implication resulted in a challenge-response feedback loop and sought a steady state where if things around you didn’t change much you didn’t change. Even then there are a lot of organizations where the right thing to do is whatever is specified by the scrum guide and anything else is wrong. This experimentation adds an additional focus on continuously trying to find a way to do better regardless of how well you are already doing.

I see “delivering value continuously” as a continuation of working software over comprehensive documentation. Working software, from the agile manifesto, is the expression of what is most valuable,” so you could say it described this concept in a narrower sense. The focus on “value” rather than “software” is helpful since sometimes software isn’t the right solution – for instance, if you have an internal API that works but nobody uses because they are having trouble with the API, activities other than writing more software would be more valuable.

If we as practitioners wanted to stop and reconsider how we go about doing work, this seems like a good place to start that conversation. I would have liked to have seen it as a more open discussion rather than appearing fully formed from the head of a consultancy. These ideas seem reasonable, but it’s not clear what the next step is in taking up this mantle. I would always try to be on a team that was doing all of these things, but I don’t know if this alone is enough to make a team that I want to work on.

Book Chat: The Senior Software Engineer

The Senior Software Engineer by David Bryant Copeland is a manual for being a Senior Engineer and covers all of the things nobody bothered to tell you about. For me, it’s all of those things that I put together from reading dozens of blog posts and watching those senior to me do the job. Most of the book was the description of what the role is and describes it in sufficient detail to help an engineer who has not yet achieved the role to target the experience to show they can do the job. It is the sort of description of the role I would have liked to have gotten from a manager early in my career.

Since I’ve been doing the job of a Senior Engineer for years now, most of it wasn’t particularly novel to me. However, I would have liked to read this before I made the transition to Senior Engineer. It differentiates between building a new feature, fixing bugs, and solving problems, which are what I would describe as the three main technical components of being a Senior Engineer. It also describes quality technical writing, working with others, making technical decisions, interviewing job applicants, and leading a team.

Interviewing is probably one of the most critical areas that you don’t get a lot of exposure to before becoming a Senior Engineer, and it’s pretty well covered here. Copeland lists four components to the technical interview (in order):

  1. Informal get-to-know-you conversation
  2. Homework assignment
  3. Technical phone screen
  4. Pair Program

It seems like the phone screen should go before the homework assignment, especially if the assignment won’t be the focus  during the phone screen. I actually interviewed at a place where the author was working some years ago and the interview was significantly different than what he described. There was no informal conversation, no homework assignment, and no pair programming. There was a phone screen and a white boarding session. I suppose he may not have had total control of the process there but it is interesting to see the difference between the ideal and the actualized.

If you are looking to become a Senior Engineer and get a grasp of all of the aspects of the job before trying it, this would be a good read for you. If you are a Senior Engineer I’d pass and try something else. It might also be good read if you are a manager and looking to find a way to articulate some of these ideas to your junior team members.