I’ve never been much for setting goals in my career. I’ve always taken problems and solved them, but never had much of a long term plan for where I wanted to go. I’ve always been interested in the architecture aspect of software engineering. But that always felt to me more like a role, and while you can have a goals to be involved in that role it isn’t a terribly productive goal in of itself.
If a goal is a SMART style goal where it is something you can actively work towards, then you can just do it and know that you’ve achieved it. But something like being “more involved” in the architecture aspect of a project isn’t something you just do on your own, or necessarily know when you’ve achieved it. To be measurable and achievable, that sort of abstract goal needs to be broken down into smaller portions that can be achieved independently. It always seemed to me that the more interesting sorts of long term goals of this nature are difficult to break down into individual aspects.
To move myself towards that goal I’ve done most of the architecture tasks for whatever team I’m on. I’ve also been actively volunteering to help other teams with their architecture problems – just as a second set of eyes. None of this really ever got me progress towards the goal of being able to work on architecture full time. I had spent a lot of time trying to come up with other specific actions to try to advance me towards this goal.
Today, I had a realization that maybe I was looking at the wrong goal. I don’t want to do architecture most of the time, because I don’t want to be in an organization that has a role where someone does that for others. I want to have the ability to pick and choose my own projects, and even define my own projects to solve problems others may not have even recognized as problems yet.
This got me to thinking what sort of intermediate goals I should be working towards for this larger goal. I haven’t been able to come up with any of the intermediate goals/actions thus far. Some ideas I’ve considered so far and rejected: being superiorly productive, being vocal about any issues I see that need attention, and just doing what I think is right regardless of what’s on the team’s agenda. Being superiorly productive was rejected because it would mean that I need to stop spending as much time helping others with their problems and therefore collectively putting out a less solid product. The issue I had with being more vocal was that I would inevitably be bringing up problems that I didn’t have a solution to or did not have time to do anything about, which feels like it would just seem like complaining. Just doing the right thing and ignoring my team’s mission is difficult since it puts me and my team in a complex situation in that we aren’t doing our specific part of how the organization is supposed to work, but if we solve another problem it might be okay, it also requires the ability to go your own way long enough to have results to show.
I’m still trying to work through the ramifications of this personal epiphany, so I haven’t taken any real action on it yet. I don’t know if I will be able to figure out a new course of action based on this, but I hope to.
I previously mentioned the new service we were spinning up and the discussion of the overhead therein. Having finished getting the initial version of the service out into production, I feel like I have some answers now. The overhead wasn’t that bad, but could have been lower.
The repo was easy as expected. The tool for setting up the CI jobs was quite helpful, although we didn’t know about a lot of the configuration options available to us. We initially configured with the options we were familiar with, but found ourselves going back to make a couple of tweaks to the initial configuration. The code generators worked out great and saved a ton of time to get started.
The environment configuration didn’t work out as well as expected. The idea was that the new service would pick up defaults for essentially all of its needed configuration, which would reduce the time we would need to spend figuring it out ourselves. This worked out reasonably well in the development environment. In the integration environment we ran into some problems because the default configuration was missing some required elements. This resulted in us not having any port mappings set up so nothing could talk to our container. We burned a couple of hours on sorting out this problem. But when we went to the preproduction environment we again found its port mapping settings were different from the lower environments and needed to be setup differently. Here we ended up burning even more time since the service isn’t exposed externally and we needed to figure out how to troubleshoot the problem differently.
In the end I still think spinning up the new services on this short timeframe was the right thing to do – we would have had to learn this stuff eventually when building a new service. Doing it all on the tight timeline was unfortunate but the idea of getting the services factored right is the best thing.
As I had alluded to previously, the project I’m currently on has some very aggressive and fixed deadlines. The project got scoped down as much as possible but there have still been some trade-offs for productivity over maintainability. Some of it happened as a team decision – we discussed ways to compress the schedule and found some shortcuts to produce the needed functionality faster. But as new requirements came up in the process those shortcuts ended up causing much more work overall.
We knew that the shortcuts would cause more total work but we thought that the additional work could be deferred until after the initial deadline. The change of requirements meant that some of the automated testing that had been deferred was an issue. The initial manual testing plus the retest after the changes probably wasn’t much less than the effort to do the automated testing the first time. This specific shortcut wasn’t responsible for much of the time we shaved off the schedule, but when you’re doing similar things all over, the total cut was significant.
The key part of this tradeoff is that it assumed we would go back after the initial deadline and fill in the bits like we intended to instead of moving to something else (which is appearing more and more likely). Product management has a request for an additional feature in another existing application. This request has a similarly tight deadline, but is much smaller in scope (15 points) than the initial request(~120 points for the scoped down version). Picking up this request would mean that we end up further from the decision to defer portions of this and we’d be less likely to be able to deal with the technical debt in a timely manner that we accrued to the do the first request.
Since I’m still new at this job I’m not sure if this behavior is regular or since both of these items are related to the same event if it’s just a one-time thing. If this is a one-off problem then we can survive more or less regardless of how well we deal with it. If this is going to be a regular occurrence without an opportunity to resolve the problem, I feel like I’d need to take a different tactic in the future about how to work with the problem of short term needs.
This feels like the overall challenge of engineering: there are problems that need to be solved in a timeboxed way. We need to make good enough solutions that fit all of the parameters, be they time or cost. We can be productive and leave behind a disaster in our wake but succeed at business objectives, or we can build an amazing throne to software architecture but have the project fail for business reasons. The balance between the two is the where this gets really hard. If you’ve already been cutting corners day to day and the request to push the needle gets made, you have no resources left. The lack of an ability to really quantify the technical debt of an application in a systematic way makes it hard to project where we are on the continuum to others who aren’t working on the system daily.
Without this information, program-level decisions are hard to make and you end up with awkward mandates from the top that aren’t based on the realistic situation on the ground. Schedule pressure causes technical debt, testing coverage mandates cause tests that are brittle to just ratchet up the coverage, mandating technology and architectural decisions results in applications that just don’t work right.
This idea came together from two different sources. First, I’ve been reading The Fifth Discipline about the creation of learning organizations. One of the elements of becoming a learning organization is Systems Thinking, which I had heard of before and seems like a great idea. Then I went to a local meetup about applying Systems Thinking to software development. We ran through a series of systems diagrams from Scaling Lean & Agile Development. These diagrams helped us to both understand Systems Thinking by analyzing a domain we were already familiar with in a structured way where we were presented the nodes of the system diagram and asked to draw connections between them.
There were several different groups each looking at the same sets of nodes and drawing different conclusions for some of the relationships. There were a few very adamant people who all felt that an increase of incentives for the developers would increase the velocity of the team, which was interesting to see since they were mostly scrum masters or software development managers with many years of experience doing the management of software projects and should have seen how that does work out. If the impact of an incentive plan on a team is significantly uncertain, it makes me curious about the sorts of teams these other leaders are running.
Through the entire process we had interesting conversations about how various aspects of the software process were related. Everyone agreed that the interaction of a strong mentoring process on the overall system would decrease the number of low skills developers on the team. There was a discussion of whether it would also directly impact the velocity of the team. Some people were adamant that it would lower velocity since the mentoring time was time that people weren’t working on building features. It’s a reasonable consideration, but it doesn’t seem to match with my specific experiences with mentoring. If you were actively spending enough time to lower your velocity significantly on mentoring activities you would be forcing the mentoring relationship instead of letting it happen organically.
The experience made me wonder if we could construct a system diagram that describes some of the other dysfunctions of many organizations. The diagram we built described (1) dysfunctions around hiring large numbers of low skill developers, (2) using certain kinds of financial incentives, and (3) the push for delivering features above all else. It didn’t describe why a lot of organizations end up in siloed configurations, “not invented here” behavior on technical systems, or lots of the other dysfunctions in multiple organizations. I made the below diagram with the intent of describing the situation about siloed organizations but without any real experience in this sort of analysis I’m not sure if I’m doing it well.
I made another simple diagram about the causes of “not invented here.”
It feels to me like these diagrams describe the dysfunctions that are common in software organizations. Expanding these diagrams to try and find the leverage points in the system might yield some greater insights into the problems. In spending some time thinking about both of the diagrams I’m not sure what other nodes should be there to further describe the problems.
I’m going to definitely do some more reading on Systems Thinking and try to expand on the thoughts behind these diagrams. If you’ve got more experience with System Thinking I’d love to hear some feedback on these charts.