Engineering Initiatives

At work we have engineering initiatives that are technical projects that cross multiple teams. They include things like upgrading to a newer version of a framework across the company or building out the infrastructure for on-demand test environments. These initiatives all seem to share a common pattern. Engineering leadership decrees it should be done and then each of the individual teams add the relevant tasks to their backlog. Product then proceeds to prioritize everything above these tasks for each team. Then Engineering leadership starts asking questions of why nothing got done and begins putting pressure on individual engineering managers to get stuff done.

At this point different teams seem to react in one of two ways. They either find reasons they can’t finish the work and hence don’t want to start the work, or they try to negotiate with product on the priority of the work. None of this work was accounted for on the product roadmaps so negotiation is tough because it means bumping work that was promised to other parties. I know when my team this year was putting together its roadmap [1] I made sure that there were two good sized blocks of time for ‘Things people don’t know they need yet.’ This is the team’s way to try to add visibility for work coming in from other sources. I know one of the other teams essentially schedules 9 months of work for the year-long roadmap as a strategy to be sure they have capacity for whatever comes up from other sources. I like the solution my team uses since we can inject the two blocks into the schedule specifically so the work is better spread out across the year.

A lot of organizations need to tackle systematic architectural improvements and the engineering initiative practice is a way to structure that work. I would like to see the engineering initiatives being better molded into the high level product roadmap. Individual product owners understand the importance of the initiatives, but there isn’t anyone on that side driving it and it really comes down to each individual engineering manager or team negotiating with the product owner about the priority of these efforts and that can result in them not getting the priority they really need if the initiative becomes security critical.

[1] Given the annual nature of the business we’re in an annual planning cycle makes more sense than more “by the book” scrum style backlog management. We still do individual sprint commits but being at the roadmap for when your team’s annual cycle comes due is important.

Advertisements

Book Chat: The Master Algorithm

The Master Algorithm is a description of the pros and cons of different machine learning techniques and the author’s quest to unify them into a single algorithm that can tackle any kind of problem. It has sections on five major kinds of learning algorithms: nearest neighbor, naive bayes, decision trees, support vector machines (SVM), and neural networks. It then covers Alchemy, the author’s attempt to unify multiple disparate styles of learning algorithms into a single overarching implementation. Overall the book succeeds at a popular science level of description of the current status of machine learning techniques, but it didn’t satisfy my needs as someone closer to the software.

The author declared an intention to keep the amount of math to a minimum. It ended up that the author tried to describe mathematical concepts in prose, and that didn’t work as well for me as just using some formulas probably would have. I wanted either enough math that I felt that I understood what was going on fully or no math at all so I didn’t feel like I had a mediocre partial understanding.

The descriptions of the individual learning algorithms made sense with a surface read. Once I stopped to try and really understand the differences it was less clear, I think the lack of hard formulas impeded my understanding. There was a section that described a unification of several different algorithms, but turned out to be a metaphor for early attempts to unify learning algorithms where the math never worked. The inclusion of this effort and the way it was described ended up being confusing to me.

The discussion of the real, but incomplete, unification scheme in Alchemy was interesting. The implication that you would still need an advanced degree in machine learning to use it says to me there is more to do. If machine learning is truly going to change the world the means of training models needs to be opened up to at least the average software engineer, if not business users.

I feel like the author’s particular perspective put him too close to the problem to really write a more popular science style book. I think he could have written the book I was looking for with more math and a more practical software aspect. From a technical prerequisite perspective this could be an excellent text for people who are trying to come at machine learning from a non-technical background.

Book Chat: Becoming A Technical Leader

I had added this to my list of books to read a while back, then after seeing a copy of it in the office getting passed from person to person it jumped to the top. I can’t think of a specific anecdote from the book I think you would need to know, but I do think if you are involved in making software you should read it. It presents a sort of zen of software management where you ensure that those doing the work have the space and resources to solve the problem. It’s expressed in a lot of short chapters that mostly center around anecdotes from the author’s career as a consultant.

It talks about how to help spur innovation and motivate others. There are ideas that I had seen in practice from my manager about how to craft and express a vision. I specifically remembered the discussion of the vision since it was really effective and highly motivating. The template for the discussion came straight from the book. Initially reading the template I felt skeptical, I’m not sure if it was because I didn’t remember the usage of the template and it was tickling the back of my mind, or because it doesn’t seem good on paper. Having seen the results of it I am convinced.

There was a chapter about power conversion which seems to me the central idea. If you have power in one realm you can use it to express power in another realm by figuring out how to leverage your strengths. This seemed like the one specific point that I personally need to learn more about. I haven’t figured out how to do it yet, but I keep circling back to this particular idea as something I need to figure out.

It strikes me as a lot of Theory Y management, although it never says that. Since this is about leadership the act not management the title, it is applicable to both engineers who want to stay on the technical track and those who are interested in the management side as well. If you’ve got a strong grasp on the technical portion of the job and want to better understand how to expand your influence check this out.

Strike Teams Part 2

This is a follow up to the original strike team post I did a while back. I’m writing this as the strike team is wrapping up. It’s been an interesting experience. We hit most of our broad goals but did it in a significantly different way than anticipated.

The first big change came early on when we were looking into the proposed changes to an existing piece of software that was going to be a consumer of what we were building. We realized that accommodating it was probably weeks of work by itself. The initial assumption of how much work was needed had been minimal, but apparently that particular solution was untenable to the people who supported the consuming software in the long term. This ended up with them pulling support from the strike team and becoming uninvolved. At the time, the scope and resourcing change was challenging since it threw everything into an uncertain state. I think it ended up being a good thing; we were able to focus on building out the functionality desired without spending a lot of time making changes to one consumer of the service. It does make the follow up more complex since we do need to make sure that work gets done otherwise the whole system will end up in a more complex state.

There was one portion of what we were going to deliver that had a complex deployment step. There were other changes that had to go either before or after this particular piece so we tried to move that deployment as far forward on the project plan as possible. The intention was to deliver it at the end of the first sprint. We encountered some significant scope changes that all unfortunately all had to go before that step. This ended up pushing the actual deployment out until the middle of the third sprint. At this point we had about ten tickets that we did the work for all sitting in this feature branch just waiting for the eventual opportunity to merge the whole thing back to master with nearly 5000 lines of changes. We ended up performing a series of messy merges bringing changes from master back to the feature branch. The final merge from the feature branch back to master was ugly since we wanted to preserve some but not all of the history of that branch. In retrospect we should have seen the merge issues coming and been more proactive about doing more small merges, but lesson learned for later.

Every team member lost some time to dealing with issues brought to them from the team they were on loan from. We had anticipated this during the first sprint but didn’t expect it to continue through the remainder of the strike team. I know I personally got sucked into some production oddities that were occurring and that after the first sprint without me my team missed my presence in code review to the point that they got me back to reviewing basically everything. Both of the other members of the team got pulled back to the their original team for issues that happened. We didn’t fight any of these needs figuring that since they seemed like critical issues for those teams, having their people available was the best overall usage of time, even if it hurt our overall ability to get things done on the strike team.

The team as a whole never really jelled to any significant degree. Everyone kept working the way their team did, which created some minor conflicts. One engineer’s team has a very lax overall structure and he would sort of disappear for days then show up with large amounts of excellent code, but it was frustrating for him to essentially go off the grid, without forewarning. The other came from a team with experience in a different technical stack and vacillated between asking not enough questions and spending too much time stuck and asking too many questions. The three of us were also in three different locations which made it difficult to really all get on the same page and get working together.

Overall the strike team model worked for this, since having representatives from some of the teams that were going to consume this service working on it made sure that we didn’t run into any issues with a misunderstanding of the domain. There were problems setting up a new team, that we should have attacked proactively since setting up any new team needs to initially get organized and form their own norms. The transient nature of the strike team prohibits a lot of identity building which, in my opinion is key to building a good team. Overall based on this experience I think that the strike team model can deal with software projects crossing multiple software team boundaries, but there may be other better ways out there to be found still.

Type Classes

I have been trying to understand type classes for a while and having a hard time figuring out exactly what they mean or how to effectively use them. The idea that they are a means to extend the behavior of a class in a consistent matter makes sense in an abstract way. But I’m not sure how exactly they accomplish that goal, or what a type class is or isn’t. A coworker had offhandedly referred to a type class as “a trait with a type parameter,” but it feels like there has to be more to it than that. This post is a sort of journal of my efforts to figure out exactly what a type class is.

The wikipedia page wasn’t that helpful to me since it defines type classes in terms of other unknown terms. It had one useful tidbit of information from my perspective: in Scala, type classes are implemented with implicits. Following some more links I ended up at the Cats documentation, which has a bunch of example type classes and code using them that I found useful. This got me thinking about whether I had seen anything that had a signature that looked like it might be using a type class. I remembered the sum method on List, which had stuck in my mind because it was unclear as to how it knew how to sum the items and what types it would be legal to sum.

This definitely looks like a type class given our definition so far. We have an implicit argument that is a trait with a type parameter. It is being used to extend numeric types to give them orderings and perform arithmetic operations, but it also signals that the type is numeric. The type class is also being used to constrain what types the sum method is available for, since if the implicit is not available it won’t compile. This constraint also plays nicely with the context bound syntax.

So we’ve got an example and some rules, but that’s not really a definition. I went looking for some more examples in the Cats codebase since that is full of type classes. Each of the individual type classes in Cats definitely follows the pattern of a trait with a type parameter. I think the missing piece for my understanding, lay in what the methods on the type class are. The methods all seem to take at least one argument of the type parameter so that appears to be a reasonable constraint on the functions.

Type classes are different than an implicit class since you can constrain type signatures with it, but they both let you add new functionality to an existing type. The implicit lookup progress imposes some constraints on the implicit class, such as only taking one non-implicit argument, which the type class methods can bypass. You could write an implicit class to expose the type class in a more fluent way like.

 implicit class NumericWrapper[T:Numeric](x: T) {
  def plus(b: T): T = {
    val n = implicitly[Numeric[T]]
    n.plus(x, b)
  }
}

Type classes seem like they would be more useful in a language without multiple inheritance. Since in Scala I can have a type implementing multiple traits that already have implementation associated with them, just mixing in more code seems like an easier way around the extension problems. I found this proposal for adding type classes to C#, which seems very cool and in line with the sorts of powerful abstractions they’ve been trying to add to the language. Seeing a different syntax for using type classes without implicits being involved helped me understand what they really are.

Going back to my initial goal of figuring out what a type class is and how it works, I think I’ve figured out both. It is a generic type that adds functionality to that type but defers the implementation of that functionality. Then you specify something that expects the type class and brings the implementation of the functionality and the data together. In Scala it accomplishes this using type bounds to specify the type class and implicit parameters to pass the type class into the implementation. I’m still not sure when I would want to write my own type class as opposed using other polymorphic concepts, but I’m now confident about using existing type classes and even breaking out some of the Cats-based ones as opposed to just some of the inbuilt ones.

Book Chat: The Psychology of Computer Programming

The Psychology of Computer Programming by Gerald Weinberg is describing the how and why of computer programming in the abstract. It covers topics like when and where to leave comments, how the choice of programming language influences the eventual program written, or how to go about hiring programmers. I read the silver anniversary edition which added some annotations about how events had changed between the original 1971 release and 1997 when the silver anniversary edition was written.

The book starts out with talking about reading programs with some example code written in PL/1. The example reads fine even if you know nothing about PL/1, it goes through several variants of the same program, dissecting the pros and cons of each implementation. While modern programming has mostly eschewed limits on memory usage and program size, similar pros and cons could be applied to things like GC pressure or context switching.

Each chapter closes with a series of introspective questions about the topic for programmers and  managers , mainly about how it could be applied to your day to day activities. After a chapter on programming as a social activity it asks the manager, “In setting your own working goals, what part is set by what is passed down from above and what part is set by what comes up from below? Are you satisfied with this arrangement, or would you like to alter it in some ways?” Whereas it asks the programmer “What part do you play in setting the goals of your team? What part would you like to play? What part would you like others to play?” These two sets of questions sort of suggest a  conversation between perspectives and helped me to understand the perspective of management better.

The section on time sharing systems vs batch systems did not age well since neither system is used anymore. It was still interesting to see a breakdown of the pros and cons of the two systems and how it impacts the culture of the workplace. It provided a case study of a company where they switched from a batch system to a time sharing system, which resulted in a breakdown of the informal communication system between the developers. Under the batch system they would congregate around the result return since when the results would be back wasn’t certain. Once they switched to a time sharing system everyone spent time in their office and there was little communication and teamwork among the programmers.

There was no single takeaway from this book where I would recommend that you should read this to achieve a particular end. Overall it was an interesting read from a conceptual perspective, but I don’t think it’s applicable to the average programmer. There is more value I think on the management side, but since that isn’t what I do it’s harder for me to judge.

Book Chat: 97 Things Every Software Architect Should Know

97 Things Every Software Architect Should Know is a selection of pithy articles with titles like “There Is No One-Size-Fits-All Solution” and “Communication is King; Clarity, and Leadership, Its Humble Servant.” To me they were all common sense ideas in the big picture. None of them were wrong, but none of them are anything anyone who has done the role well or is really ready to do the role shouldn’t be aware of already.

If you ended up thrust into a software architecture position without having a strong idea of how to do it check this out, it will give you some of the big picture stuff to check out. Otherwise just skip it.

Book Chat: Programming in Scala

I had previously mentioned Programming in Scala when discussing Scala for the Impatient, saying that Scala for the Impatient was written as a reaction to the ~800 page bulk of Programming in Scala for people who wanted just enough to get started with. After having read Programming in Scala it feels like criticism of its length is fair. The first half of the book was massive overkill for people who had experience in any C derived language or any other sort of object oriented language. There were sections that were marked off as optional reading  if you were familiar with Java because the behavior being described was similar. In comparison, the second half of the book was a wonderful experience even for an experienced programmer, since there were in depth explanations of all of the advanced language features.

Some of the things I learned were simple. For instance, that regular expressions can be used as extractors, which is a straightforward idea. Or, that predef is implicitly imported everywhere. Or that the arrow operator is actually defined as an implicit conversion in predef and not an explicit part of the language.

Other sections were more complex. The rules for how for expressions get mapped into other syntax were similar to what I had figured out, but the rule about how conditionals and assignments within the expression are evaluated added a lot of clarity to what I had learned by doing. I learned the ways you can use type bounds to improve variance indicators. The authors also discussed the transform method on futures that will be available in Scala 2.12, which has me excited to get to that upgrade.

There were some other things covered that even after a considerable study I’m not sure I understand. I understand the syntax for refinement types but I don’t think I understand the value even after the provided in-depth example using currencies. There was also an in-depth discussion of how the designers arrived at CanBuildFrom in the collections package. CanBuildFrom enables extraction of common operations from many collections but returns a collection of that same type and not some supertype. It makes sense in an abstract sense, but I don’t think I could implement a similar pattern without copying it directly out of the book.

Despite the book’s heft, there were a couple of topic I would have liked to know more about. I was hoping for a discussion of the reflection capabilities provided by manifests, type tags, and class tags, but since they are just library pieces and not integral to the language they weren’t covered. There were some oblique references to how bytecode gets generated from various Scala structures, but I was hoping for more insight into how to make interfaces that are less susceptible to breaking changes under the hood even when the Scala side looks fine.

Overall it’s a good read and not as long to read as you would think a book this size would be. It’s easily divided up into small sections so you can easily sit down and read a page or two and make progress over time.

Book Chat: How To Solve It

How To Solve It isn’t a programming book. It’s not exactly a math book either, but you will find yourself doing geometry while reading it. It isn’t a book on logic, but it is all about structured thought processes. I would describe it as a manual to teaching a systematic approach to problem solving to others, using geometry and a series of examples. It tries to lay out all of the thoughts that whiz through your head when you see a problem and understand how to solve it without really contemplating how you knew it. It’s a fast read, assuming that you know the geometry he uses in the examples.

The problem solving process is broken into four basic steps: understanding the problem, devising a plan, carrying out the plan, and looking back. At first it seems obvious, but that’s thing about a structured approach, you need to cover everything and be exhaustive about it. For example, to understand the problem you identify the unknown, identify the data, identify what you want to accomplish, try to draw a picture, introduce suitable notation, and figure out how to determine success. If you wanted to know should you buy milk at the store this sort of formal process is overkill, but if you are struggling with a more complex problem like trying to figure out what’s causing a memory leak or setting up a cache invalidation strategy it might be valuable to structure your thoughts.

I haven’t had a chance to apply it to a real problem yet. I did use some of the teaching suggestions – how to guide the pupil to solve their own problems – with one of the junior engineers I mentor and it seemed productive. I got him to answer his own question, however not enough time has passed to see if it improves his problem solving abilities in the future.

Overall the book was an interesting experience to read and seems practically applicable to the real world.

Monads for the Working Programmer

The monad is a backbone of functional programming. However, the mathematical definition of what a monad is highly inaccessible. I want to try and describe a monad for a programmer who is new to functional programming. This isn’t intended to be a precise definition, it is intended to be enough to help understand what the monad is, how to use it, and why it’s a helpful construct.

Consider the monad as a box. The box can be empty and the box doesn’t really care what’s in it. You can go get a box and put something in it. Someone can open up the box, take out what’s in there, work with it, then put their output back in the box. That’s really all there is to it.

Sometimes the box is less tangible than other boxes. For instance, Future in Scala or IO in Haskell don’t have an actual value yet, but the promise of a value later. Think of it like a package in the mail, the box may not be here yet but you can make plans for when it gets to you. You can plan to open the box and combine it with another value you have and put it back in the box. In programming terms it’s a continuation, you take the first box and attach some code to it so that when the value in the box does show up you run the continuation with it.

Sometime the box is a bit bigger than others, like the List monad. The List monad has more than one thing in the box like an egg carton, but they’re all of the same type. You can still open up the box and take out an egg and do something with it, or you can process each of the eggs in turn.

When you want to use a monad it’s as easy as that – your functions all start wrapping their return types in that monad. Their callers then open the box and do their work and return the new value to the box. So instead of pseudo code that looks like


values = [3, 4, 5]

for(int i =0; i<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>< values.length; i++) {

	values[i] = values[i] * 2;

}

 

 

You get pseudo code that looks like


values = [3, 4, 5]

doubled = values.map(value => value * 2)

But what’s the point of this programming style, what does it get you? First, it helps you work with immutable data. In the first example above you are mutating whatever values is. That’s fine in object oriented or procedural programming, but functional programming sacrificed mutable data for thread safe composability. Second, it produces code that is easily composable. If I wanted to subtract one before doubling the values in the above monad example, I could add another map statement entirely, separating the two concerns. In the initial example the two concerns would be mixed into the same control flow. Third, the monad itself can contain a fair bit of logic to enrich the programming experience. The ability to interact with a Future, an Option, a List, or any other monad in the same way simplifies a lot of refactorings. If you want to replace an Option with a List application logic doesn’t need to change much. Hopefully all of this helps demystify the Monad and helps you to write better code.