Book Chat: Creative Confidence

While this isn’t the normal sort of thing I would discuss here, I read this book recently and it meshed more into software development than I had expected. It’s about tools to be more creative, and how to be consistently creative. It asserts that while you can’t have an amazing insight every day you can move from a ‘when inspiration strikes’ cadence to being able to generate novel insights at regular intervals.

From the software development perspective, there are some interesting ideas around low fidelity prototyping. The idea is that if you can see more ideas more quickly you can find bits and pieces that work and stitch them into a complete solution. One example the book used was doing prototypes with a giant paper iPhone cut out and putting paper UI elements on it. This has obvious applications to front end development. However, it also has applications on the backend, where you could write out a couple of different interface definitions and try different use cases with them and see what works best.

There is also an interesting chapter about constructing creative teams. There is some of the normal tips for forming creative teams, such as having a diverse team in skills and backgrounds. But it also goes more into building real relationships so you feel more open to share and express ideas that you would otherwise keep to yourself; even if those ideas aren’t immediately valuable they could be fuel for further ideas. A lot of it is about creating a mental space for those who would not normally consider themselves creative to contribute. The physical space should be great for changing. Walls you can write on, furniture and walls that can move. Eliminate as many barriers to demonstrating or testing an idea as possible.

Overall it’s an interesting topic and interesting to read. Nobody would say they are creative enough, so anybody could find value in it.

Advertisements

Book Chat: Designing Data-Intensive Applications

I start most of these book chats by describing the book and end by describing the reader who it seems like it would benefit. This time I’m going to describe the reader first.

If you write software that interacts with another computer you should read this. To me, the title is somewhat misleading since it focuses on the data aspect, however the intensity comes from the size of the data and implies a distributed system. You also get an amazing survey of different database implementations as a side benefit. The only complaint that I have is that the first part of the book starts very much at the beginning of data-intensive systems (e.g.,  data locality and how it is organized on disk), which is important if you are building your own database system but isn’t as applicable to the average reader.

There are sections on a vast number of different topics. It covers both SQL-style ACID databases as well as BASE-style NOSQL databases. It even gets into things like graph databases that don’t fit neatly in either box. The authors cover topics as varied as locking and commit strategies, the levels of consistency available in a database and what they really mean, distributed consensus, replication, and streaming.

The majority of the text is written in a technology-agnostic way but it will reference specific implementations that demonstrate a concept. There is also a deep academic rooting with a well-referenced selection of footnotes to satisfy any further curiosities you may have on the topics. It seems like it should be fairly accessible as a whole text to a relative beginner since it introduces concepts in a way that doesn’t require much prior knowledge. I don’t think a beginner could jump into a chapter in the middle and be able to follow along, but given the complexity of the topic I don’t think that’s an unreasonable thing.

Even if you aren’t building a database, deeply understanding the tradeoffs of the database you are using will make your application more correct. The difficulty of testing into a lot of the concurrent failure scenarios makes understanding the system at a logical level the only way to attempt to handle all failure cases. I do think all software engineers working on the web would benefit from the material here. It won’t make your day-to-day much better but it will help keep you out of the really bad places where the system is intermittently failing.

Book Chat: Debugging Teams

Debugging Teams is the second edition of Team Geek, rewritten to apply to a more general audience but still fundamentally the same material. It has a variety of fun and interesting anecdotes from the authors’ time running teams at Apple and Google as well as organizing a major open source project. There is some pragmatic advice about how to work within an organization as well as lead a team. Most of the book is directed towards a team leader, not a leader of a department, as would be expected by the title.

I feel as though the rewrite doesn’t make it apply to a general audience sufficiently enough. I felt that the rewrite does get away from the idea of running a software team, but doesn’t fully bring it to the any team perspective. The rewrite does get to the level of a team that works like a software team where you can iteratively deliver value. You couldn’t use much of the advice  to run a lot of other professional work where a report/result of knowledge work is delivered then you move onto another project. Some of the specific advice would still apply in these scenarios but it doesn’t work in the general management sense.

Overall it’s a fun read, but didn’t hit the target they were aiming for. It is valuable advice for someone who is running a software-like team. It still focuses on the basic practices of what to do as a leader, but doesn’t go enough into the how for me. I know I don’t personally run a team because I don’t enjoy some of the required aspects of project management, but I do really appreciate the skills and abilities required to do it well.

Book Chat: Manage It!

Manage It! is an overview of modern project management techniques. Most of it was accessible to me as a person who has never formally run a project, but has been involved in many. The less accessible material was concentrated towards the end which I dutifully worked through thinking there might be more immediately relevant portions after. It heartily embraced three different practices: strong meeting facilitation, rolling wave planning, and avoiding schedule games.

The meeting facilitation advice started out fairly straightforward – have an agenda, stick to the topic at hand, and hold one-on-ones with the team. It then goes on to discuss some more radical advice, like don’t go to meetings that aren’t about solving problems, question why you had the meeting if it ends and nobody has any action items, and avoid serial status meetings. If your project has a problem, getting the relevant people into a room and coming out with a solution is a great way to break the impasse. Other sorts of meetings can impact the progress of a project, but to me that doesn’t make them immediately a bad idea. From the perspective of the project manager I can see that other sorts of long-term work or out of band activity can impact the potential of the project, but it seems necessary for the functioning of a healthy engineering organization. I agree that the lack of action items coming out of a meeting seems like a warning sign it wasn’t a good usage of time. If a decision was made generally one or more people would leave the room and do something because of it. Serial status meetings are more complex; if you are holding a meeting where people tell you about progress being made on initiatives where the others in the room aren’t involved or impacted, it may be a good use of your time but it’s a bad use of everyone else’s time. If you are being invited to meetings to provide status, the book advises to send status via email and skip the meeting, the idea being that if the organization doesn’t accept that behavior then it isn’t the place you should be. Daily standups are not impacted from this practice because they’re about the impediments, not just the status. Overall it seems like a good package of advice as to how to interact with meetings.

Rolling wave planning is an implementation of the idea that your plan will fail, but that the exercise of planning is valuable regardless. Your short term plan should be pretty solid but the further out, the more vague the plan gets. So, you don’t worry as much about the long term, and as you acquire more information you update the plan. This works both for changes from outside the project and things you learn from executing the project itself. The one experience I have had with explicit rolling wave planning did not go well, but I feel that was because we were trying to keep the near-term solid plan too far into the future and engaging in some devotion to the schedule.

The schedule games section was the part that felt the most real to me, it listed out 16 ways a project plan can go awry and ways to cope with each of them. I felt like I had seen the vast majority of the pitfalls out in the wild. This previous visibility involved me more in the rest of the material since I felt the authenticity of this compared to a lot of books which don’t give practical guidance on  how to get from poor practices to good ones. The section on “schedule chicken” felt particularly familiar to me after having been involved in a high stakes version many years ago. We even made progress in getting out of the situation using one of the techniques described.

I would recommend this book to someone who has already led a team or project for a little bit and is interested in doing more of it. If you’ve been doing that sort of work as your primary role it probably still has some bits of interest. If you haven’t run a team or project yet you may get something from it but I feel that for a lot of the information you need to have seen the problems in action some before you can appreciate the importance of  avoiding it.

Book Chat: Extreme Programming Explained

Extreme programming (XP) is an alternative software development methodology that would be described as an agile methodology. It’s a competitor to scrum, but more focused on the developer experience, less prescriptive of specific organizational practices, and more prescriptive of technical practices. I was familiar with the concepts of XP and recently picked up the second edition of Extreme Programming Explained. This new edition refined some of the technical practices about deployment since tools now exist for even more rapid deployment than what was initially conceived.

The build time practice is interesting, the idea being that a continuous integration build/test cycle should take ten minutes. While you could make the build faster than 10 minutes, keeping it a bit longer generates a decent mental break to allow someone to get a cup of coffee or get up and stretch. Whereas, if it’s slower than that, there is a tendency to move onto a different task and you can lose context on the old task and the new task. It matches with my experience; although I hadn’t been able to articulate the solution, I had seen the problem.

The overall methodology seems solid, however it doesn’t market itself to the whole business the way scrum does which seems to have impacted the adoption of the methodology as a whole. The practices suggested are all pretty straight forward:

  • colocate the team,
  • construct a team with all necessary skills on the team,
  • have visible progress locators,
  • work when you can really concentrate on it,
  • pair program,
  • user stories,
  • a weekly cycle,
  • a larger quarterly cycle,
  • slack,
  • the above build time practice,
  • continuous integration,
  • test first programming, and
  • incremental design.

Most modern software teams would be in favor of most, if not all, of these practices. Some of the practices are outside the control of the team and would need significant management support, but most are things the team can control.

I don’t think that the differences between this and other agile project management methodologies are that significant. The biggest difference with scrum I can see would be that scrum has fixed reflection periods whereas XP has continuous reflection with impromptu kaizen events. I think that this difference between XP and scrum would allow you to differentiate yourself from all of the scrum implementations that are out there but never finished. I don’t think that the book adds much to my understanding of software engineering, however it’s an excellent selection of software engineering practices. If you’re looking for a different perspective on agile methodologies this would be an interesting read.

Book Chat: Working Effectively With Unit Tests

Working Effectively With Unit Tests is a discussion not of when to unit test or how to unit test, but how to know when you’ve done it well. It works backwards from the idea that tests should be Descriptive And Meaningful Phrases(DAMP) as opposed to the traditional software pneumonic Don’t Repeat Yourself (DRY). By allowing some duplication in tests and focusing on the clear intention of what is to be accomplished you get tests that are easier to read and tests that are more focused on the object under test rather than the collaborators of the test.

The style being described forces out a lot of the elaborate mock setups common in most first attempts at unit testing. This is a definite good intention, however like most resources, I feel it comes short at describing a means to actually get rid of these sorts of problems in real applications, as opposed to toy applications in books and articles. The ideas it provides do work towards those ends admirably. To me, the ideas presented seem to drive towards a more functional style of programming; methods were getting more arguments which made the methods more flexible, and the objects they lived on were less prone to carrying around extraneous state. The book didn’t discuss this in functional programming terms, but sort of implied that was a goal around the edges.

Compared to some of the other books on unit testing I’ve read, this felt more concise, and it was definitely less focused on a specific framework for doing testing. It feels written for someone who has been doing unit testing for a while and has not been getting value from the activity, or has been having maintainability problems with tests. For those audiences it seems like it is a good perspective towards trying to get out of their problems. For people new to unit testing, it may be a little to broad in what you should do and not prescriptive enough.

Book Chat: Perspectives on Data Science for Software Engineering

Perspectives on Data Science for Software Engineering is a collection of short research papers on using the tools provided by data science to do research into software engineering. It isn’t about the concepts of data science for software engineers as I thought it would be when I initially picked it up. This difference had me put it down the first time I picked it up to read it, but when I came back around to it I found myself interested not in the data science aspect of it, but the software engineering research aspect.

While none of the individual papers was something I read and immediately knew how I could apply in my own practice, the overall package helped me feel positive for progress in software engineering. Outside of language design, it sometimes feels like most of the software engineering learning we’ve done going as far back as the 70’s and 80’s hasn’t been applied in practice. I think part of the difference is because the research is disconnected from the way software is built in the wild. The research is hyper-specific, (e.g., focusing on a particular kind of software in a single language) or defines problems but not solutions (e.g., the work on code quality metrics). The research isn’t wrong, but it’s missing a step about how to apply the work to what you’re doing.

The only piece in here that I saw and felt had an immediate connection to what I was doing was the piece on bug clustering. That showed that the more bugs a file had the more likely it was to have more bugs in future iterations. This seems like it may lend some credence to the idea of rewriting a piece of code that has quality problems to effectively blank the slate and start over again.

Overall the book was intellectually stimulating but has no real practical usage for what I do or what I feel would be the average software developer. If your role straddles the practical and academic worlds then this may have more value to you.

Book Chat: The Architecture of Open Source Applications Volume 2

The Architecture of Open Source Applications Volume 2 has writeups describing the internal structure and evolution of nearly two dozen different open source projects, ranging from tools to web servers to web services. This is different from volume one, which didn’t have any web service-like software, which is what I build day to day. It is interesting to see the differences between what I’m doing and how something like MediaWiki powers Wikipedia.

Since each section has a different author the book doesn’t have a consistent feel to it or even a consistent organization to the sections on each application. It does however give space to allow some sections to spend a lot of time discussing the past of the project to explain how it evolved to the current situation. If looked at from the perspective of a finished product some choices don’t make sense, but the space to explore the history shows that each individual choice was a reasonable response to the challenges being engaged with at the time. The history of MediaWiki is very important to the current architecture whereas something like SQLAlchemy(a Python ORM) has evolved more around how it adds new modules to enable different databases and their specific idiosyncrasies.

I found the lessons learned that are provided with some of the projects to be the best part of the book. They described the experience of working with codebases over the truly long term. Most codebases I work on are a couple of years old while most of these were over 10 years old as of the writing of the book, and are more than 15 years old now. Seeing an application evolve over longer time periods can truly help validate architectural decisions.

Overall I found it an interesting read, but it treads a fine line between giving you enough context on the application to understand the architecture, and giving you so much context that the majority of the section is on the “what” of the application. I felt that a lot of the chapters dealt too much with the “what” of the application. Some of the systems are also very niche things where it’s not clear how the architecture choices would be applicable to designing other things in the future, because nobody would really start a new application in the style. If you have an interest in any of the applications listed check out the site and see the section there, and buy a copy to support their endeavours if you find it interesting.

Book Chat: Learn You a Haskell for Great Good

Haskell was the white whale of functional programming in my mind, something that is the definitive form of functional programming but with such a steep learning curve that it put off all but the most determined students. I had been recommended Learn You a Haskell for Great Good a while ago but kept putting it off because of the intimidating nature of the material. I eventually had a big block of time where I was going to be home and didn’t have many responsibilities so I figured this would be a great opportunity to take a crack at it.

I sat down with it with an expectation that it would be mentally taxing like Functional Programming in Scala was, however having put in the work already reading that and Scala with Cats I was way ahead of the curve. While the Haskell syntax isn’t exactly friendly to beginners I understood most of the concepts; type classes, monads, monoids, comprehensions, recursion, higher order functions, etc. My overall expectation of the difficulty of the language was unfounded. Conceptually it works cleanly, however, coming from a C style language background the syntax is off putting. Added to the basic syntax issues most of the operators being used do give it an aura of inscrutability, especially being difficult to search as they are. I did find this PDF that named most of them which helped me look for additional resources about some of them.

The book explained some of the oddities around some of the stranger pieces of Haskell I had seen before. Specifically the monad type class not also being applicatives, it’s a historical quirk that monads were introduced first and they didn’t want to break backwards compatibility. The other fact that I had not fully appreciated Haskell dates from 1990 which excuses a lot of the decisions about things like function names with letters elided for brevity.

The other differentiating fact about the book is that it tries to bring some humor, rather than being a strictly dry treatment of the material. The humor made me feel a stronger connection with the author and material. A stupid pun as a section header worked for me and provided a little bit of mental break that helped me keep my overall focus while reading it.

Book Chat: Effective DevOps

Effective DevOps is about the culture of the DevOps movement. The technical practices that today coincide with DevOps are the result of the culture practices, not the cause. The cause is an underlying culture that is safe, and respectful to those in it, which truly empowers the team to try things to improve the way that work is done and leads to the technical practices associated with DevOps. The book is overall written more from a management perspective than an individual contributor perspective. The book is centered around the four pillars of effective DevOps: Collaboration, Affinity, Tools, and Scaling.

Collaboration is the normal sort of mentoring, and workflow information that would be familiar to most agile or lean practitioners. The Affinity pillar though builds on top of Collaboration with the idea that it takes time and work to forge a group of individuals into a team and explores the requirements to build those bonds. These two pillars lead i7nto the Scaling pillar nicely since, while you can eliminate waste and automate things, at the end of the day the biggest scaling maneuver is in hiring. Hiring renews the importance of the Collaboration and Affinity aspects of this since as you bring new people into the system you must fully integrate them.

The section on the Tools pillar is written in a tool agnostic fashion, wherein it describes categories of tools commonly used to the DevOps. That makes it much more interesting than any other book that is tied to a particular set of technologies since it is focused on the concept not the implementation.

Overall it’s an interesting read. The focus on the social aspects of what’s going on makes it less useful in my day to day activities, but the longer I do this job the more I think that the technical aspect is essentially table stakes to doing the job and everything else is where more long term growth come from.