Scala Varargs and Partial Functions

I ran into a piece of code recently that looked like

foo(bar,
   {case item:AType => …}
   {case item:AnotherType => …}
{case item:YetAnotherType => …}
// 10 more cases removed for simplicity
)

I was immediately intrigued because that was a very odd construction and I was confused why someone would write a function accepting this many different partial functions and what they were up to. I went to look at the signature and found the below.

def foo(bar: ADTRoot, conditions: PartialFunction[ADTRoot, Map[String, Any]]*):  Map[String, Any]

It was using the partial functions to pick items from the algebraic data type (ADT) and merge them into the map. More interestingly it used the the ability of the partial function to identify if it can operate on the type that bar happened to be. Overall it was interesting combination of language features to create a unique solution.

Part of is that the ADT was missing some abstractions that should have been there to make this sort of work easier, but even then we would have had three cases not a dozen. I’m not sure if this pattern is a generalizable solution or even desirable if it is, but it got me thinking about creative ways to combine language features provided by Scala.

Advertisements

Book Chat: Functional Programming in Scala

I had been meaning to get a copy of this for a while, then I saw one of the authors, Rúnar Bjarnason, at NEScala 2017 giving a talk on adjunctions. Before seeing this talk I had been trying to wrap my head around a lot of the Category Theory underpinning functional programming, and I thought I had been making progress. Seeing the talk made me recognize two facts. First, there was a long way for me togo. Second, there were a lot of other people who also only sort of got it and were all there working at understanding the material. At the associated unconference he gave a second talk which was much more accessible than the linked one. Sadly there is no recording, but I started to really feel like I got it. Talking with some of the other attendees at the conference they all talked about Functional Programming in Scala in an awe inspiring tone about how it helped them really get functional programming, and the associated category theory.

The book is accessible to someone with minimal background in this, so I came in a somewhat overqualified for the first part but settled in nicely for the remaining three parts. It’s not a textbook, but it does come with a variety of exercises and an associated repo with stubs for the questions and answers to the exercises. There is also a companion pdf with chapter notes and hints about how to approach some of the exercises that can help you get moving in the right direction if stuck.

Doing all of the exercises while reading the book is time consuming. Sometimes I would go read about a half a page and do the associated exercises and spend more than an hour at it. The entire exercise was mentally stimulating regardless of the time I committed to the exercise, but it was draining. Some of the exercises were even converted to have a web-based format that is more like unit testing at Scala Exercises.

I made sure I finished the book before going back to NEScala this year. Rúnar was there again, and gave more or less the same category theory talk as the year before, but this time around I got most of what was going on in the first half of the talk. In fact, I was so pleased with myself, that I missed a key point in the middle when I realized how much of the talk I was successfully following. I ended up talking with one of the organizers who indicated he encouraged Runar to give this same talk every year since it is so helpful to get everyone an understanding of the theoretical underpinnings of why all this works.

This book finally got me to understand the underlying ideas of how this works as I built the infrastructure for principled functional programming. It leaned into the complexity and worked through it whereas other books (like Functional Programming in Java) tried to avoid the complexity and focus on the what not the why. This was the single best thing I did to learn this information.

NEScala 2018

I attended NEScala 2018 recently for the second time and wanted to discuss the experiences I had there. It’s three loosely affiliated conferences across three different days. The first day was an unconference, the second was NEScala proper, and the third day was a Typelevel summit. I saw a bunch of great talks and met other practitioners who all brought a different perspective to the same sorts of problems I work with every day, as well as some people who have radically different problems.

There were a pair of talks from presenters at Twitter on how they deal with their monorepo using Scalafix and Pants. These were interesting solutions to the problems of the monorepo. During the transition to the microservices at my current job the code base has shattered into hundreds of repositories, which comes with problems, and you sometimes look and wonder if doing this another way would solve those problems. This was a clear reminder that there are problems on the other side that are just as difficult – just different.

The sections on Http4s(talk) and sttp were especially interesting to see the way they tackled HTTP servers and clients as purely functional structures. HTTP was in my mind difficult to describe purely functionally because it is all about referentially untransparent actions. Sttp was especially interesting because we had built a similar abstraction at work in the last year and seeing how others made different tradeoffs was interesting.

The big takeaway for me was that functional programming purity is a pragmatic thing. Functional programming is a tool to tackle complexity in software, but it’s not the only tool available to do that. There are ways to use local effects to wrap small bits of imperative code, but outside of the function where the imperative code lives, none of the callers can tell. You have a thin imperative wrapper on the outside and possibly little imperative nuggets on the inside that resolve performance issues and occasionally improve algorithmic readability, but the whole program retains the composability and readability of an immutable program.

Burnout

Burnout is a common topic in our industry. I’m thinking about it right now because I got burnt out fairly badly recently and as a result stopped blogging for a while. It broke the commitment device I had formed by posting weekly. I think I’ve recovered and want to take this opportunity to discuss what happened and how I think I could have avoided burning myself out.

My team got split in two, some people left to form a new team with a new mission and some stayed in the existing team to continue the existing work. This meant that there was roughly the same work to do and fewer hands to do it. Also when this happened we started reporting up to a different executive. All of this change together was a bit of a shock to the team; our overall output took a hit from the loss of people, but our productivity stayed good. We lost our leader and most of the other senior engineers. The other remaining senior engineer moved to management, like he had been hoping to.

This left us with myself and two junior engineers to commit code day to day. It was a slow process, we got an experienced front end engineer relatively quickly to complement my skill set. Overall the majority of the work we had to do was on the backend and the new management was putting the pressure on schedule-wise. We also had an influx of QA automation resources to the team, which we sorely needed to build out our suite of API and UI automation. This build out of the test suite did, however bring to light a number of edge cases in the API that hadn’t been accounted for, which needed to be cleaned up. I felt this influx of bugs and the schedule pressure as a weight mostly on myself. I tried to take on too much, and let my newly promoted boss try to handle the new executive.

Retrospectively, I should have pushed back sooner and taken a more active role in dealing with the new executive. It’s not that my new boss was doing poorly, he was definitely doing better than I had the first time I was put into that situation. It was just that being thrust into that situation of the first time isn’t easy for anyone.

I ended up reading The Truth About Burnout to try and get a better grip on what was happening to me. It suggested that the path forward was to take more direct control in what is happening, essentially that the cause of burnout was a lack of control, not the situation itself. This is an interesting idea, but in the situations where I have experienced burnout it wasn’t a lack of attainable control, it was the lack of any mechanism to take control and fix the situation that did the most damage.

It’s a weird sort of mental knot, the lack of being able to fix the problem is the real problem not the initial problem itself. On one hand it feels like victim blaming – you are unhappy because you aren’t fixing your own problem. On the other hand it’s a much more powerful statement about what you can do.