Yesterday I mentioned that I was starting a new project that involved initializing a new repository/service using react. I basically spent the whole day on that today, but I want to kind of zoom in on about two hours this afternoon because it surfaced what I think is an interesting and common dilemma in programming, and that's large scale organization of code.

First, some back story. We've long had a service-oriented architecture at our company. We run a frontend node service and a Java API. Within that paradigm, code has roughly been organized by area of concern or by entity, if you will. Except for the frontend server which was one large code base. We've been working in that large (and ever-growing) code base for probably 6 and 1/2 years or more at this point, and so the cruft and messiness have accumulated. We've recently started migrating code out of that code base and organized it into other repositories/services, not necessarily by concern or entity, but by focus. So we have a frontend server for marketing heavy pages with primarily static content (things like pricing and product descriptions), and we recently wrote a second frontend server for speed and SEO critical pages. This project I'm working on now is the third piece to this, a third frontend server to function essentially as a member portal. These are non-SEO critical pages and while page speed always matters to a degree, it's less of a focus here. Which is part of why we're comfortable using a framework like React. Not that React is big as far as frameworks go, but the other two frontend servers don't use a framework at all, and we serve very little javascript for dynamic content.

There are certainly some benefits to this kind of organization. The complexity within each server is significantly less because the focus of each is clearly defined. They are free to use vastly different technologies without stepping on each other. They're smaller and more streamlined. And obviously, there's a certain separation of concerns working. But through starting the setup process of this new server and having conversations with a coworker, there are definite disadvantages to this structure as well.

All three of the servers are written using the same server-side framework - fastify - and templating engine - squirrelly. They share a lot of common request code (plugins in fastify, which are similar to express middleware if that's your area of expertise), and the templates include many of the same (or very similar) partials. For example, on every request to all three stacks, we fetch A/B configuration from s3 and randomly assign traffic to test buckets, we assign and calculate some visitor specific information, we authenticate the logged in member with our API, we look up geolocation information. And for every page render in all three stack, we load the same scripts for interacting with the dom and for setting cookies, we generate google tag manager data and track pageviews, we wrap the page content in a common header and footer, and we include a common search bar.

Can you see the problem? These services all want to be one service. Because there's not really any good way to account for their similarity. You basically have 3 choices, none of which are great.

Choice 1

Duplicate everything in all three places. This is what we did with the first two servers. But that means every time you want to change something, you have to change it in multiple places. Like, say marketing suddenly wants to test a different navbar color. You've got to set that test up three separate times. Code duplication isn't always avoidable, and I don't necessarily think it has to be a code smell, but here we're talking about a lot of very similar code.

Choice 2

Write very advanced tooling that wraps all three servers. You need tools for starting the servers, running the tests, building (and probably copying around) generated assets (like minified javascript bundles or purged css). That doesn't help you, however, with changing things in multiple places. I just wouldn't trust a find and replace across multiple repositories. And there's certainly nothing that I know of for replicating changes in one repository into another. You still have to go manually setup that navbar test three separate times.

Choice 3

Abstract the common elements into a separate, installable repo. In that way, you can change things in only one place (though you still have to then install the new version in all the servers). But common elements never seem to be common enough for this approach, and you always end up engineering hacks to make something work one way here and a different way there. Plus there's just something gross about sharing html templates across servers.

I spent about two hours in analysis paralysis between these approaches, but ultimately moved forward with choice 3. But you know what? I'm not particularly excited about it. I'm not convinced there was a right choice here, but so far, the one I chose feels wrong. Choice 1 is the most immediately flexible (albeit probably the most painful long term). Choice 2, while tempting because I enjoy orchestrating build flows and constructing development tooling, in my experience will take too long and be too difficult to maintain (I mean, they're all kinda difficult to maintain, but maintaining messy code is a different kind of difficult than maintaining custom tooling). Choice 3 feels, so far, like forcing a square peg in a round hole. Templates, like a 404 page for example, just aren't meant to be abstracted into separate modules and installed.

I don't know. There's no right answer. Chances are, tomorrow I'll actually undo some or all of what I did. As much as I hate duplicating such similar code, I feel like choice 1 may be the most sustainable. Certainly it is the simplest. BUT, maybe it would have been simpler if we'd kept these as one server in the first place. There are ways to keep code separated within a repository such that the complexity doesn't mount as the codebase grows. And focuses like pagespeed don't necessarily have to be comprised on one page just because you're using a framework and a SPA on another. Separation of concerns can happen within a codebase just as they can across different code bases.