Yesterday I mentioned that I was starting a new project that involved initializing a new repository/service using react. I basically spent the whole day on that today, but I want to kind of zoom in on about two hours this afternoon because it surfaced what I think is an interesting and common dilemma in programming, and that's large scale organization of code.
There are certainly some benefits to this kind of organization. The complexity within each server is significantly less because the focus of each is clearly defined. They are free to use vastly different technologies without stepping on each other. They're smaller and more streamlined. And obviously, there's a certain separation of concerns working. But through starting the setup process of this new server and having conversations with a coworker, there are definite disadvantages to this structure as well.
All three of the servers are written using the same server-side framework - fastify - and templating engine - squirrelly. They share a lot of common request code (plugins in fastify, which are similar to express middleware if that's your area of expertise), and the templates include many of the same (or very similar) partials. For example, on every request to all three stacks, we fetch A/B configuration from s3 and randomly assign traffic to test buckets, we assign and calculate some visitor specific information, we authenticate the logged in member with our API, we look up geolocation information. And for every page render in all three stack, we load the same scripts for interacting with the dom and for setting cookies, we generate google tag manager data and track pageviews, we wrap the page content in a common header and footer, and we include a common search bar.
Can you see the problem? These services all want to be one service. Because there's not really any good way to account for their similarity. You basically have 3 choices, none of which are great.
Duplicate everything in all three places. This is what we did with the first two servers. But that means every time you want to change something, you have to change it in multiple places. Like, say marketing suddenly wants to test a different navbar color. You've got to set that test up three separate times. Code duplication isn't always avoidable, and I don't necessarily think it has to be a code smell, but here we're talking about a lot of very similar code.
Abstract the common elements into a separate, installable repo. In that way, you can change things in only one place (though you still have to then install the new version in all the servers). But common elements never seem to be common enough for this approach, and you always end up engineering hacks to make something work one way here and a different way there. Plus there's just something gross about sharing html templates across servers.
I spent about two hours in analysis paralysis between these approaches, but ultimately moved forward with choice 3. But you know what? I'm not particularly excited about it. I'm not convinced there was a right choice here, but so far, the one I chose feels wrong. Choice 1 is the most immediately flexible (albeit probably the most painful long term). Choice 2, while tempting because I enjoy orchestrating build flows and constructing development tooling, in my experience will take too long and be too difficult to maintain (I mean, they're all kinda difficult to maintain, but maintaining messy code is a different kind of difficult than maintaining custom tooling). Choice 3 feels, so far, like forcing a square peg in a round hole. Templates, like a 404 page for example, just aren't meant to be abstracted into separate modules and installed.
I don't know. There's no right answer. Chances are, tomorrow I'll actually undo some or all of what I did. As much as I hate duplicating such similar code, I feel like choice 1 may be the most sustainable. Certainly it is the simplest. BUT, maybe it would have been simpler if we'd kept these as one server in the first place. There are ways to keep code separated within a repository such that the complexity doesn't mount as the codebase grows. And focuses like pagespeed don't necessarily have to be comprised on one page just because you're using a framework and a SPA on another. Separation of concerns can happen within a codebase just as they can across different code bases.