The first version of our DNS filtering service was written in Node.js. The version that stayed was a custom CoreDNS plugin written in Go. It would be easy to retell that as a poor technology choice followed by the correct one. That is not what happened.
Node.js let us change the product quickly while we were still learning what it needed to do. The mistake came later: we kept optimising after the evidence had started pointing below the part of the system we controlled.
This was one bounded service inside LucidView's larger internet-management platform, not a rewrite of the whole product. The platform combines DNS and firewall enforcement, customer-specific policies, reporting, and network integration. The LucidView product overview describes how its content filtering works across devices without installing client applications.
We were replacing a manual operating model
The starting point used BIND9 and legacy tooling that required too much manual configuration. The real work was larger than answering DNS queries. The product needed to apply different filtering policies, update configuration as settings changed, collect useful query information, integrate with network equipment, and deploy consistently in several locations.
At that stage, our largest uncertainty was product behaviour. We needed to work through rules, data structures, configuration changes, logging, and the operational flow around the service. Fast iteration mattered more than extracting the lowest possible latency from a design that was still moving.
Node.js was useful for exactly that reason. We could change the implementation quickly, test different approaches, and learn which parts of the design survived contact with real use. Starting immediately with a lower-level specialised stack might have produced a faster first benchmark. It would also have made each product experiment more expensive.
I would still choose Node.js, Python, or PHP for many early product stages. Developer time is a real performance constraint. A service delivered soon enough to teach you something can be more valuable than a technically faster service based on assumptions.
The failure looked like memory pressure
The DNS process later began to freeze under certain conditions. For DNS, a process-wide pause is a serious failure: one bad query should not stop unrelated lookups.
The symptoms looked like memory pressure or garbage collection. We followed that evidence. We tuned memory, simplified the processing path, moved hot data from PostgreSQL to Redis, and tried several smaller optimisations. Each change had a reasonable explanation behind it. Some improved the system, but the freeze remained.
The sequence is important because this was not random activity:
| Working theory | Change we made | What it taught us |
|---|---|---|
| PostgreSQL access was too expensive in the query path | Moved frequently used data to Redis | Data access became cheaper, but it was not the freeze's owning cause |
| Memory or garbage collection caused the pauses | Tuned memory and reduced work in the hot path | The symptoms remained after the expected pressure was reduced |
| General application overhead was the problem | Stripped and optimised the service further | Broad optimisation stopped producing useful diagnostic evidence |
| Particular DNS input triggered the failure | Isolated queries that could stall the server | The decision moved from tuning our code to investigating or replacing the DNS implementation boundary |
We eventually identified specific DNS queries that could stall the entire server. The visible symptoms resembled garbage collection, but the problem survived the changes that should have reduced that pressure.
Optimisation stopped reducing uncertainty
The turning point was not a particular CPU percentage or memory threshold. We no longer had a useful application-level experiment to run, and another broad optimisation would have repeated a theory that the previous changes had weakened.
We had two choices. We could continue into the internals of the Node.js DNS implementation and debug a complex failure below our application code. Or we could retain the product behaviour we had already learned and replace this one service with a foundation built around DNS workloads.
We chose CoreDNS and Go. CoreDNS is a DNS server written in Go whose functionality is composed from plugins. Its plugin
model gave us a clear place to implement LucidView's filtering rules while letting CoreDNS own the lower-level DNS
server behaviour. The CoreDNS project describes that plugin architecture, and its
plugin guide shows how requests move through a
ServeDNS handler.
Go was also a better fit for a small network service that needed predictable resource use and straightforward concurrency across available CPU cores. That does not make Go universally better than Node.js. It made the CoreDNS ecosystem and Go runtime a better boundary for this particular component.
The rewrite worked because the design was no longer moving
Rewrites are risky when they attempt to rediscover an entire product while replacing it. This one had a narrower boundary: replace the DNS implementation while preserving the filtering model and the connections around it.
By the time we started the CoreDNS plugin, the Node.js version had already answered the expensive questions:
- We knew how company and global filtering rules should interact.
- We knew which configuration the service needed and how updates should reach it.
- We knew which query information had operational value.
- We knew where the service connected to the rest of the platform.
- We had removed several avoidable costs from the data path.
The implementation changed, but the product model did not need to be invented again. That made the rewrite smaller and faster than starting with CoreDNS would have been during the exploratory phase.
In our production comparison, the replacement worked on its first production path, stopped the query-triggered freezes, used less memory and CPU, and handled more query load with faster responses. I do not have publishable benchmark numbers from the original transition, so I will not manufacture precision around that comparison. The operational result was clear enough to keep the CoreDNS implementation.
The part I would change is the timing. We spent too long assuming that another round of application optimisation would reveal the answer. Once we had isolated input that could freeze the server and exhausted the useful experiments in our own code, the cost of staying had become harder to defend than the bounded rewrite.
A rewrite decision should be narrow
This experience did not turn me into an advocate for rewriting systems whenever another language benchmarks better. Most rewrites throw away working behaviour, rediscover old edge cases, and delay improvements users can see.
I now look for four conditions before considering the same move:
- The failure belongs to a bounded component with a clear interface.
- The current implementation has already stabilised the product behaviour.
- Focused investigation shows that the constraint sits inside the component's present foundation.
- Replacing that component is cheaper and easier to verify than continuing below the current abstraction.
If those conditions are absent, improving the existing system is usually safer. If they are present, refusing to rewrite can become its own form of sunk-cost thinking.
Modern coding tools reduce some of the cost of trying another language, but they do not remove the important part of the decision. Teams still need to know whether the product model is stable, where the failure lives, and how they will prove the replacement behaves correctly.
Let the first version finish its job
The useful lesson was not "start in Go." The Node.js service did the job we most needed at the beginning: it let us discover the filtering product quickly. CoreDNS did the job we needed once the remaining problem was a specialised, query-sensitive network service.
A first implementation can be the right choice even when it is not the final one. The engineering mistake is either replacing it before it has taught you enough, or keeping it after it has stopped teaching you anything useful.
You can read more about the wider work in the LucidView DNS filtering case study. If a critical service has reached a similar decision point, my technical audit work is designed to separate evidence from assumption before committing to a larger change.