Hardly any changes matter
In May 2024, we did an intentional 180 on many foundational theses underpinning Kiwi News. The most major being that we had allowed the website to be 100% usable by any user, paying or non-paying, logged in or not logged in. As a method of discovering more information, since I found that allowing anyone to use the site enabled free riding, we hence pay-walled the site, disabled our Warpcast and Twitter bots, and forced users to add an application key and an active wallet connection. We actually didn’t do this because we thought this was the better way forward but because we had learned that how we had done things prior hadn’t worked out for us.
At the time, and even to this day, if you’d ask me to categorize how drastic these changes were to the core functionality of the app, I’d say that they affected the usability of the application pretty much, and so just making these changes from one day to another still feels like YOLO’ing into a bunch of risks with a project we had otherwise painstakingly built up over many months. The affected functionality and whether anonymous visitors could fully use the app had been, for the entirety of the project, a core thesis to grow the number of self-submitting writers to the app. That is, as we had reasoned that more clicks would lead to higher motivation and prestige to be a renowned writer on Kiwi News. You’d think that if you forbid anon-visitors to use the site and make it, most likely, unavailable on many users’ second device (something I believed in at the time), most would drop off, and so this would put at risk core KPIs, like Daily Active Users, Weekly Active Users, and Monthly Active Users. But looking at the protocol stats, which mainly track active engagement it’s even hard to see any change that would suggest we even had introduced this drastic change. Look at them below; isn’t it really hard to spot?
And even if you seem to make out the drop in user engagement in the charts, what makes you think this was that particular roadmap’s fault? After all, northern hemisphere summer started in May, too, which could also sufficiently explain the minor breakdown. And there are like another ten things that happened in those days that could adequately describe the artifact in the data, too.
Now, to be fair, our Google Analytics numbers are down much more, and they certainly suggest that we booted off a bunch of random visitors since May. Yet, I can’t even describe how much disconnect I feel between adding the paywalls and Google Analytics being down—there are just so many other variables that it is tough to discern causes and effects.
So what’s the lesson here?
For most of the time working on Kiwi News, I always had worked on the product with the idea that any new feature could be its breakout point: “Just this one lil feature and we’ll see hockey stick growth! Just that one additional blog post more, and people will get it!” My model had been that of a viral outbreak where we’d just have to engineer for that, bringing the R0 value above one. Frankly, I still believe in this idea of engineering a milieu where more people stick around than leave. Yet, I now do see this as much more laggy than I had originally thought throughout the first year. In fact, I had always believed in a very direct cause-and-effect relationship between me shipping and the crowd using. I now think it takes months for this to actually play out as the replication dynamics run slowly, and since I’ve learned how exceptionally willing to adapt our users are.
This, in turn, also affects the way I think about decision-making. Whereas before, I had always waited for the “breakout” moment to happen after a week of feature shipping or yapping about the product on social media, I now feel like waiting for the lag might actually unnecessarily slow us down when, in fact, we can have real conviction in changing the product straight away. For example, last week, we just fixed a bunch of broken stuff so that older Android devices can use the site better. It is also somehow true what they say about ChatGPT commoditizing software development: More often than not, it isn’t the complexity of implementing code that results in enabling a net new functionality for our users that is difficult or worthwhile doing. What’s the hardest is actually finding the fleeting truth floating around in the data, the process of learning and discovering the right models to handle the incredible uncertainty that comes with building a social network for over one and a half years. Throughout this process, the super-engaged Kiwi News community in our Telegram has been an invaluable help in informing us how to adjust our trajectory. It’s incredible to see how much insider knowledge we have amassed in a year of shipping.
The lesson here, that hardly any changes matter, is somewhat liberating and depressing at the same time. Yes, that one tiny feature might finally tip the scale and spur sustainable viral growth some weeks down the road, yet it’s incredibly difficult to understand which of these actually make an impact, and matching cause and effect is essentially a fool’s errand.
I have, however, always thought of great businesses as very sluggish in nature. Sluggish in the sense that a successful business accumulates information over time and internalizes it as a business process or logic. A business that meets demand organically in a reasonably defensible way is like a rock in the surf. The water eventually affects it and shapes it in the way of the current, but fundamentally, the rock remains in place and character for most of its lifetime. Although not exciting to the startup-brained founder who always romanticizes incredible growth and volatility, all the above is good news because it ultimately informs the process of building. Aim to make a 1% improvement a day, and you’ll end up with an incredibly well-working machine in some years of time. Hardly any changes matter in the moment, but their accumulation leads to incredible results.