Ship it before it's done
The learning comes after the launch
Welcome to Scaling Down, a newsletter written by the team at Statsig for engineers and product builders who made the leap from Big Tech to a startup.
A word from the author: We penned this note before a software bug caused a massive systems outage around the world. Whether we post it now or later, we stand by the message – we’re not advocating for shipping untested features or forgoing release processes. While the catchy title may seem controversial, we encourage you to read the whole post before passing judgment. We’re writing for startups and for entrepreneurs who are pre-product market fit, and for whom speed is everything. In the interest of “shipping before it’s done,” we’ve decided to post this today.
Reviewing what is launch blocking
We have a saying at Statsig - “ship it before it’s done”. It’s a controversial take on the order of “Move fast and break things.” But the longer I work, the more I learn that same lesson over and over.
At face value, people often mistake the saying as a leeway or encouragement to ship a buggy product or a bad experience. Instead, it should serve as a reminder that your customers may not care about the features you see as “launch blocking.” It should force you to reconsider your p0 items, drop dead date, and why you can’t ship part of a larger feature today.
If you don’t get something out there, even if it doesn’t have all the bells and whistles, you’ll be building in a vacuum forever.
You may have noticed this attidue in small ways before. Take this article as an example. I can brainstorm with people what I want to write and ask for feedback on the different topics. But it’s only when I actually put words on paper and riff on ideas, do I get much more actionable feedback. A few small tweaks here, move this up, expand on that story…thoughts that take my musings in a document into a cohesive post worth sharing.
Just pick a direction and go. The sooner you do, the sooner the future decisions reveal themselves. They actually get easier!
Launching in five weeks
The journey at Statsig has been no exception. We started on February 17, and set a goal: launch a beta five weeks later, on March 31. We had an advantage for sure: we started with 7 engineers and 1 data scientist who all knew the tool we were setting out to build.
Many startups spend their first year just trying to hire that big of an engineering team. Others stay in stealth for months or even years, building products in isolation, or with a handful of partners. But we were going to ship the damn thing before it was done!
Building a piece of the vision first
Although we set out to build an entire platform and suite of tools that any company could use, we needed to ship something before that vision had time to materialize. We focused on a single product (feature flags) with one differentiated feature: automatic A/B testing for partial rollouts. We picked a single tech stack to work with, JavaScript on the client and server. And we set out to make it all work.
I’ve worked on SDKs since day one. I still remember figuring out how to set up a new npm package – battling with Babel and Webpack. We didn’t even think to write the packages in Typescript – we just wrote plain Javascript.
On the console side, we had some sketches and design mocks that Vijaye put together. It looked something like this:
We used NextJS for the web app, so we put some API routes in there as well, knowing we could separate out an API deployment later. On day two, our initialize endpoint for client SDKs looked like this
// src/api/sdk/init.ts
export default async (req: NextApiRequest, res: NextApiResponse): Promise<void> => {
const sdkKey = req.query.sskey as string;
const userID = req.query.user as string;
const vc = await EndUserViewerContext.genFromSDKKey(sdkKey, userID);
const companyID = vc.getCompanyID();
if (!vc.getUserID() || !companyID) {
res.json({ success: false });
return;
}
const allGates = await EntityGate.genCompanyGates(vc, companyID);
res.json({ gates: allGates });
};For our infrastructure, we knew we would need to handle multiple pods in multiple regions in multiple availability zones, so we started out with two regions, us-east, and us-west.
On the data side, we needed an exposures table and an events table in order to generate event metrics, and some daily jobs to compute experiment results.
And of course, we needed a web landing page, email collection to sign up for updates, and account creation.
The learning comes after the launch
After that month, we could have kept building instead of putting out a beta. Things were still very rough around the edges. We had feature gates and some event logging, but we didn’t have support for multivariate tests, mutual exclusion, custom metrics, mobile apps, or non-Javascript backends (like we do today)…the list went on and on.
But the act of launching forced us to learn how to deploy our applications, publish our SDKs, run our pipelines, and monitor our services. It gave us something we could market – even if it was only to the few people who signed up, our friends and colleagues who watched us leave Facebook to chase this dream.
It helped us hire our first designer, who at least had something to picture and go off of. The fact that it was ugly only made it easier to hire a designer! And it gave us internal feedback. We were all Statsig users at that point and had opinions on how things should look and work.
Had we stayed in stealth mode, we would have built so many features that nobody cared for, and supported integrations nobody needed. But we shipped Statsig before it was done, and we still do that to this day.
The tradeoff of moving fast
The decision to ship instead of staying in stealth represents a massive tradeoff. While I’ve presented the positive side effects of shipping sooner, it doesn’t come at no cost.
Some moments are finite – coming out of stealth, announcing your beta, or publishing version 1.0 is a big moment. If things aren’t in the right place, you could miss out on the set of people who experience your product for the first time. But it’s not just about capitalizing on big moments.
Shipping Statsig added a lot of extra work for our small team. Now, in addition to building out more of the features and functionality of the platform, we also had to spend time on maintenance, release management, real-time monitoring, bug fixes, and public documentation. We also couldn’t make breaking changes as easily.
I remember a few months in having to do a database migration, and since we had actual production data, the process was much more involved than if we could wipe the slate and start clean.
The balance for every startup will be different – your time in stealth mode is a gift, but that time is limited. Statsig will continue to be on the side of shipping sooner rather than later – for us, it has made all the difference.
hello
this is poC