September 22, 2021
October 20, 2020
·
7
Min Read

No Spec, No Problem: How I Autogenerated an API Spec for Notion

by
Chris Corcoran
He man releasing all his power
Share This Article

Software is eating the world and APIs are at the forefront. Not only do APIs give developers instant access to powerful functionality, but their programmability also makes it easy for developers to integrate disparate pieces of functionality.

But today, this only applies to documented APIs. This means the company responsible for the API has made a conscious choice to expose the API—and they’ve allocated the resources to creating and maintaining documentation. This also means there are a lot of SaaS APIs out there that could make users’ lives easier, but that haven’t been documented.

This blog post is about a new feature that my team and I built after I spent a painful couple of days figuring out how to script against Notion. After my experience trying to learn the undocumented Notion API, we decided to automate the process of learning web APIs so that nobody would have to suffer like this again.

In this blog post, I talk about:

  • How I tried to automate against the Notion the hard way.
  • How this led my team at Akita to build a new feature to automatically learn undocumented APIs.
  • How to use Akita to automatically learn APIs, including the Notion API. (See what we learned on SwaggerHub here!)

To anyone who wants to learn an undocumented API–or document your own API so people don’t have to do this—you may be interested in checking out our private beta!

🐕 First, some background

I’m Chris, the product manager at Akita. I joined Akita after eight years at Twilio, where I helped define products like Twilio Marketplace and Twilio Channels. While I was making APIs easier to use at Twilio, I also saw how APIs made software development harder. For instance, the rise of internal and external APIs make it much more difficult to prevent breaking changes. These observations led me to join Akita about a year ago.

At Akita, we’ve been working to give structure to the interaction graph of APIs, for instance to detect breaking changes in web apps. Akita watches API traffic to automatically generate API specs and infer implicit API contracts. Core to Akita’s approach is diffing API specs and contracts learned across different environments. We’ve been working hard to make this technology easily accessible to developers, no code changes required, in just minutes.

💡 An observation about Notion

Now for how I came to autogenerate an API spec for Notion.

As the product manager at a small startup, I spend a lot of time making sure that the right information gets to the right place. Customer feedback, blog posts like this one, product usage, and engineering metrics are just a few of the things I need to track. I also require some sleep, food, and the occasional trip outside. How do I manage to keep up? That’s easy: AUTOMATE ALL THE THINGS!!!

When I joined Akita, I realized that Notion was what we needed to organize information across the team. But the problems started when we needed to add new information to Notion. For instance, we keep team meeting information in Notion. Most of our team meetings began with “Did someone create a Notion page?” or “Where is the page from last week’s meeting?” The same thing would happen during customer meetings.

Given how many meetings we have a week, I figured we could get several hours back a week if we could get computers to automate creating the Notion page for us. All we needed was to figure out how to automate Notion.

😰 Thwarted by the Private API

The catch? Notion doesn’t have a publicly documented API.

The good news is that it’s still possible to use an undocumented API. It’s easy. You could set up a headless browser to automate clicks or turn to that ol’ standby Grease Monkey script for watching your browser traffic. I recently read a blog post where the author ran a proxy and executed a man-in-the-middle attack in order to build a calendar widget for one of his favorite websites.

dog_typing.gif

Okay, it’s not easy. It took me a couple days of hacking to get a reasonable solution working the first time. And the undocumented API problem is prevalent among newer SaaS tools, and it makes sense: APIs are hard to build, document, and maintain. If more of these tools had APIs, I would automate more and integrate them more deeply into our workflows.

⚡️ Using Akita to learn the Notion API

I wondered if there was a better way.

Then I realized: oh wait, I’m working on building a tool that lets you generate API specifications by monitoring network traffic. Surely we could use Akita to build API specs from watching my interactions with Notion?

But it wasn’t completely straightforward. Up to this point, we had set up Akita with the assumption that the user wants to document their own API, so Akita worked by listening to network traffic through an agent.

But when I posed this problem to my engineering team, they came up with a solution that let me generate this beautiful rendition of the Notion API spec.

Spec.gif

This feature was a game-changer! Now that Akita could learn this spec, I had an OpenAPI specification that detailed every endpoint, parameter, and property that the Notion frontend used. And what made this really useful was Akita’s analysis layered on top.

Screen Shot 2020-10-16 at 5.53.52 PM.png

Akita Data Formats, detecting precise data formats like unique IDs and specific datetime RFCs,  and API Relationships, surfacing which endpoints accept values of the same data type, made it easier to understand how the endpoints worked together.

You can check out the API spec we were able to generate on SwaggerHub here.

🛠 How it works

The secret to this new feature? HAR Files.

Previously, to run Akita you ran an agent alongside where you hosted a service. The agent would listen to network traffic and allow you to autogenerate API specs without having to make code changes or use a proxy. While this was super easy to use if you were learning your own API, it was less helpful for learning other people’s web APIs. Our new HAR file ingest allows you to learn any web API that relies on AJAX-style requests, using any major web browser, in a matter of minutes.

HAR Files are a little known feature of the Webkit Developer Console (available in major browsers including Chrome, Firefox, Safari, and Edge) that allows you to record requests, performance metrics, and other interesting data from a debugging session. It turns out that it’s not so hard to extend Akita to accept HAR files instead of network traffic. HAR files are key to making it easy to autogenerate API specs, without needing headless servers or proxies.

power_of_greyskull.gif

Below I show a screen capture of me trying to break this new feature by returning to my old nemesis, Notion. In just minutes, I was able to automatically learn what had previously taken me days to uncover!

Browsing.gif
Dev Tools.gif

All you need to do in order to generate a spec is:

  1. Navigate to the website you want to learn an API for and start network log recording. (See instructions across different browsers here.)
  2. Interact with the website to generate traffic.
  3. Save your HAR file and use the Akita CLI to upload it to Akita.
  4. akita har ingest --service {SERVICE} {PATH_TO_HAR_FILE}
  5. You’re done! You’ll get a link for the generated API spec. 🎉

Note that if you’re on the other side and want to learn a spec for your own API, the steps you follow are even simpler!

🏝  It’s your turn to live your best life

Now that HAR file ingest is an official Akita feature, you, too, can wield this incredible power. With our new HAR File Ingest feature, you can now use Akita to learn the spec of any website you are using—and write time-saving automations like my script here.

If you’re interested in autogenerating API specs, sign up for our private beta here. We’d love to see what you think!

Share This Article

Join Our Private Beta now!

Thank you!

Your submission has been sent.
Oops! Something went wrong while submitting the form.