Happy Thursday! This post is a day late only because the past two weeks have been incredibly busy, from implementing server-side predictions for NatML to finalizing the upcoming NatDevice update. There have been a lot of exciting developments focused specifically on giving developers a tightly-integrated, extremely quick, and extremely performant platform to develop and deploy machine learning-powered interactive media applications. The next major aspect of this strategy lies with NatML Hub.

The Need for a Hub

NatML started out as a Unity library to run machine learning models with hardware acceleration. But once we got to first inference, it became clear that the developer experience was still pretty poor, even with the simple API. There were two problems that needed to be addressed: providing important model data to predictors (think normalization coefficients and class labels); and creating ‘bundles’ of these model-with-model-data packages for developers to easily integrate into their app.

What we settled on was NatML Hub, an aggregation of machine learning predictors. With predictors, developers wouldn’t have to care about the actual implementation of the ML model they were using. And with a web platform that aggregated these predictors, developers had a springboard to find and integrate different models for different tasks. But we thought to take it one step further by making Hub not only an aggregator, but a marketplace.

Enticing Market Dynamics

Perhaps the best type of platform to build, and the most difficult, is a two-sided marketplace. Marketplaces at scale provide the most desirable dynamics: structured incentives, network effects, and revenue that scales with activity. In the case of NatML, there were some dynamics we had especially hoped that a predictor marketplace would help us solve:

Application Coverage

Developers almost never care about a specific machine learning model; what they care about is any model that can help them achieve a specific task, be it pose detection or image classification for example. As such, we needed NatML Hub to have at least one predictor for each of the many different tasks we could think of. Having a Hub marketplace would free us of this responsibility, as ML practitioners could bring their own models to the platform, incentivized by revenue they could generate from sales.

Model Coverage

For any given task, there is typically a multitude of models from research labs, universities, and companies that solve said task, at least generally so. Naturally, each of these models will have a specific characteristic that makes it unique, typically a tradeoff somewhere on the performance-accuracy spectrum. With this in mind we would expect that some developers would want a really accurate model (think facial recognition), whereas other developers want something that works as fast as possible (think AR face filters on mobile). Having a Hub marketplace would provide developers with options—orders of magnitude more options than almost any other managed ML service we know of.

Aggregation Feedback Loop

Finally as the marketplace scales, and as more and more models are published by authors, we would expect that it would get more unlikely that a model that solves a specific task isn’t already available on Hub. As a result the marketplace would feed the aggregation, which in turn would feed the marketplace, and on and on.

The Problem with a Marketplace

In all the points above, we had a fundamental assumption that turned out to be false. It centers around model specificity. Simply put, except for only a very (very) small set of tasks, machine learning models need to be trained on specific data in order to provide value to end users. For example, a developer looking to classify images of specific breeds of fish will be out of luck with all the pre-trained image classification models out there.

Such a developer will have no choice but to train a model on their custom data. And at this point, none of the marketplace dynamics apply: there’s little reason to believe that other developers are looking for a model trained on such specific data; training on proprietary data raises intellectual property concerns, preventing such models from being published on an online marketplace; and even if such a model was published on said online marketplace, all sorts of issues around dataset quality and model accuracy and bias quickly become prominent.

Going Back to an Aggregator

With this in mind, we are shifting NatML Hub back to strictly being an aggregator. This decision has a few notable consequences:

1. All new predictors published publicly on the platform will be available to all users—no up-front purchase required.

2. Hub will shift to charging a monthly subscription for access to the platform and all predictors listed on the catalog.

3. There will be a free tier, where a small-enough amount of usage will require no subscription fee at all. We want to encourage developers to experiment with different predictors without having to hit a paywall.

4. Hub Predictors, which perform inference server-side, will cost a price that scales with the number of predictions made. This is consistent with other cloud ML providers like Google Cloud and Azure, because we have to pay for server processing time and GPU hardware. There will also likely be a free tier for this.

5. Predictors on Hub will link to external package managers for each framework that the predictor has been implemented for. For example, predictors for Unity will link to the Unity Asset Store; predictors for NodeJS will link to NPM; and predictors for Python will link to PyPi.

For current users of the platform, nothing changes in the immediate term. Our next steps are to roll out Hub Predictors, publish the NodeJS and Python SDK’s, then setup subscriptions. Once that’s done, we will be focusing on all things performance. An exciting development worth mentioning is that NatML has joined Google’s TensorFlow Lite for Android early access program, giving us access to the TFLite runtime embedded on certain devices. Until next time, happy coding!