Convect deploys machine learning (ML) models to instantly callable, serverless API endpoints. Using Convect, Jupyter notebook users can deploy trained models from their notebooks and share them with the world in seconds. No web development or infrastructure experience is needed. Convect simplifies the process by being more opinionated than other model deployment workflows.
Deploying ML models to be used in production entails a different set of skills than training models in a sandbox environment and can get pretty complicated depending on what you’re trying to do. For many data scientists, this “sandbox” environment is a Jupyter notebook. One common approach to “deploying to production” that I saw while previously working as a data scientist at Intuit, is turning a model trained with scikit-learn in a notebook into an API endpoint.
From my experience, there are a few ways to deploy a model to an API endpoint and all of them involve a nontrivial level of effort and time. Examples of some of the steps in the process include pickling a model and uploading it to cloud storage, Dockerizing a model’s prediction code and environment, deploying a Flask app, or getting set up with an ML framework (e.g. MLFlow) or platform (e.g. SageMaker) so you can use the deployment feature in their SDK.
While complex workflows make sense for deploying complex models, I haven’t seen any dead simple deployment solutions for simple models, and that’s what I am working on building with Convect. In this case, simplicity comes at the cost of flexibility, i.e. you give up the ability to customize your infrastructure and runtime environment in exchange for a simple, one-click workflow. My hypothesis is that this tradeoff is worth it in many situations, and I’m curious to see what people are enabled to build when this aspect of the ML workflow is drastically simplified.
Under the hood, Convect creates two artifacts by serializing 1) the model prediction code and 2) all the variables that are in scope in the Python session at deployment time. I’ve made this part of the deployment code public here: https://github.com/convect-ml/convect/. These artifacts are then loaded and executed in an AWS Lambda function upon invocation by an API Gateway endpoint.
I’ve talked with 50+ data scientists at small to medium size companies (2-300 employees) and many have identified deployment as a pain point in their workflows. I’ve also spoken with a few data scientists who have indicated that this would help save time on their after-work/weekend side projects.
I’m sharing this now on Show HN because I’d love for people to try out Convect and to hear about how you use it or how I can improve it to make it useful. I’ve also put together a gallery of examples for training and deploying models to make it easy to quickly get started (https://convect.readme.io/docs) and provided example endpoints that you can use query or even build ML-powered apps on top of (https://app.convect.ml/#/demo). Find out more at https://convect.ml. Thanks for having a look!
> 2) all the variables that are in scope in the Python session at deployment time.
My eyebrows raised in surprise at this point! Obviously there's a tradeoff between convenience ("Oooh it's working! great let me deploy right now") and reproducibility ("Hmm I need to re-deploy what my teammate was working on because <reason>, let me go and find their notebook... ah here it is, oh weird why do I have this undefined variable") and this is hard on the convenience side.
Was it surprising to you too that you'd have users who wanted this location on the continuum?
You’re absolutely right that this product is heavy on the convenience side of the tradeoff.
> Was it surprising to you too that you'd have users who wanted this location on the continuum?
I’m still working on the “have users” piece, but there were signals for wanting more simplicity/convenience in conversations that I had with data scientists. I also felt as a data scientist that existing deployment solutions sometimes felt over-engineered for flexibility and reproducibility, at the cost of convenience. I'm exploring the other end of the continuum with Convect -- how opinionated can we get with ML workflows to simplify and still be useful?
Thanks! app.convect.ml is the ML model admin page.
The workflow looks somewhat like this —
Within a notebook, you can: 1) train a model, 2) deploy a model, and 3) get a shareable serverless API endpoint.
Outside the notebook, app.convect.ml provides a UI for editing, sharing, or deleting deployed models as well as getting the API key for deploying models.
At the moment, Convect only deploys scikit-learn models, but I am planning to support inference with PyTorch and other ML frameworks. Not planning to run on GPUs, as there are other products that already help with this, unless I see interest in this from my users. Convect would support concurrency according to AWS Lambda's quota, which is in the hundreds of thousands.
before the "boo birds" show up, I want to support this move as practical and useful. Real graduate projects using Scikit-Learn plus domain knowledge (gasp) make powerful results regularly. (I have no prior connection to this product)
The variety of data problems you can work on with good old logistic regression (or the other models in scikit-learn) is really quite vast and seemed like a reasonable place to start.
Very cool! I have run into the issue of deploying models recently. I see that this is using AWS lambda functions. How does this hold up with larger models (3-5gb)?
Trying to deploy a model to Azure Functions didn't play nice because of my file sizes.
Thanks! AWS Lambda supports Docker image sizes up to 10 GB (according to their docs), so on the back end, 3-5 GB could still work.
Convect, in its current state, is still limited to small models (e.g. < 0.5MB). This is because the deployment happens by posting to a REST API, which hasn't yet been tested for large payloads.
I wrote up some tips for decoupling a large model from the data that's causing it to be large (https://convect.readme.io/docs/news-topic-model). However, it sounds like you're asking about a truly large model. Convect is still in the MVP stage right now, but we plan to handle larger models. 3-5 GB models will likely be feasible in the future.
Is there support for including custom code as part of the deployment?
Models trained on tabular data (which are the majority of use cases in industry) usually require some pre-processing before feeding the model (i.e., feature engineering).
Congrats!
I frequently deploy sklearn models to lambda with API gateway so this is spot on for me. I hate integrating lambda with gateway, building custom layers etc so this scratches a real itch.
I’m not a fan of deploying my models on your infra, but maybe that’s an opportunity for you. I’d pay a subscription if the tool would let me deploy to my aws account. I think you can sell a cloud formation template through the aws marketplace as a manageable way to execute that.
Anyway, looks awesome and definitely scratches a real itch. Good luck
Thanks! Super glad to hear that this is addressing a real pain point.
The infra aspect has been on my mind — how to easily onboard users who prefer to use their own infra. Love the idea of selling a CF template on AWS Marketplace. I’ll look into that. Really appreciate this feedback :)
Thanks! FastAPI/ColabCode is a fair comparison. And I just tried it in Google Colab, and it does work! You can install Convect in a notebook cell with "!pip3 install convect"
Congrats on the launch, Sameeran! As a non AI/ML engineer, I love the demo page with the pre-populated examples. The idea that anyone can deploy and share ML models through a REST api seems quite cool.
I think there should be at least an approximate pricing on the homepage. It would help people decide if this is something they can think seriously about using.
Thanks for the feedback! That’s totally fair. My plan for now is to work with early users to understand what makes sense to them. At this early stage, any price I come up with would be almost a wild guess.
Convect deploys machine learning (ML) models to instantly callable, serverless API endpoints. Using Convect, Jupyter notebook users can deploy trained models from their notebooks and share them with the world in seconds. No web development or infrastructure experience is needed. Convect simplifies the process by being more opinionated than other model deployment workflows.
To give Convect a try, visit https://app.convect.ml. You can also try out models without signing into an account on the demo page (https://app.convect.ml/#/demo). I would love your feedback.
Some background/context:
Deploying ML models to be used in production entails a different set of skills than training models in a sandbox environment and can get pretty complicated depending on what you’re trying to do. For many data scientists, this “sandbox” environment is a Jupyter notebook. One common approach to “deploying to production” that I saw while previously working as a data scientist at Intuit, is turning a model trained with scikit-learn in a notebook into an API endpoint.
From my experience, there are a few ways to deploy a model to an API endpoint and all of them involve a nontrivial level of effort and time. Examples of some of the steps in the process include pickling a model and uploading it to cloud storage, Dockerizing a model’s prediction code and environment, deploying a Flask app, or getting set up with an ML framework (e.g. MLFlow) or platform (e.g. SageMaker) so you can use the deployment feature in their SDK.
While complex workflows make sense for deploying complex models, I haven’t seen any dead simple deployment solutions for simple models, and that’s what I am working on building with Convect. In this case, simplicity comes at the cost of flexibility, i.e. you give up the ability to customize your infrastructure and runtime environment in exchange for a simple, one-click workflow. My hypothesis is that this tradeoff is worth it in many situations, and I’m curious to see what people are enabled to build when this aspect of the ML workflow is drastically simplified.
Under the hood, Convect creates two artifacts by serializing 1) the model prediction code and 2) all the variables that are in scope in the Python session at deployment time. I’ve made this part of the deployment code public here: https://github.com/convect-ml/convect/. These artifacts are then loaded and executed in an AWS Lambda function upon invocation by an API Gateway endpoint.
I’ve talked with 50+ data scientists at small to medium size companies (2-300 employees) and many have identified deployment as a pain point in their workflows. I’ve also spoken with a few data scientists who have indicated that this would help save time on their after-work/weekend side projects.
I’m sharing this now on Show HN because I’d love for people to try out Convect and to hear about how you use it or how I can improve it to make it useful. I’ve also put together a gallery of examples for training and deploying models to make it easy to quickly get started (https://convect.readme.io/docs) and provided example endpoints that you can use query or even build ML-powered apps on top of (https://app.convect.ml/#/demo). Find out more at https://convect.ml. Thanks for having a look!