bentoml

We have hosted the application bentoml in order to run this application in our online workstations with Wine or directly.

Quick description about bentoml:

BentoML simplifies ML model deployment and serves your models at a production scale. Support multiple ML frameworks natively: Tensorflow, PyTorch, XGBoost, Scikit-Learn and many more! Define custom serving pipeline with pre-processing, post-processing and ensemble models. Standard .bento format for packaging code, models and dependencies for easy versioning and deployment. Integrate with any training pipeline or ML experimentation platform. Parallelize compute-intense model inference workloads to scale separately from the serving logic. Adaptive batching dynamically groups inference requests for optimal performance. Orchestrate distributed inference graph with multiple models via Yatai on Kubernetes. Easily configure CUDA dependencies for running inference with GPU. Automatically generate docker images for production deployment.

Features:

• Online serving via REST API or gRPC
• Offline scoring on batch datasets with Apache Spark, or Dask
• Stream serving with Kafka, Beam, and Flink
• Automatically generate docker images for production deployment
• Model Deployment at scale on Kubernetes
• Fast model deployment on any cloud platform

Programming Language: Python.
Categories: