Automating machine learning lifecycle with AWS

the documentation. In this article, machine learning lifecycle can be replaced with dat

Data Acquisition

Streaming Data

Amazon Kinesis used for clickstream analytics. Image source -https://aws.amazon.com/kinesis/

Batch Data

Data Lake

Amazon Simple Storage Service (Amazon S3)

Databases

AWS built in databases. Image source — https://aws.amazon.com/blogs/publicsector/purpose-built-databases-model-building-applications-cloud/

Data Processing

Amazon EMR (previously called Amazon Elastic MapReduce)

image source — https://makeameme.org/meme/data-increased-by

Amazon MSK(Managed Streaming for Apache Kafka)

Data Cleaning and Wrangling

Amazon SageMaker Data Wrangler (Data Wrangler)

Data Labeling

Amazon SageMaker Ground Truth Plus

Amazon SageMaker Ground Truth

Data Visualization

https://aws.amazon.com/quicksight/

Amazon QuickSight

Feature Engineering

Amazon SageMaker Feature Store

Amazon SageMaker Notebook sagemaker

Model Training

Amazon Elastic Compute Cloud (Amazon EC2)

Amazon Batch

SageMaker Training Compiler

Hyperparameter Tuning

SageMaker Auto Tuning

Model Selecting

Autopilot

https://aws.amazon.com/sagemaker/autopilot/

Amazon SageMaker Experiments

Model Tracking

Amazon SageMaker ML Lineage Tracking

SageMaker Debugger

Model Monitoring

Amazon SageMaker Model Monitor

Amazon SageMaker Clarify

Model Registry

SageMaker model registry

  • Catalog models for production.
  • Manage model versions.
  • Associate metadata, such as training metrics, with a model.
  • Manage the approval status of a model.
  • Deploy models to production.
  • Automate model deployment with CI/CD.

Model Serving

Amazon SageMaker Serverless Inference

Amazon Elastic Container Registry (Amazon ECR)

Amazon Elastic Kubernetes Service (Amazon EKS)

Model Deployment

SageMaker project

Amazon SageMaker Neo

Workflow Manager

Amazon Step Function

CI/CD

Amazon CodeCommit

Amazon CodeBuild

Source

Amazon CodePipeline

Amazon Code Deploy

Amazon CodeGuru Reviewer

Amazon CodeArtifact

  • securely store packages
  • sharing packages during application development
  • ingest from third party repositories making it easy for organizations to securely store and share software packages used for application development.Use case using codeartifact for developing serverless application.

Conclusion

--

--

--

I develop machine learning models and deploy them to production using cloud services.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Robert John

Robert John

I develop machine learning models and deploy them to production using cloud services.

More from Medium

MLOps — Building End to End Pipeline using Sagemaker SDK & AWS CodeCommit

How to Build and Run your Entire End-to-end ML Life-cycle with Scalable Components

Experience report: Data Version Control (DVC) for Machine Learning Projects

https://dvc.org

Machine Learning CI/CD Pipeline with Github Actions and Amazon SageMaker