Platform Products for Machine Learning

Metadata

• Author: Misbah Uddin
• Full Title: Platform Products for Machine Learning
• Document Note: # Why ML Platform teams ML stream aligned teams (those facing ML projects for end users) cognitive load might be reduced if superflous MLOps tasks are managed by the ML platform. The lesser the team needs to think about Ops the more the team can take on solving high value-added questions.

  What ML Platform products

  1. Data/ML exploration:
  2. Code management
  3. Data/feature management
  4. Model management
  5. Reporting:
• URL: https://towardsdatascience.com/platform-products-for-machine-learning-3d3749443d2

Highlights

• or an ML team, there can be three types of cognitive load, as adapted from Managing cognitive load for team learning.
  1. Intrinsic: relates to fundamental Data/ML tasks, such as, how to write Spark transformations, formulate ML pipeline, etc.
  2. Germane: relates to complex Data/ML tasks that require special attention, such as, how to prepare features for a specific operational use, how to interpret a model performance before/during/after its usage?
  3. Extraneous: relates to the environment in which the Data/ML task is carried out. For example, where perform code quality analysis and how to carry out rudimentary code quality analysis in that environment, where to manage modeling activities, and how to carry out common model management tasks in that environment? (View Highlight)
• The intrinsic cognitive relates to fundamental skills in a stream-aligned ML team and should be handled through hiring, training, pair/mob programming, hackathon, etc. The reduction efforts of such loads should be driven by engineering/data science/analyst managers. The germane cognitive load relates to advanced tasks in a stream-aligned team and can be reduced through sufficient opportunity to work on such problems and review support by experts. However, extraneous cognitive load relates to routine superfluous MLOps tasks that add little value in retaining working memory. Most of these relate to platform management. (View Highlight)
• Common ML platform activities are as follows:
  1. Data/ML Exploration: Explore data access/wrangling, ML methods, visualization, …
  2. Code Management: source code editing, version control, continuous code quality evaluation, continuous code release, …
  3. Data/Feature Management: data pipeline orchestration, data snapshotting, feature, …
  4. Model Management: ML pipeline orchestration, track experiments, handle model lifecycle, model serving, …
  5. Reporting: dashboards, error logging, monitoring, … (View Highlight)