Cerebro Documentation

Cerebro is a data system for optimized deep learning model selection. It uses a novel parallel execution strategy called Model Hopper Parallelism (MOP) to execute end-to-end deep learning model selection workloads in a more resource-efficient manner.

Installation

The best way to install the Cerebro is via pip.

pip install -U cerebro-dl

Alternatively, you can git clone and run the provided Makefile script to install the master branch

git clone https://github.com/ADALabUCSD/cerebro-system.git && cd cerebro-system && make

Note

You MUST be running on Python >= 3.6 with Tensorflow >= 2.3 and Apache Spark >= 2.4

Table of Contents

• What is Cerebro?
• Install
  • Basic Installation
  • Spark Cluster Setup
• Getting Started
  • Executing a Model Selection Workload
  • Visualizing the Model Selection Process
  • Training on Existing Petastorm Datasets
  • End-to-End Example
• Cerebro API
• Acknowledgement

Indices and tables

• Index

• Module Index

• Search Page