## Weights & Biases

- [https://www.wandb.com/](https://www.wandb.com/)
- W&B is a key piece of our fast-paced, cutting-edge, large-scale research workflow: great flexibility, performance, and user experience.
- Experiment tracking for deep learning
- Instrument training scripts
- Stages covered: only from Model training to deploy (run multiple experiments, track and look at the data visualisations - only helps track, and use the info to plot graphs and make your decisions)
- Pros:
  - Supports model execution and tracking of the execution
  - Gathers good stats / metrics to make sound decisions after or during execution of experiments
  - Has support for comparisons of experiments (and models) and performances
  - Tracks the best model producing during experiments execution
  - Supports multiple python frameworks (keras, tensorflow, pytorch)
  - Getting started with tool is really easy and great way to integrate hooks into our experiment code
  - Very good UX, docs and site layout, user-friendly
- Cons:
  - Does not support or have tooling for stages before experiment execution (i.e. data cleaning, visualisation, data validation, feature extraction)
  - Not sure if they provide infrastructure on which we can run our experiments
- Installation or getting started
  - https://docs.wandb.com/docs/started.html
- Pricing & features: https://www.wandb.com/pricing
- Documentations
  - https://docs.wandb.com/docs/started.html
  - Sweeps: https://docs.wandb.com/sweeps
  - Framework docs: https://docs.wandb.com/library/frameworks
  - Group: https://docs.wandb.com/library/advanced/grouping
  - Resuming tracking: https://docs.wandb.com/library/advanced/resuming
- Examples
  - https://docs.wandb.com/docs/examples.html
- Code & concepts
  - [Code snippets](./wandb/code-snippets.py)
  - [Quick and Dirty CNN](./wandb/Quick-and-Dirty-CNN.py)
  - [Activation Function](./wandb/Activation-Function.png)
- Videos
  - Tutorial: https://www.wandb.com/classes/intro/overview
- GitHub
   - Organisation: https://github.com/wandb
   - W&B Client: https://github.com/wandb/client
   - W&B User issues: https://github.com/wandb/user-issues 
   - W&B Examples: https://github.com/wandb/examples
   - W&B Artifacts examples: https://github.com/wandb/artifacts-examples
   - W&B CodeSearchNet: https://github.com/wandb/codesearchnet (an F/OSS benchmarking project in collaboration with GitHub)
   - W&B Training: https://github.com/wandb/wandb-training
   - W&B Tutorial: https://github.com/wandb/tutorial
   - W&B GitBook: https://github.com/wandb/gitbook
   - W&B Docs: https://github.com/wandb/docs
- Additional resources
  - [Error caused by missing input_shape in your first layer](https://stackoverflow.com/questions/52690293/tensorflow-attributeerror-nonetype-object-has-no-attribute-original-name-sc)
  - [Bloomberg summary colab notebook](https://colab.research.google.com/drive/1lfLR9WRzmjOMmnNmePys4-8WNfZ5xC90#scrollTo=wbjXyjFRaT1d)
  - [Implementation of W&B for model param tuning](https://www.kaggle.com/robertehshi/basics-of-nfl-expost)
  - https://talktotransformer.com/ - Adam Daniel King's implementation of GPT-2 on the back of the PyTorch version
  - https://app.wandb.ai/sayakpaul/jigsaw-toxic/reports/Kaggle-Starter-Kernel---Jigsaw-Multilingual-Toxic-Comment-Classification--Vmlldzo3NjE1MQ
  - https://www.wandb.com/kaggle | https://www.youtube.com/watch?v=MFJnkgUTMrM&
  - https://www.wandb.com/articles/model-explorations-and-hyperparameter-search-with-w-b-and-kubernetes (Robert Porsch)
  - https://www.kaggle.com/robertehshi/basics-of-nfl-expost (Robert Lutz)
  - https://app.wandb.ai/cayush/pytorchlightning/reports/Use-Pytorch-Lightning-with-Weights-%26-Biases--Vmlldzo2NjQ1Mw
  - [Automate Kaggle model training with Skorch and W&B:](https://app.wandb.ai/cayush/undefined/reports/Automate-ML-experiments-with-Skorch-and-W%26B--Vmlldzo4NTQ1NQ?utm_source=social_twitter&utm_medium=report&utm_campaign=report_author)
  - [W&B Sweep: NeRF-–-Representing-Scenes-as-Neural-Radiance-Fields-for-View-Synthesis](https://app.wandb.ai/sweep/nerf/reports/NeRF-%E2%80%93-Representing-Scenes-as-Neural-Radiance-Fields-for-View-Synthesis--Vmlldzo3ODIzMA) | [Notebook 1](https://github.com/lavanyashukla/neural_networks/blob/master/Hyperparameter_Sweeps_with_W%26B.ipynb) | [Notebook 2](https://github.com/lavanyashukla/neural_networks/blob/master/Hyperparameter_Optimization_on_a_NeRF.ipynb)
  - [Track model performance](https://app.wandb.ai/lavanyashukla/visualize-models/reports/Track-Model-Performance--Vmlldzo1NTk2MA)
  - [Hyper parameter sweeps](https://colab.research.google.com/drive/1gKixa6hNUB8qrn1CfHirOfTEQm0qLCSS)
  - [GPU metrics](https://app.wandb.ai/stacey/estuary/reports/System-metrics-for-model-comparison--Vmlldzo1NzI5Mg)
  - [Visualise model prediction](https://app.wandb.ai/lavanyashukla/visualize-predictions/reports/Visualize-Model-Predictions--Vmlldzo1NjM4OA/)
  - [Visualise Scikit Models](https://app.wandb.ai/lavanyashukla/visualize-sklearn/reports/Visualize-Scikit-Models--Vmlldzo0ODIzNg)
  - [Save and restore models](https://app.wandb.ai/lavanyashukla/save_and_restore/reports/Saving-and-Restoring-Models-with-W%26B--Vmlldzo3MDQ3Mw)
  - [Keras and W&B](https://www.kaggle.com/robertehshi/basics-of-nfl-expost)
  - [Pytorch and W&B](https://app.wandb.ai/cayush/pytorchlightning/reports/Use-Pytorch-Lightning-with-Weights-%26-Biases--Vmlldzo2NjQ1Mw)
  - [Tracking experiments](https://towardsdatascience.com/machine-learning-experiment-tracking-93b796e501b0)
  - Implementing W&B sweeps: [Sweeps](https://docs.wandb.com/sweeps/overview ) | [Notebook](https://colab.research.google.com/drive/1gKixa6hNUB8qrn1CfHirOfTEQm0qLCSS )
  - ...for more [see this](./wandb/More-resources.md)

---

- [ ] [AI/ML/DL Library / Package / Framework: applicable]
- [ ] [Inexpensive crowd-sourced infrastructure sharing: applicable]
- [ ] [Data cleaning: manual / no tools available] 
- [ ] [Data querying: manual / tools available] 
- [ ] [Data analytics: manual / tools available]
- [ ] [Data visualisation: manual / tools available] 
- [ ] [Data validation: manual / no tools available] 
- [ ] [Feature extraction: manual / no tools available] 
- [x] **[Model creation: available]**
- [x] **[Execute experiments: available]**
- [x] **[Track experiments: available]**
- [x] **[Hyper parameter tuning: available]**
- [x] **[Model saving: available]**
- [x] **[Visualisations: available]**

Back to [Programs and Tools](./programs-and-tools.md#programs-and-tools). <br/>
Back to [Data page](./README.md#data).