# Introduction

After a tremendous age of innovation and research in the AI field, we are moving towards the age of appliance. But how and where do you get started as a developer?

In this workshop we dive into the AI platform on Microsoft Azure and find out how you as a developer you can use the power of AI in your applications.

We start with pre-build cognitive services and move through the landscape to Azure Machine learning services, where we create a model from scratch and deploy it in a Docker container into the Azure cloud.

In this intimate workshop format, you will have time to get into the details of how to get the most out of Microsoft AI. We will combine the knowledge of the participants in the workshop through sharing of ideas, experiences and discuss the relevant options where relevant.

## The workshop will include the following topics:

* Azure AI basics: what is available on the Azure stack and what to use when
* Cognitive Services, ready to use AI
* Azure Machine Learning Service

At the end of the workshop you can integrate different AI components in your application and have a good overview of what is available in Microsoft AI stack and how to use it.

## Who Is It For?

Everyone who want to learn about AI.


# Azure Passes

### Get an Azure Pass

* Visit [https://aka.ms/ep/azure
  ](<https://aka.ms/ep/azure&#xD;&#xA;>)
* Login with your GitHub account
* Copy code on your screen

**Activate your Azure Pass**

* Navigate to: [www.microsoftazurepass.com](https://www.microsoftazurepass.com/?WT.mc_id=aiml-36264-heboelma)
* Click **start** and follow the on-screen instruction

To activate your Azure Pass you need an Microsoft Account, you can create an account on [www.outlook.com](https://workshops.henkboelman.com/developers-guide-to-azure-ai/workshop/www.outlook.com).


# Azure CLI

To get started we need to setup a few resources in Azure. For this we are going to use the Azure CLI. If you don’t have the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/?WT.mc_id=aiapril-blog-heboelma\&view=gaic-github-latest) installed on your machine you can follow the [tutorial on MS Docs](https://docs.microsoft.com/cli/azure/install-azure-cli?WT.mc_id=aiml-36264-heboelma\&view=azure-cli-latest) here.

* Install on [Windows](https://docs.microsoft.com/cli/azure/install-azure-cli-windows?tabs=azure-cli\&WT.mc_id=aiml-34189-heboelma)
* Install on [macOS](https://docs.microsoft.com/cli/azure/install-azure-cli-macos?WT.mc_id=aiml-36264-heboelma)

> All demos and content have been tested on a Windows PC, however all options should run from macOS and Linux machines as well. Please provide information via an issue or pull request if you have feedback on other operating systems.


# Lab 1 - Environment Setup

## Create a Azure Machine Learning Workspace

To get started we need to setup a few resources in Azure. For this we are going to use the Azure CLI. If you don’t have the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/?WT.mc_id=aiapril-blog-heboelma\&view=gaic-github-latest) installed on your machine you can follow the [tutorial on MS Docs](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?WT.mc_id=gaic-github-heboelma\&view=azure-cli-latest) here.

### Install the Azure Machine Learning CLI extension

To install the machine learning extension, use the following command:

```
az extension add -n azure-cli-ml
```

### Create a resource group

The Azure Machine Learning workspace must be created inside a resource group. You can use an existing resource group or create a new one. To create a new resource group, use the following command. Replace  with the name to use for this resource group. Replace  with the Azure region to use for this resource group:

**Example name and location:**&#x20;

* resource group name: pytorchworkshop
* location: WestEurope

```
az group create --name <resource-group-name> --location <location>
```

### Create the workspace

To create a new workspace where the services are automatically created, use the following command:

```
az ml workspace create -w <workspace-name> -g <resource-group-name>
```

> You can now view your workspace by visiting <https://ml.azure.com>

![Azure Machine Learning studio](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F73f6fb6e259d619e7f5abcb858215210f4a96fdf.png?generation=1597759372555732\&alt=media)

### Create a Compute Cluster

To train our model we need an Azure Machine Learning Compute cluster. To create a new compute cluster, use the following command.

This command will create an Azure Machine Learning Compute cluster with 1 node that is always on and is using STANDARD\_NC6 virtual Machines.

*To speed up the training process you can use a GPU enabled NC6 machine*

```
az ml computetarget create amlcompute -n gpu-cluster --min-nodes 1 --max-nodes 1 --vm-size STANDARD_NC6 -w <workspace-name> -g <resource-group-name>
```

> View your created Azure Machine Learning Compute cluster on <https://ml.azure.com>
>
> *Creating compute can take a few minutes to complete*

![Create Azure Machine Learning Compute](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F4f026a8a3d50afbd06e215add1bf68830b0daa2b.png?generation=1597759372809521\&alt=media)

### Create a Compute instance

To train our model we are going to use a notebook. To run a notebook in Azure Machine Learning studio we need to create a Compute Instance.

*Choose a unique name*

```
az ml computetarget create computeinstance -n <name> --vm-size Standard_D2_V2 -w <workspace-name> -g <resource-group-name>
```

> View your created Azure Machine Learning Compute cluster on <https://ml.azure.com>
>
> *Creating compute can take a few minutes to complete*

![Create Azure Machine Learning Compute Instance](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F01797e1d3b38c4f97754f8ed5e77058da7d47a6d.png?generation=1597759373131966\&alt=media)

### Create a Notebook

* Navigate to the Notebook section in Azure Machine Learning Workspace.&#x20;
* Create a new file with name 'simpsons' and File type 'notebook'

![Create new folder](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F90321df9948ad93cc2b341dd5e906ec0b969dec8.png?generation=1597759372658288\&alt=media)

### Setup completed

If everything went correctly you should be looking at a screen that looks like the one below and see that your notebook is running on your created Compute Instance.

![Create new folder](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Fd2b7a02ddbfae2aea040ed4819300aed226aef70.png?generation=1597759372720246\&alt=media)

[**Continue with lab 2 >**](/developers-guide-to-azure-ai/azure-machine-learning/lab-2)


# Lab 2 - Train your model

In this lab we are going the train a PyTorch Model that can classify Simpsons using the resources we have created in the [previous lab](/developers-guide-to-azure-ai/azure-machine-learning/lab-1).

## 1. Connect to your resources

### Import dependencies

Start with importing dependencies. If you are using a Notebook in Azure Machine Learning Studio, you have all the latest versions install. If you are running your own Jupyter notebook then you have to install the azureml-sdk (pip install azureml-sdk).

* Paste the code below in the first cell and run this cell.

```
import os, random
import azureml
import shutil
import urllib.request
from pathlib import Path
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import AxesGrid
import cv2
import urllib3 
import zipfile

from azureml.core.model import Model, InferenceConfig
from azureml.core import Workspace, Datastore, Experiment, Run, Environment, ScriptRunConfig

from azureml.core.compute import ComputeTarget, AmlCompute, AksCompute, ComputeTarget
from azureml.train.dnn import PyTorch
from azureml.widgets import RunDetails

from azureml.core.webservice import Webservice, AksWebservice, AciWebservice
from azureml.core.dataset import Dataset
from azureml.core.resource_configuration import ResourceConfiguration
from azureml.core.conda_dependencies import CondaDependencies 

# check core SDK version number
print("Azure ML SDK Version: ", azureml.core.VERSION)
```

![Import dependencies](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F3e89d1f56ad5773c7defa605da725160826c10ee.png?generation=1597759379306347\&alt=media)

### Connect to workspace

* Create a new code cell by clicking on the '+ Code' button
* Paste the code below

  ```
  ws = Workspace.from_config()
  print("Connected to workspace: ",ws.name)
  ```
* Run the cell
* Performing the interactive authentication using the link and code provide
* Run the cell again
* It should say: "Connected to workspace: "

### Connect to Azure Machine Learning Compute Cluster

```
compute_target = ComputeTarget(workspace=ws, name="gpu-cluster")
print("Connected to compute target:",compute_target.name)
```

### Connect to the default datastore

```
ds = Datastore.get_default(ws)
print("Connected to datastore:",ds.name)
```

![Connect to resources](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Fea0fa83a1c1778c31bb5e95232b79340500dd80a.png?generation=1597759377867115\&alt=media)

### Create an experiment

```
exp = Experiment(workspace=ws, name='Simpsons-PyTorch')
print("Experiment created:",exp.name)
```

> View your created experiment on: [https://ml.azure.com](https://ml.azure.com/experiments)

## 2. Data

### Download the dataset from Github

```
data_url = "https://github.com/hnky/dataset-lego-figures/raw/master/_download/train-and-validate.zip"
data_path = "./data"
download_path = os.path.join(data_path,"train-and-validate.zip")
if not os.path.exists(data_path):
    os.mkdir(data_path,);
urllib.request.urlretrieve(data_url, filename=download_path)
```

### Unzip the dataset

```
zip_ref = zipfile.ZipFile(download_path, 'r')
zip_ref.extractall(data_path)
zip_ref.close()
print("Data extracted in: {}".format(data_path))

os.remove(download_path)
print("Downloaded file removed: {}".format(download_path))
```

> View your the downloaded dataset: [https://ml.azure.com](https://ml.azure.com/fileexplorerAzNB)
>
> *Choose the refresh button above your folder structure to see data files*

![Dataset](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Ffb6811ca83b548c5f46199c9227645b42907412b.png?generation=1597759379577765\&alt=media)

### Preview the dataset

To take a peak at the images in the dataset paste and the run the code below.

```
from mpl_toolkits.axes_grid1 import AxesGrid
import random
import cv2
import matplotlib.pyplot as plt

path = r"data/train"
random_filenames = []
for tag in os.listdir(path):
    random_filenames.append(path+"/"+tag+"/"+random.choice([
        x for x in os.listdir(os.path.join(path,tag))
        if os.path.isfile(os.path.join(path,tag, x))
    ]))

grid = AxesGrid(plt.figure(1, (20,20)), 111, nrows_ncols=(4, 5), axes_pad=0, label_mode="1")

i = 0
for img_name in random_filenames[0:10]:
    image = cv2.imread(img_name)
    image = cv2.resize(image, (352, 352))
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    # Show image in grid
    grid[i].imshow(image)
    i = i+1
```

![Dataset](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F9120866bed8886424db4daa189854dd0e910f1db.png?generation=1597759378977194\&alt=media)

### Upload the data to the datastore

```
ds.upload(src_dir=data_path, target_path='simpsonslego', overwrite=True, show_progress=True)
```

### Create a dataset from the data in the datastore

```
datastore_paths = [(ds, 'simpsonslego/**')]
simpsons_ds = Dataset.File.from_files(path=datastore_paths)
```

### Register the dataset

```
simpsons_ds.register(workspace=ws,
             name='LegoSimpsons',
             description='Simpsons dataset with Lego Figures',
             create_new_version = True)
```

### Connect to the dataset

```
simpsons_ds = Dataset.get_by_name(ws, name='LegoSimpsons')
```

## Train the model

### Download the training script

```
project_folder = "./trainingscripts"

training_script_url = "https://raw.githubusercontent.com/hnky/DevelopersGuideToAI/master/amls/resources/train.py"

training_script_download_path = os.path.join(project_folder,"train.py")
if not os.path.exists(project_folder):
    os.mkdir(project_folder);
urllib.request.urlretrieve(training_script_url, filename=training_script_download_path)
```

> Refresh your files and validate that 'train.py' is downloaded in the folder 'trainingscripts': [https://ml.azure.com](https://ml.azure.com/fileexplorerAzNB)

![Training Script](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Fee62d3251ac3b58f54ef7e945a5a33146d508740.png?generation=1597759378572779\&alt=media)

```
curated_env_name = 'AzureML-PyTorch-1.6-GPU'

pytorch_env = Environment.get(workspace=ws, name=curated_env_name)
pytorch_env = pytorch_env.clone(new_name='pytorch-1.6-gpu')
```

```
args = [
    '--data-folder', simpsons_ds.as_named_input('simpsons').as_mount(),
    '--num-epochs', 15
]

project_folder = "./trainingscripts"

config = ScriptRunConfig(
    source_directory = project_folder, 
    script = 'train.py', 
    compute_target=compute_target,
    environment = pytorch_env,
    arguments=args,
)
```

### Submit the PyTorch estimator

```
run = exp.submit(config)
```

### Follow the progress of the run

```
RunDetails(run).show()
```

* Click on 'View run details' to view all the details of the run under the experiment.

![Training starting](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Ff59573e3d1b184dddbce1386dde9c5f52e100bc5.png?generation=1597759379146174\&alt=media)

> **This step can take up to 15 minutes to complete**

![Training done](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Fb49f81a6090d90277fb33a04c7cf01b06b2fe1f0.png?generation=1597759378087123\&alt=media)

### Register the model

```
model = run.register_model(model_name='Simpsons-PyTorch',
                           model_path='outputs',
                           model_framework='PyTorch',
                           model_framework_version='1.3',
                           description="Simpsons PyTorch Classifier",
                           tags={'Conference':'Awesome AI Workshop'},
                           resource_configuration=ResourceConfiguration(cpu=1, memory_in_gb=2))

print("Model '{}' version {} registered ".format(model.name,model.version))
```

![Test results](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2Fcd099854de313c592ba03e972301729f076751f3.png?generation=1597759379565495\&alt=media)

> Validate that your model is visible under 'Models': [https://ml.azure.com](https://ml.azure.com/model/list)

## Download and test your model

### Download the model

```
model.download(exist_ok=True)
```

### Download test images

```
test_images_url = "https://github.com/hnky/dataset-lego-figures/raw/master/_download/test-images.zip"
test_images_path = r"./data/test"
test_images_download_path = os.path.join(test_images_path,"test-images.zip")
if not os.path.exists(test_images_path):
    os.mkdir(test_images_path,);
urllib.request.urlretrieve(test_images_url, filename=test_images_download_path)
```

### Unzip test images

```
zip_ref = zipfile.ZipFile(test_images_download_path, 'r')
zip_ref.extractall(test_images_path)
zip_ref.close()
print("Data extracted in: {}".format(test_images_path))

os.remove(test_images_download_path)
print("Downloaded file removed: {}".format(test_images_download_path))
```

### Run the model over the test images

```
import os
import torch
import torch.nn as nn
import torchvision
from torchvision import transforms
import json
import urllib
from PIL import Image

# Load the model
loaded_model = torch.load(os.path.join('outputs','model.pth'), map_location=lambda storage, loc: storage)
loaded_model.eval()

# Load the labels
with open(os.path.join('outputs','labels.txt'), 'rt') as lf:
    global labels
    labels = [l.strip() for l in lf.readlines()]

def scoreImage(image_link):
    # Load the image to predict
    input_image = Image.open(image_link)

    # Pre process
    preprocess = transforms.Compose([
        transforms.Resize(225),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    ])
    input_tensor = preprocess(input_image)
    input_batch = input_tensor.unsqueeze(0)

    # Predict the image
    if torch.cuda.is_available():
        input_batch = input_batch.to('cuda')
        loaded_model.to('cuda')

    with torch.no_grad():
        output = loaded_model(input_batch)

    index = output.data.cpu().numpy().argmax()
    probability = torch.nn.functional.softmax(output[0], dim=0).data.cpu().numpy().max()

    #Return the result
    return {"label": labels[index], "probability": round(probability*100,2)}



path = r"data/test"
grid = AxesGrid(plt.figure(1, (20,20)), 111, nrows_ncols=(4, 5), axes_pad=0, label_mode="1")

i = 0
for img in os.listdir(path):
    
    #Score the image
    result = scoreImage(os.path.join(path,img))
    
    # Download image
    image = cv2.imread(os.path.join(path,img))
    image = cv2.resize(image, (352, 352))
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    
    cv2.rectangle(image, (0,260),(352,352),(255,255,255), -1)
    cv2.putText(image, "{} - {}%".format(result['label'],result['probability']),(10, 300), cv2.FONT_HERSHEY_SIMPLEX, 0.65,(0,0,0),2,cv2.LINE_AA)    
    
    # Show image in grid
    grid[i].imshow(image)
    i = i+1
```

![Test results](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F630f19023d9b05e1942bc18aa67e82a27242a7ed.png?generation=1597759378513330\&alt=media)

[**Continue with lab 3 >**](/developers-guide-to-azure-ai/azure-machine-learning/lab-3)


# Lab 3 - Deploy your model

In this lab we are going to deploy the model wrappend in an API in an Azure Container Instance and sending data to it with postman.

## 1. Deploy to an Azure Container Instance

### Download the scoring script

```
inference_folder = "./inference"

inference_script_url = "https://raw.githubusercontent.com/hnky/DevelopersGuideToAI/master/amls/resources/score.py"
inference_script_download_path = os.path.join(inference_folder,"score.py")
if not os.path.exists(inference_folder):
    os.mkdir(inference_folder);
urllib.request.urlretrieve(inference_script_url, filename=inference_script_download_path)
```

### Create an inference environment

```
inference_env = Environment(name="simpsons-inference")

conda_dep = CondaDependencies()
conda_dep.add_pip_package("azureml-defaults")
conda_dep.add_pip_package("torch")
conda_dep.add_pip_package("torchvision")
conda_dep.add_pip_package("pillow==5.4.1")

inference_env.python.conda_dependencies=conda_dep
```

### Create an Inference config

```
inference_config = InferenceConfig(
    entry_script="inference/score.py", 
    environment=inference_env 
)
```

### Create a Azure Container Instance deployment config

```
deploy_config = AciWebservice.deploy_configuration(
    cpu_cores = model.resource_configuration.cpu, 
    memory_gb = model.resource_configuration.memory_in_gb,
    description='Simpson Lego Classifier'
)
```

### Deploy the model to an ACI

```
aci_service = Model.deploy(ws, 
                name="simpsons-pt-aci", 
                models = [model], 
                inference_config = inference_config, 
                deployment_config = deploy_config, 
                overwrite = True)

aci_service.wait_for_deployment(show_output=True)
print("Scoring endpoint:",aci_service.scoring_uri)
```

> *This step can take up to 10 minutes*
>
> You can find the deployment location from your model back under the model: [https://ml.azure.com](https://ml.azure.com/model/list)&#x20;
>
> *Note: if you don't see it immediately refresh the tab*

![Scoring URL](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F901f3ddb13990e27ebbc042e36e4771e9f63e987.png?generation=1597764446317855\&alt=media)

## 2. Test the model in the API

### Post an image to the endpoint

The easiest way to test your scoring endpoint is the code below.

```
image_uri = "https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg"
result = aci_service.run(input_data=json.dumps({ "url": image_uri}))
print(result)
```

### Use Postman

* Download [Postman to your local machine](https://www.postman.com/)
* Get the scoring uri

  ```
  print("Scoring endpoint:",aci_service.scoring_uri)
  ```
* Create a new request in Postman
  * POST request
  * Put scoring URL in call
* Send a raw body with the JSON below. *Make sure JSON is selected in orange next to the raw radio button*

  ```
  { "url": "https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg"}
  ```

![Scoring URL](https://475228360-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MF0ro-V-CjhNEKidSUU%2Fsync%2F4b3e3e2c454ed5d79313a21bb13be0599386eca5.png?generation=1597764446492453\&alt=media)

### Try other images

```
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Krusty.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Flanders.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Homer.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Lisa.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/marge.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Milhouse.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/MrBurns.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Wiggum.jpg
```

**End**


# Lab 1 - Computer Vision

Analyze an image by sending it to the Computer Vision API

Microsoft Azure Cognitive Services contain some pre-built models for the most typical tasks, such as object detection in pictures, speech recognition and synthesis, sentiment analysis and so on. Let us test the [**Computer Vision API**](https://azure.microsoft.com/services/cognitive-services/computer-vision/?WT.mc_id=gaic-github-heboelma) service to see if it can recognize some specific objects in one particular problem domain.

Suppose we need to create an application that recognizes Simpson Lego Figures:

We will use the [Simpsons Lego Figure Dataset](https://github.com/hnky/dataset-lego-figures) from Henk Boelman.

Let us start by looking at how pre-trained [Computer Vision cognitive service](https://azure.microsoft.com/services/cognitive-services/computer-vision/?WT.mc_id=gaic-github-heboelma) can see our images:

### 1. Setup the Azure Resources

#### Create a resource group

The Computer Vision Endpoint must be created inside a resource group. You can use an existing resource group or create a new one. To create a new resource group, use the following command. Replace  <\<resource-group-name>> with your name to use for this resource group. Replace <\<location>>  with the Azure region to use for this resource group.

```
az group create --name <resource-group-name> --location <location>
```

#### Create the Cognitive Service Account

Run the command below to create a free Computer Vision Endpoint. Replace <\<resource-group-name>> with the name you used above and use the same location. Replace <\<name>> with a name for your resource like: my-computervision.

```
az cognitiveservices account create --name <name> --kind ComputerVision --sku F0 --resource-group <resource-group-name> --location <location> --yes
```

#### Get the endpoint details from the Cognitive Service Account

To get the endpoint from the Cognitive service account run the CLI command below. Replace <\<name>> and <\<resource-group-name>> with the names used above.&#x20;

**Get the API keys**

```
az cognitiveservices account keys list --name <name> --resource-group <resource-group-name> 
```

**Get the endpoint URL**

```
az cognitiveservices account show --name <name> --resource-group <resource-group-name> -o json --query properties.endpoint
```

#### At this point you should have:

\- An endpoint URL looking like this: https\://\<region>.api.cognitive.microsoft.com/ \
\- 2 keys looking like this: 06a611d19f4f4a88a03f3b552a5d2379

### 2. Test the Computer Vision Endpoint

#### Using the command line

Run the command below to test the endpoint for an image with Bart Simpson on it.

```
curl -H "Ocp-Apim-Subscription-Key: <<key>>" -H "Content-Type: application/json" "<<endpoint-url>>/vision/v3.2/analyze?visualFeatures=Description" -d "{\"url\":\"https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg\"}"
```

The response should look like:\
`{`\
`"description":{` \
&#x20;   `"tags":[ "text", "indoor", "toy" ],` \
`"captions":[ {` \
&#x20;   `"text": "a hand holding a toy",` \
&#x20;   `"confidence":0.4660031199455261` \
`}]}}`

#### Using a web interface

Navigate to: <https://cgntv-cv.azurewebsites.net/>\
Enter the endpoint URL, the API key, a link to an image and click 'analyze image'. This website shows all the information that comes back from the Computer Vision API.

> **Tip: Try it with your own images!**

### 3. Conclusion

In this lab you have created a Computer Vision Endpoint in Azure and send images to the endpoint using the command line and a through a visual interface.

While some of the objects (such as Toy) can be recognized by the pre-trained model, more specialized objects (like this is Bart Simpson or Marge Simpson) are not determined correctly.

## Resources

* <https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/>
* <https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account-cli?tabs=windows>

[**Continue with lab 2 >**](/developers-guide-to-azure-ai/cognitive-services/lab-2)


# Lab 2 - Custom Vision

Create a custom vision model using the Microsoft Custom Vision service.

Using Microsoft Azure Custom Vision service you can start to build your own personalized image classification and object detection algorithms with very little code. In this exercise we will create a Lego Simpsons classification model.

#### Lab requirements

* Azure CLI
* Python 3.6+ environment

## 1. Setup the Azure Resources

### Create a resource group

The Custom Vision Endpoint must be created inside a resource group. You can use an existing resource group or create a new one.&#x20;

To create a new resource group, use the following command. Replace  <\<resource-group-name>> with your name to use for this resource group. Replace <\<location>>  with the Azure region to use for this resource group.

```
az group create --name <resource-group-name> --location <location>
```

### Create the Cognitive Service Account

Run the command below to create a free Custom Vision Training Endpoint. Replace <\<resource-group-name>> with the name you used above and use the same location. Replace <\<name>> with a name for your resource like: my-custom-vision-training.

```
az cognitiveservices account create --name <name> --kind CustomVision.Training --sku F0 --resource-group <resource-group-name> --location <location> --yes
```

### Get the endpoint details from the Cognitive Service Account

To get the endpoint from the Cognitive service account run the CLI command below. Replace <\<name>> and <\<resource-group-name>> with the names used above.&#x20;

**Get the API keys**

```
az cognitiveservices account keys list --name <name> --resource-group <resource-group-name> 
```

**Get the endpoint URL**

```
az cognitiveservices account show --name <name> --resource-group <resource-group-name> -o json --query properties.endpoint
```

#### At this point you should have:

\- An endpoint URL looking like this: https\://\<region>.api.cognitive.microsoft.com/ \
\- 2 keys looking like this: 06a611d19f4f4a88a03f3b552a5d2379

## 2. Create a Custom Vision Model

Every Machine Learning journey starts with a question you want to have answered. For this example, you are going to answer the question: Is it a Homer or a Marge Lego figure.

### Download the dataset

Now that we know what to ask the model, we can go on to the next requirement; that is data. Our model is going to be a classification model, meaning the model will look at the picture and scores the pictures against the different classes. So, the output will be I’m 70% confident this is Homer and 1% confident that this is Marge. By taking the class with the highest score and setting a minimum threshold for the confidence score we know what is on the picture.

I have created a dataset for you with 50 pictures of a Homer Simpson Lego figure and 50 pictures of a Marge Simpsons Lego figure. I have taken the photos with a few things in mind, used a lot of different backgrounds and took the photos from different angles. I made sure the only object in the photo was Homer or Marge and the quality of the photos was somehow the consistent.

Use the Python code below to download and extract the dataset to the folder "LegoSimpsons" or [download the dataset](https://github.com/hnky/dataset-lego-figures/raw/master/_download/simpsons-lego-dataset.zip) and extract it manually in the folder "LegoSimpsons".

```
import os
import urllib.request
import zipfile

# Download the dataset from Github
data_url = "https://github.com/hnky/dataset-lego-figures/raw/master/_download/simpsons-lego-dataset.zip"
data_path = "./LegoSimpsons"
download_path = os.path.join(data_path,"simpsons-lego-dataset.zip")
if not os.path.exists(data_path):
    os.mkdir(data_path);
urllib.request.urlretrieve(data_url, filename=download_path)

# Unzip the dataset
zip_ref = zipfile.ZipFile(download_path, 'r')
zip_ref.extractall(data_path)
zip_ref.close()
print("Data extracted in: {}".format(data_path))

os.remove(download_path)
print("Downloaded file removed: {}".format(download_path))
```

### Create a Custom Vision Project

For the training we are going the use the [Custom Vision Service Python SDK](https://docs.microsoft.com/python/api/overview/azure/cognitiveservices/customvision?view=azure-python\&WT.mc_id=aiml-0000-heboelma), you can install this package using pip.

```
pip install azure-cognitiveservices-vision-customvision
```

Create a new Python file called 'train.py' in Visual Studio code or your favorite editor and start adding code or use a Jupyter Notebook.

Start with importing the packages needed.

```
from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient
from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateEntry
```

Next, create variables for the Custom Vision endpoint, Custom Vision training key and the location where the training images are stored.

```
cv_endpoint = "https://westeurope.api.cognitive.microsoft.com"
training_key = "<INSERT TRAINING KEY>"
training_images = "LegoSimpsons/TrainingImages"
```

To start with the training, we need to create a Training Client. This method takes as input the endpoint and the training key.

```
trainer = CustomVisionTrainingClient(training_key, endpoint= cv_endpoint)
```

Now you are ready to create your first project. The project takes a name and domain as input, the name can be anything. The domain is a different story. You can ask for a list of all possible domains and choose the one closest to what you are trying to accomplish. For instance if you are trying to classify food you pick the domain “Food” or “Landmarks” for landmarks. Use the code below to show all domains.

```
for domain in trainer.get_domains():
  print(domain.id, "\t", domain.name) 
```

You might notice that some domains have the word “Compact” behind them. If this is the case it means the Azure Custom Vision Service will create a smaller model, which you will be able to export and run locally on your mobile phone or desktop.

Let’s create a new project with the domain set to “General Compact”.

```
project = trainer.create_project("Lego - Simpsons - v1","0732100f-1a38-4e49-a514-c9b44c697ab5")
```

### Upload and tag the images

Next you need to create tags, these tags are the same as classes mentioned above. When you have created a few tags we can tag images with them and upload the images to the Azure Custom Vision Service.

Our images are sorted per tag/class in a folder. All the photos of Marge are in the folder named 'Marge' and all the images of Homer are in the folder named 'Homer'.

In the code below we do the following steps:

* We open the directory containing the folders with training images.
* Loop through all the directories found in this folder
* Create a new tag with the folder name
* Open the folder containing the images
* Create, for every image in that folder, an ImageFileEntry that contains the filename, file content and the tag.
* Add this ImageFileEntry to a list.

```
image_list = []
directories = os.listdir(training_images)

for tagName in directories:
 	tag = trainer.create_tag(project.id, tagName)
 	images = os.listdir(os.path.join(training_images,tagName))
 	for img in images:
 		with open(os.path.join(training_images,tagName,img), "rb") as image_contents:
 			image_list.append(ImageFileCreateEntry(name=img, contents=image_contents.read(), tag_ids=[tag.id]))  
```

Now you have a list that contains all tagged images. So far no images have been added to the Azure Custom Vision service, only the tags have been created.

Uploading images goes in batches with a max size of 64 images per batch. Our dataset is 300+ images big, so first we need to split the list into chunks of 64 images.

```
def chunks(l, n):
 	for i in range(0, len(l), n):
 		yield l[i:i + n]
batchedImages = chunks(image_list, 64)
```

Now we have our images split in batches of 64, we can upload them batch by batch to the Azure Custom Vision Service. *Note: This can take a while!*

```
for batchOfImages in batchedImages:
    upload_result = trainer.create_images_from_files(project.id, ImageFileCreateBatch(images=batchOfImages))
    if not upload_result.is_batch_successful:
        print("Image batch upload failed.")
        for image in upload_result.images:
            print("Image status: ", image.status)
    else:
        print("Batch uploaded successfully")
```

### Train the classification model

From this point, there are only two steps remaining before you can access the model through an API endpoint. First you need to train the model and finally you must publish the model, so it is accessible through a prediction API. The training can take a while, so you can create a while loop after the train request that checks the status of the model training every second.

```
print ("Training...")
iteration = trainer.train_project(project.id)
while (iteration.status != "Completed"):
    iteration = trainer.get_iteration(project.id, iteration.id)
    print ("Training status: " + iteration.status)
    print ("Waiting 10 seconds...")
    time.sleep(10)
```

Now you have successfully trained your model!

A small recap of what have we done:

* You created an Azure Resource group containing an Azure Custom Vision service training and prediction endpoint
* You have created a new Project
* In that project you have created tags
* You have uploaded images in batches of 64 and tagged them
* You have trained an iteration of your model
* You have published the iteration to a prediction endpoint

## 3. Test your model

To test our model we are going to export our model in the ONNX format, download the model and run it locally.

### Export the iteration to an ONNX model

```
platform = "ONNX"
flavor = "ONNX12"
iteration_id = iteration.id 
project_id = project.id
export = trainer.export_iteration(project_id, iteration_id , platform, flavor, raw=False)
```

```
while (export.status == "Exporting"):
    print ("Waiting 10 seconds...")
    time.sleep(10)
    exports = trainer.get_exports(project.id, iteration_id)
    # Locate the export for this iteration and check its status  
    for e in exports:
        if e.platform == export.platform and e.flavor == export.flavor:
            export = e
            break
    print("Export status is: ", export.status)
```

### Download and unzip the export

```
import os
import requests
import zipfile

if export.status == "Done":
    # Success, now we can download it
    export_file = requests.get(export.download_uri)
    with open("export.zip", "wb") as file:
        file.write(export_file.content)
        
# Unzip the downloaded export
if not os.path.exists("./model"):
    os.mkdir("./model");
zip_ref = zipfile.ZipFile("export.zip", 'r')
zip_ref.extractall("./model")
zip_ref.close()
print("Data extracted in: ./model")
```

### Predict an image

```
pip install onnxruntime
pip install "pillow!=8.3.0" #pillow 8.3.0 and numpy don't work together
```

```
import onnxruntime as nxrun
import numpy as np
import PIL

image_filepath = "./Bart.jpg"
model_path = "./model/model.onnx"

sess = nxrun.InferenceSession(model_path)
image = PIL.Image.open(image_filepath).resize([224,224])
input_array = np.array(image, dtype=np.float32)[np.newaxis, :, :, :]
input_array = input_array.transpose((0, 3, 1, 2))[:, (2, 1, 0), :, :]

input_name = sess.get_inputs()[0].name
outputs = sess.run(None, {input_name: input_array.astype(np.float32)})

print("Label: " + outputs[0][0][0])
print("Score: " + str(outputs[1][0][outputs[0][0][0]]))
```

**Great work!** \
You have created your specialized Simpsons classification model using the Azure Custom Vision Service.

## Resources

* <https://github.com/Azure-Samples/cognitive-services-onnx-customvision-sample>
* [https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/quickstarts/image-classification](https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/quickstarts/image-classification?tabs=visual-studio\&pivots=programming-language-python)
* <https://github.com/Azure-Samples/customvision-export-samples>
* [https://docs.microsoft.com/python/api/overview/azure/cognitiveservices/customvision?view=azure-python](https://docs.microsoft.com/en-us/python/api/overview/azure/cognitiveservices/customvision?view=azure-python)

[**Continue with lab 3 >**](broken://pages/-MF0sHVyDKDEo2EVEd3N)


# Slides

{% embed url="<https://speakerdeck.com/hnky/workshop-build-and-deploy-pytorch-models-with-azure-machine-learning>" %}


# PyTorch Fundamentals

Learn the fundamentals of deep learning with PyTorch! This beginner friendly learning path will introduce key concepts to building machine learning models in multiple domains include speech, vision, and natural language processing.

[https://aka.ms/ep/pytorch <br>](<https://aka.ms/ep/pytorch&#xD;&#xA;>)


# Microsoft Learn

Whether you're just starting or an experienced professional, our hands-on approach helps you arrive at your goals faster, with more confidence and at your own pace.

* [Create machine learning models](https://docs.microsoft.com/learn/paths/create-machine-learn-models/?WT.mc_id=aiml-36264-heboelma)
* [Create no-code predictive models with Azure Machine Learning](https://docs.microsoft.com/learn/paths/create-no-code-predictive-models-azure-machine-learning/?WT.mc_id=aiml-36264-heboelma)
* [Build and operate machine learning solutions with Azure Machine Learning](https://docs.microsoft.com/learn/paths/build-ai-solutions-with-azure-ml-service/?WT.mc_id=aiml-36264-heboelma)
* [PyTorch Fundamentals](https://docs.microsoft.com/learn/paths/pytorch-fundamentals/?WT.mc_id=aiml-36264-heboelma)