Lab 2 - Custom Vision

Using Microsoft Azure Custom Vision service you can start to build your own personalised image classification and object detection algorithms with very little code. In this exercise we will create a Lego Simpsons classification algorithm.

Create Resource Group

First create a Resource Group.

  • Go to the Azure Portal main dashboard.

  • Click 'Create a Resource' in the top left

  • Search for 'Resource group'

  • Enter details to create:

    • A name for the resource group

    • Select the location

    • Click Create

Resource Group Details

Create Custom Vision instance

Now create a Custom Vision instance in your Azure account.

  • Go to your created Resource group

  • Click +Add

  • Search for Custom Vision

  • Click Create

  • Enter details to create:

    • A name for the service

    • Select your subscription

    • Select the data centre location (in this example West Europe, but you can select your own region)

    • Choose the S0 tier for both 'Prediction pricing tier' and Training pricing tier. F0 is possible, but gives you an error with the logic app part.

    • Select your created Resource group and make sure it is in the same data centre location (in this case 'globalaibootcamp' in West Europe

    • Click Create

Build Classifier

Now we can build our classifier, navigate to https://www.customvision.ai and choose sign in. Sign in with your Azure credentials account.

Accept the terms and conditions box to continue.

Create Project

Once loaded choose 'New Project' which opens a window to enter details:

  • Name: choose a suitable name

  • Description: add a description of the classifier (example shown in image below)

  • Resource Group: choose the resource group you created your custom vision service in (example: workshop[SO])

  • Project Types: Classification

  • Classification Types: Multiclass (Single tag per image)

  • Domains: Retail (compact)

  • Export Capabilities: Basic platforms

Create Custom Vision Project

Click on 'Create Project' and you will land on an empty workspace.

Add Images

Now you can start adding images and assigning them tags to create our image classifier.

  • Download and unzip the Simpsons Lego Dataset

  • In the top left, select 'Add images', browse for the first folder of images - Bart Simpson - and select all the images in the folder.

  • Add the tag 'Bart Simpson' to the Bart Simpson images and select 'Upload files'

Once successful, you receive a confirmation message and you should see that your images are now available in the workspace.

Upload images of drills

Now complete the same steps of uploading and tagging images for the other Simpsons in the folder. For each type of Simpson:

  • Click 'Add images'

  • Select all the images

  • Add the class label (Lisa, Marge, etc.)

  • Choose upload

  • Confirm images uploaded into the workspace

Now you should have all categories uploaded and on the left hand side you can see your Simpsons classes and you can filter depending on type of Simpson image.

Train Model

Now you are ready to train your algorithm on the Simpsons image data you have uploaded. Select the green 'Train' button in the top right corner. For this demo, you can use the "Fast Training" option.

Once the training process is complete it will take you to the Performance tab. Here you will receive machine learning evaluation metrics for your model. Here you algo get information regarding the class imbalance, as some Simpsons have less images than others.

Evaluation Medtrics

Test Model

Now you have a model, you need to test the model. Choose the 'Quick Test' button in the top right (next to the train button) this will open a window where you can browse for a local image or enter a web URL.

Use one of the image links below (images the model have not been trained on) and paste the link in the Image URL field. The image will be analysed and a result returned of what Simpson the model thinks it is (prediction tag) and the models confidence of its result (prediction probability).

https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg
Quick Test

Repeat this process for other image in the test folder, or search online for other images to see how the model performs.

https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Krusty.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Bart.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Flanders.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Homer.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Lisa.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/marge.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Milhouse.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/MrBurns.jpg
https://raw.githubusercontent.com/hnky/dataset-lego-figures/master/_test/Wiggum.jpg

Retrain Model

If you click on the 'Predictions' tab on the top toolbar - you should see all the test images you have submitted. This section is for re-training, as you get new data you can add this to your model to improve its performance. The images are ordered by importance - the image, which if classified correctly, will add the most new information to the model is listed first. Whereas the last image might be very similar to other images already learnt by the model so this is less important to classify correctly.

To add these images to the model - select the first image, review the results the model provided and then in the 'My Tags' box enter the correct tag and click 'save and close'.

Add Re-training Tag

This image will disappear from the your predictions workspace and be added to the training images workspace. Once you add a few new images and tags you can re-train the model to see if there are improvements.

Publish Model

To use this model within applications you need the prediction details. Therefore, you have to go to the Performance tab from the top bar, click the Publish button.

Prediction model

Please provide a name for your model and select the Prediction resource, and click on Publish. Please take notice of you Publication Resource, which you need in the second task.

Prediction model resource

You can now select the Prediction URL button to gain all information you need to create a Postman call to your API.

Prediction model URL

Use Model with Postman

Now create a new request.

You can use the prior info to set the URL, the Header and the Body (using both an image or an image URL):

Postman Header
Postman Body

Now click on Send... Which Simpson did you upload?

Postman Result

Great work! you have created your specialised Simpsons classification model using the Azure Custom Vision Service

Continue with lab 3 >