ml5.js Image Classifier
A live example of the code: https://dmarcisovska.github.io/ml5js-image-classifier/
Github code: https://github.com/dmarcisovska/ml5js-image-classifier
I was thinking of new JavaScript project ideas I could do and thought it would be really neat to try out TensorFlow.js. Every time I went to Google I/O, that is what got me most excited. So I headed over to TensowFlow.js and found out about ml5.js which has pre-trained models. I thought this would be a good way to ease into TensorFlow.js.
For my first project I wanted to try something simple, so I decided to create a ml5.js image classifier. You can upload an image to it, and the ml5.js image classifier will classify the image for you and give you a percentage confidence level of the image. I am using the MobileNet pre-trained model for the ml5.js image classifier.
HTML
To set up the HTML, I added in a link needed to run ml5.js. I also added in Google Font and Bootstrap links for styling. Inside the body, I added an input element to be used to upload the image that will be classified. The input element is linked to imageUpload function using onchange. I created a div with an id of a gallery, which I will use to display the photo the user uploads. The gallery contains a placeholder for the user to upload an image. The gallery also contains blank paragraph tags which will be used to house the image’s classified name — using an id of item, and the confidence level the pre-trained MobileNet model has for the image — using an id of confidence.
<!DOCTYPE html>
<html lang="en">
<head>
<title>Image Classifier</title>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="icon" type="image/png" href="assets/denisa-favicon.png">
<link href='https://fonts.googleapis.com/css?family=Lato:300' rel='stylesheet' type='text/css'>
<link href="https://fonts.googleapis.com/css2?family=Permanent+Marker&display=swap" rel="stylesheet">
<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js" defer></script>
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous"><link rel="stylesheet" href="styles.css" />
<script src="scripts.js" defer></script>
</head>
<body><div class="container">
<div class="folded-corner mt-5">
<h3>Image Classifier</h3>
<hr>
<p> Use the button below to upload a file and have your photo classified using a pre-trained model. </p>
<input class="input" type="file" accept="image/*" onchange="imageUpload(this.files)"></div><div id="gallery">
<img class="image" src="">
<p id="item"> </p>
<p id="confidence" class="pb-0 mb-0"> </p>
</div>
</div><!-- jQuery first, then Popper.js, then Bootstrap JS -->
<script src="https://code.jquery.com/jquery-3.4.1.slim.min.js" integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js" integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js" integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous"></script></body>
</html>
CSS
I wanted the page to look like someone was pinning up their own photo on a bulletin board. To do this, I added in some CSS to give the page a wood background. I wanted the photo displayed to look like a polaroid so I added in some padding and shadows behind the image. I also added in CSS to rotate the photo, and I used permanent marker font from Google Fonts to display underneath the photo. I wanted the top heading on the page to look like a folded paper corner so I added in CSS to make it look folded.
body {
background: url("assets/wood.jpg") no-repeat fixed 0 0 / cover;
font-family: 'Lato', sans-serif;
color: white;
}hr {
display: block;
border: 0;
border-top: 1px solid #ccc;
margin: 20px auto 25px auto;
}#gallery {
padding: 10px 10px 40px 10px;
background: #eee;
border: 1px solid #fff;
max-width: 625px;
box-shadow: 0px 2px 15px #333;
-webkit-box-shadow: 0px 2px 15px #333;
-moz-box-shadow: 0px 2px 15px #333;
-webkit-transform: rotate(-5deg);
-moz-transform: rotate(-5deg);
font-family: 'Permanent Marker', cursive;
color: black;
font-size: 32px;
margin: 50px auto 50px auto;
display: none;
}@media screen and (max-width: 1000px) {
.folded-corner {
width: 100% !important;
}
}#gallery img {
width: 100%;
}#gallery p {
font-size: 32px;
}p {
font-size: 20px;
}.folded-corner {
position: relative;
width: 50%;
padding: 1em 1.5em;
margin: 0 auto;
color: #fff;
background: #47a79a;
overflow: hidden;
}.folded-corner:before {
content: "";
position: absolute;
top: 0;
right: 0;
border-width: 0 20px 20px 0;
border-style: solid;
border-color: #15253c #15253c #308277 #308277;
-webkit-box-shadow: 0 1px 1px rgba(0,0,0,0.3), -1px 1px 1px rgba(0,0,0,0.2);
-moz-box-shadow: 0 1px 1px rgba(0,0,0,0.3), -1px 1px 1px rgba(0,0,0,0.2);
box-shadow: 0 1px 1px rgba(0,0,0,0.3), -1px 1px 1px rgba(0,0,0,0.2);
display: block;
width: 0;
}
JavaScript
I first initialized the imageClassifier method with the MobileNet pre-trained model. I then grabbed the elements image, item, confidence and gallery by using document.getElementById. I created an async function called getResults that waits until the image classifier returns its results in the form of an array. I then grabbed the first item of the results array — the label and the confidence level and attached those to my html elements. Lastly, I created a function called imageUpload. Upon uploading of the image, I grab the image location and call the getResults function to classify the image.
const classifier = ml5.imageClassifier("MobileNet", modelLoaded);function modelLoaded() {
console.log('Model Loaded!');
}const image = document.querySelector(".image");
const item = document.getElementById("item");
const confidence = document.getElementById("confidence");
const gallery = document.getElementById("gallery");async function getResults() {
const results = await classifier.classify(image);
gallery.style.display = "block";
item.innerText = "Item - " + results[0].label;
confidence.innerText = "Confidence Level - " + results[0].confidence.toFixed(2) * 100 + "%";
}function imageUpload(files) {
image.src = URL.createObjectURL(files[0]);
setTimeout(getResults, 50);
}
