Bildegjenkjenning gjort enkelt. Slik gjenkjenner du en bestemt gjenstand ved hjelp av et kamera.

• Utfordringen
• Litt teori
• Hvor starter jeg?
• La oss kjøre bildegjenkjenning
• Bonus: Video

Utfordringen

Denne uken har jeg fått utfordringen å foreslå en løsning på følgende problem:

“Med en enkel datamaskin, som en Raspberry Pi, et kamera og et fuglebrett, lag et program som kan artsbestemme hvilke fugler som er innom brettet og spiser i løpet av en dag.”

Siden spørsmålet var av mer teoretisk art, bestemte jeg meg for å begrense dette innlegget til å introdusere byggeblokkene og gi en forklaring på hvordan du typisk kan bygge et slikt system.

Å gjenkjenne gjenstander er en ganske vanlig oppgave nå for tiden, og den har blitt løst på flere forskjellige måter, og med forskjellige tilnærminger. La oss se på hvordan dette virker.

TL;DR Det fins noen tradisjonelle måter å lage bilde- og gjenstandsgjenkjenning på, som i OpenCV, og det fins noen teknikker basert på Deep Learning, som i TensorFlow. Vil du vite mer? Fortsett å lese.

Litt teori

De mer tradisjonelle prinsippene bak OpenCV for å oppdage er godt beskrevet i dette blogginnlegget, så vel som i denne veiledningen for å gjenkjenne (hold dere fast!) katter på bilder.

Mens dette innlegget forklarer hvordan bildegjenkjenning virker i TensorFlow, som bruker en modell kalt dypt konvolverende nevrale nettverk, vil det også la deg trene opp din egen bildeklassifiserer, slik som OpenCV.

Hvor starter jeg?

I utgangspunktet har du to valg – du bør enten lære opp din egen modell eller finne en modell som har blitt lært opp av noen andre, og du vil kanskje alltid starte med å se etter en forhåndsopplært modell. Men jo mer spesifiserte bildeklassifiseringskravene dine er, jo høyere er sjansen for at du må lære opp din egen modell. Dette vil være det samme enten du velger OpenCV eller TensorFlow.

Siden vi vil kjøre modellen på en relativt enkel datamaskin, kan du vurdere å gjøre prosesseringen i skyen. Det bør imidlertid også være mulig å kjøre OpenCV og TensorFlow på den nyeste Raspberry Pi.

La oss kjøre bildegjenkjenning

Nå vet du litt om teorien. La oss så ta en rask titt på hvordan vi kan lære opp modellene i våre behov.

TL;DR Måten dette virker på, er at du viser modellen ganske mange bilder av en gjenstand, og en tilsvarende mengde bilder uten gjenstanden på.

La oss si at du vil bruke modellen til å gjenkjenne fugler utenfor huset ditt i Norge. Et godt utgangspunkt vil da være å få en liste over de typiske artene som du mest sannsynlig ser i hagen din, og samle så mange bilder av hver art som mulig.

Her er hva du kan gjøre:

1. Start med å finne en liste over norske fugler på Wikipedia, eller Store Norske Leksikon.
2. Søk på nettet etter bilder av hver fugleart. Det kan være du vil automatisere dette arbeidet, og sørg for at du plukker ut bilder med de riktige opphavsrettstillatelsene.
3. Bruk disse bildene i opplæringen av din modell.
4. Sett opp din Raspberry Pi med et kamera og fuglebrett, og gjør deg klar til å identifisere fuglearter. Du bør optimalisere programvaren slik at bildeklassifiseringen ikke skjer hele tiden, men kun når man ser en bevegelse.

Pro tip: Det kan også være litt utfordrende å ta gode bilder av våre fjærkledde venner, så sørg for at kameraet ditt er godt plassert, og materen er på et godt opplyst sted. Uten å forstyrre dyrelivet, så klart.

Lurer du på om lignende systemer har blitt implementert eller om det er mulig? Svaret er ja. Her er det noen lenker til inspirasjon for videre lesning:

• En fugleklassifiserer med tensorflow
• Trene opp en fugleklassifiserer med Tensorflow og TFLearn
• Kjøretøyklassifisering og -telling med OpenCV
• Blomstergjenkjenning i naturen (på dansk)
• Maskinlæring er morsomt! Del 3: Dypt konvolverende nevrale nettverk. (Ta også gjerne en titt på de andre delene i denne artikkelserien.)

Prøv nå å sette alt dette sammen, og fortell meg gjerne hvordan det går!

Lykke til!

Bonus: Video

En video vi har laget av mine tegninger og en kort forklaring på hvordan dette virker.

Image recognition made simple. How to recognise a specific object from a camera feed?

• The Challenge
• A Tiny Bit of Theory
• Where to Start?
• Doing Image Recognition

The Challenge

This week I have been given the challenge to suggest a solution to the following problem:

“Given a simple computer, like a Raspberry Pi, a camera and a bird feeder, create a system that can identify the birds feeding there throughout the day.”

Since the question was more of a theoretical kind, I decided to limit this post to introducing the building blocks and giving an explanation of how you would typically build a system like that.

Recognising objects is a pretty common task these days, and it has been solved in quite a few ways by different approaches. Let’s have a look at how this works.

TL;DR There are some traditional ways of doing image recognition and object detection, like in OpenCV, and there are some techniques based on Deep Learning object detection, like in TensorFlow. Want to know more? Keep on reading.

A Tiny Bit of Theory

The more traditional principles behind OpenCV are well described in this blog post, as well as in this tutorial for detecting (brace yourselves!) cats in images.

While this post explains how image recognition works in TensorFlow that is using a model called a deep convolutional neural network. Like OpenCV, it will also let you train your own image classifier.

Where to Start?

Basically, you have two choices – you should either train your own model, or find a model that has been trained by someone else, and you might always want to start by looking for a pre-trained model. However, the more specific your image classification requirements are, the higher are chances that you will need to train your own model. This will be the same whether you go for OpenCV or TensorFlow.

Since we want to run the model on a relatively low-end computer, you might consider doing the processing in the Cloud. However, it should also be possible to run OpenCV and TensorFlow on the latest Raspberry Pis.

Doing Image Recognition

By now, you should know a bit about the theory. So, let’s have a quick look into how we can train the models for our needs.

TL;DR The way this works, is that you feed the model with quite a few pictures of an object, and the similar amount of images without that object.

Say you want to use the model to recognize birds outside your house in Norway. A good starting point would be to get a list of the typical species you are most likely to see in your backyard and collect as many pictures of each type as possible.

Here is what you can do:

1. Start by looking at Wikipedia for a list of the Norwegian birds, or Norwegian Encyclopaedia (in Norwegian).
2. Search the web for the images of each bird type. You might want to automate that task and make sure you are picking images with the right copyright permissions.
3. Use those images to train your model.
4. Set up your Raspberry Pi with a camera and the bird feeder, and get ready to identify. You might want to optimise the software not to do the image classification all the time, but only when movement is detected.

Pro tip: It might also be a bit challenging to take a good picture of our feathered friends, so make sure your camera is well-placed, and the feeder is in a well-lit location. Obviously, without disturbing the wildlife.

In case you wonder if similar systems have been implemented, or if it is even possible. The answer is yes. I will provide you with some links to inspire for further reading:

• A Bird Classifier with Tensorflow
• Training a Bird Classifier with Tensorflow and TFLearn
• Vehicle classification and Counting with OpenCV
• Wild Flower Detection (in Danish)
• Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks. (Also, check out the other parts of the article series.)

Now, try putting it all together, and let me know how it goes!

Good luck!

• Motivation
• Building Blocks
• Now what?

Motivation

Playing around with Internet of Things (IoT) devices provides several advantages. Apart from the obvious ability to easily prototype systems tailored to your specific needs, it can also give you a more engaging way to gain insight into the tools and infrastructure that usually comes with IoT devices, and is relevant for most of the modern developers:

• Cloud
• Distributed systems
• Data analysis
• Data visualization
• Machine learning and prediction models

This post is meant to give you a simple and minimal set-up to get you started playing around with IoT. I would like to show you just a few devices I have used and to provide you with the code examples for them.

Building Blocks

A Computer Brain

First, we will need something to orchestrate the events, collect the data and communicate with other services, like Cloud services or other machines. It can be a Raspberry Pi, or any other (preferably small) computer. It can also be Arduino, NodeMcu or similar.

In my case, it is a Raspberry Pi 2 Model B with Wifi and Bluetooth 4.0 (BTLE) adapters. The Pi will be running Raspbian – a computer operating system, based on Debian (Linux).

Sensors

Now we need some sensors. It can be anything – sensors connected directly to the Raspberry Pis GPIO pins, or separate sensors that can deliver information over the main machine for collection and processing.

I have good experience using SensorTag from Texas Instruments. It uses Bluetooth Low Energy for communication and offers quite a few sensors and hardware buttons, such as:

• Infrared and Ambient Temperature Sensor
• Ambient Light Sensor
• Humidity Sensor
• Barometric Pressure Sensor
• 9-axis Motion Tracking Device – Accelerometer, Gyroscope, and Compass
• Magnet Sensor
• Hardware buttons

Getting started code: You will find some code for getting started with Node.js and SensorTag with examples and references to documentation here.

Other devices

After setting up a computer to orchestrate the events and getting communication with some sensors up and running, we can add more devices that we can control and/or use to collect more data. In the sections below, I have mentioned some of the devices I have been working with, as well as linking to some code examples to get you started.

Razberry (Z-Wave)

Since we are using Raspberry Pi, we can extend it even more by using Razberry platform, that brings all the components needed to turn a Raspberry Pi board into a fully operational and inexpensive Z-Wave gateway. Z-Wave is a wireless communication technology often used for smart homes.

Getting started code: In this repo I have provided some information about the REST APIs and the commands that can be sent through them.

Micro:bit

You can also connect some other devices, like this ARM-based embedded system designed by the BBC for use in computer education.

Getting started code: My previous post gives some pointers for getting started, links to the editor and documentation. It also contains some example implementations.

It should be also possible to connect a micro:bit to a computer or a phone via Bluetooth to send or receive data to or from it.

Other

Previously, I have also used devices like Zumo robots and drones:

• Zumo robots
• Parrot drones

Just keep on adding devices, sensors, and components as you wish. Sky is the limit!

Now what?

Now that you have connected a bunch of sensors and started collecting some sensor data, we might like to send the data to the cloud, visualize it, or use some machine learning and prediction techniques on that data. Here are some ideas that might help you get started:

• Google Firebase: A simple solution for storage and publishing of the data.
• Mashape.com: A list of different APIs, most of which are available to free, that might give you some ideas on how you can use your data.
• Any simple JavaScript visualization library, like D3, for making your data available in an intuitive way.

• The Challenge
• The Building Blocks
• Extra

This post is the challenge for the Race Condition badge at Arctic IoT Challenge. The description for the badge is as follows:

At random, the jury will present a challenge. Timeboxed to one hour, the first team with a working solution gets all the points.

This time I am designing the challenge, so here it comes:

The Challenge

We want you to simulate a popular Scissor-Paper-Rock game using micro:bits. Each team gets two devices.

• On shake the micro:bit should pick a random shape (scissors, paper, or rock) and show it using LED array on the device
• The two devices should connect and send the data over to each other
• The opposite device compares then its shape to the one received and decides whether it won or lost
• Each of the two micro:bits shows then “Won” or “Lost” on the LED array.

The Building Blocks

Connecting to a micro:bit

Connect the micro:bit to your computer using a micro USB cable. It should appear as a USB disk on your machine.

Editor and Language

Yan can use Python and the editor here: http://python.microbit.org/editor.html.

After you are done writing the code, click Download, and you will get a binary file you will be using in the next step. Check out the help section for more information.

Python documentation for the API can be found here.

Deploying to a micro:bit

Copy the binary *.hex file from the previous step on to the micro:bit, just like you would copy to any USB drive.

Sending Data Between micro:bits

Now, over to connecting micro:bits and sending data over. Take a look at the example below. Make sure to change group=1 to a number between 0 and 255. Both chips should belong to the same group to connect with each other. You will be assigned a number.

from microbit import *
from radio import *
from random import*

#turn radio on and configure group
on()
config(group=1, length=251)

while True:
    if button_a.was_pressed():
        #send the images
        send("Hello, World!")
     
    #receive data    
    data = receive()
    
    #check that we have received any data
    if data != None:
        
        #split the data into different pictures by the marker
        display.scroll(data)

        #clear the screen
        display.clear()

The Scissor-Paper-Rock Game

Have a look at the simple implementation of the game. You will need to translate it to MicroPython and extend.

Extra

[Spoiler alert!] The winning team gets to submit their code to this GitHub repo.