How important is machine learning today?

02/09/2017

How important is machine learning today?

Providing relevant, useful results is one of the most important things to us here at Qmee. We’ve always made sure we only show our sidebar with results IF we have something relevant at the time. If we don’t, then we simply don’t show. This is because we understand how frustrating it is when results are shown that don’t have anything to do with what was searched for – just because someone wants to advertise to you.

We believe that there can be a happy medium  – show someone something that is useful and relevant to them and you have more chance of them taking an interest.

But how do you make sure your results are relevant when you need to be able to distinguish between someone looking for a ‘credit card holder’ and a ‘credit card transfer’? Or when someone is looking for a ‘Greek restaurant to dine at’, versus ‘Greek takeout food’?

To understand these subtleties, you need to understand what is being looked for on a deeper level and not just what category they are searching for – e.g. sports, food, entertainment. If you think of all the searches you do in a day and then times that by just the people in your work office, you can begin to imagine how much data there is to go through – it’s not possible for any human to do. This is where technology comes in.

Technology using Artificial Neural Networks (ANN or NNs) and Deep Neural Networks (DNN) is very common these days, and used in several industries. Google’s Alpha Go uses a series of convolutive Neural Networks and recurrent learning. They are even able to scan medical images to detect hidden features that might represent an early indicator of cancer.

Neural Network is essentially an old technology  – it just happens to have been re-activated by recent hardware advancements. Its’ simplicity and popularity has created many free-to-use repositories in Github and even Google’s Tensorflow is open sourced, making it more widely available for people to use.

It’s not just futuristic-type products like Siri and Amazon Echo that use machine learning, and it’s not just being done by companies that we think of as having huge budgets like Google and Microsoft. Many smaller technology companies are likely to already be running more efficiently, and making more money, because of machine learning. It’s not just Google that needs smart search results.

But NN is only a tool – to create something with it, one must have data and in particular, training data. However sophisticated the AI, it still needs something to learn from. Alpha Go only beat Lee Sedou because it had ample amounts of training from known games that it had learnt from.

The most difficult part is obtaining good training data – and of course there isn’t really a place with all the training data ready in the format you want it; you have to collect it, organize it, label it and then stream it through a time-consuming training process.

So in order to provide the most relevant results possible, you need a classifier that can intelligently categorize exactly what was asked. It involves collecting data from available resources, formatting the data and labelling with the correct category and keywords. And this is happening in an ever-changing environment so the classifier needs to incorporate the newest searches in to the training process and update the model used with new, up-to-date data – constantly.

A good training model means a classifier can tell the difference, with confidence, between a “credit card holder”, a “credit card transfer” and even a “share holder”. Having a classifier you can trust is a great approach to keyword classification problems; it can be designed to run automatically which means minimum management effort and minimum supervision.

Most of the conversation about machine learning today revolves around AI personal assistants and self-driving cars but nearly every website you interact with is using machine learning behind the scenes. Big companies are investing in machine learning – not because it makes them seem cutting edge, but because they’ve seen positive ROI. And that’s why machine learning, and innovation, will continue.