Data mining is the process of analysing data sets with a database management system in order to discover patterns and meaning. What begins with large collections of linked or disparate information becomes valuable knowledge through the workings of highly specialised computer software. Statistics, artificial intelligence and databases come together to translate all sorts of hard facts into something understandable. From there, a business can see what trends and predictions should be informing their next move as well as identify what transactions may be fraudulent, which clients are likely to leave for a competitor and what measures to take to avoid these scenarios.

In short, a computer’s data management software turns a mountain of straw into a sheet of pure business intelligence gold. Sounds complicated, doesn’t it? As it turns out, that’s not exactly true, and retail, finance, healthcare, transportation and manufacturing organisations are all keen to make use of the vast amount of data waiting for them.

How does it work?

Through the use of database software, data mining is able to summarise, or make patterns from, a body of information. The patterns can take several shapes:

  • Unusual records, or anomaly detection, where certain observation points called outliers do not match the typical pattern. These may take the form of product contaminants, newly emerged trends or fraud – basically they show where something unusual has happened that does not match the rest of the observations. They provide clear red flags once the data is analysed, a key point for problem-solving, security breaches and quality control for businesses.
  • Dependencies or association rule mining put together often unexpected things and highlight the relationship between the two. An apocryphal anecdote regarding association rule mining alleged that young men buying diapers also usually also bought beer.
  • Groupings of data records, or cluster analysis, to show similarities and differences between data sets with things in common. If Product A is bought by middle-aged men as well as teenage girls every March or April (a strange similarity perhaps), a business may discover the item is the perfect Mother’s Day present.

From there further analysis may take place, such as predictive analysis or machine learning (both of which go beyond data mining itself but are the next logical step). This can help a business increase productivity through a better understanding of its raw data.

As a side note, databases are measured in gigabytes and terabytes, with 1 terabyte being equivalent to 2 million books. A large corporation may experience tens of millions of point-of-sales transactions in a single day. Think of how much customer information could be gleaned in evaluating an hour’s worth of credit card or telephone use – it’s staggering.

Why might businesses need it?

By not truly examining the data gathered in every aspect of its workings, a business loses a huge opportunity. With data mining, surveys, demographics and sales figures and so much more can flesh out a fuller picture of where a company stands, as well as help create precise risk models and spot-on marketing campaigns. Below are a few examples of what a business can do with data mining:

  • Improve marketing and strengthen branding. Customer surveys and client feedback can be used by a marketing department to focus on new areas for growth and ways to fix recurrent problems. What products would sell like hotcakes and where frequent gripes accumulate are identified here.
  • Increase revenue. Data mining will uncover your best sellers and sort through the statistics to paint an accurate picture of what customers actually want (versus what the Head of Sales thinks they want).
  • Communicate more effectively. Here you’ll be able to see what contact strategies are proving most popular. Can you really know whether it’s printed ads, emails or social media presence that is reaching the most people without data mining? Sure, if you want to add it up by hand. Finding out how to target a ready audience effectively will mean you won’t need to waste time, money and postage reaching out in the wrong way.
  • Don’t repeat past mistakes. Because data mining turns facts and figures into a complete representation of a business’s position, it can also show progress – or lack of it. Whether it’s a graph showing a sales slump or emerging trends a company has kept in line with, data mining patterns can help with predicting and preparing for the next opportunities.
  • Enter new markets. Some databases offer information gathered by other companies that can be used to investigate potential customer areas, improve the sales tactics currently out there and provide better services. It’s worth noting, though, that sharing information is mostly illegal so check your sources carefully to make sure consumers’ privacy isn’t being breached. Data sharing is usually done between partner organisations, so if you have just such a valuable link don’t let this gold mine go to waste.

One interesting example of effective data mining was the launch of the United Nations Federal Credit Union (UNFCU) global credit card in 2011. Aimed at frequent overseas travellers, the Visa card needed to be marketed as effectively as possible in order for maximum take-up. They contained the now ubiquitous embedded computer chips that made customer signatures redundant (a feature less popular in the US but very common everywhere else). Through the use of its marketing database UNFCU advertised to 30,000 individuals in high-income households who travelled frequently. The result was an astounding 3 per cent response rate, when a large financial institution could only garner 0.5 per cent on average. After a 10-week campaign, applications for the card rose more than 100 percent – a clear case of data mining driving success.

Data mining now versus 10 years ago

Almost 50 years ago, data was mined through ledgers, tapes and floppy disks. By the 1980s computers picked up pace and their increased storage capacity allowed for relational databases to be kept. From there we went to online analytical processing and data warehouses, and now the storage capability has further increased, with advanced computer algorithms doing the heavy lifting.

At the moment, data mining is something that most businesses are able to incorporate – and really should. It’s not just another buzzword. Getting some help from a bespoke software company in building a database and harnessing the power of data mining can be the most effective and least painful way of doing so. 

What’s in store for the future?

As consumer groups multiply and diversify we will likely see smaller, niche marketing campaigns designed to catch their attention. Information will be much more widely available to everyone (especially with the advent of Big Data collecting and linking everything), and savvy companies will use it to get the edge on their particular offerings. The sooner this can be done, of course, the better!

See our other posts in the Tech 101 series

Tech 101: What is Software?

Tech 101: What is Open Source Software?

Tech 101: What is Programming?

Tech 101: What is .NET?

Tech 101: What is SQL?

Tech 101: What is Data Mining?

Tech 101: What is SaaS?

Tech 101: What is Hosted Software?

Tech 101: What is Bespoke Software?