Data campaign

17 Spotting patterns Another area where data can promote agility is through sensing changes or recognizing patterns in the environment. For example, machine learning can be used to detect and respond to anomalies. We can train a machine learning model with historical or routine data so it becomes used to what is "normal" and then apply it to find activity that is not normal. This technique can be used, for example, to spot fraudulent transactions or network intrusions by hackers. Or to spot equipment on a factory production line that is diverging from its normal behavior and might have to be repaired or replaced—and to do so before it actually fails. When we collect large amounts of data, we may find that we can identify relationships that we didn't know were there. Social media companies build large databases of relationships between people. Homeland Security might find that a potential terrorist they are investigating once lived at the same address as someone who is already known to be a terrorist—which might lead them to ask questions when they next encounter the person. A number of fraudulent immigration applications might turn out to have all been prepared by the same immigration lawyer. Here, we have moved well beyond simply using data to process transactions: We can now find important and interesting relationships between those transactions. But once again, we don't know exactly what relationships we might find; agility, flexibility, and curiosity are the keys to deriving value from data. To cite one more example of using data to "keep an eye on events," the existence of a data point can serve as confirmation that an activity took place—for example, when audit trail logs are created automatically. By following the trail of activities, auditors may be able to validate compliance or investigate improper activity. Blockchain is often used to store data that confirms that activities took place—for example, a transfer of money between two parties or an approval of a contract by the parties involved. By using automated guardrails and audit data to establish compliance, enterprises can often avoid heavyweight compliance processes that reduce agility. There are, of course, challenges in using data to support business agility. As we noted above, it requires skill to draw the appropriate inferences from data. The data does not always tell us what action to take: We have to interpret it and make good decisions. Often, we face a trade-off between false positives and false negatives—for instance, if we use the data to spot anomalous transactions to identify potential fraud, we run the risk of flagging too many transactions as anomalous and annoying our customers or flagging too few and allowing fraud to sneak through. The larger the data set becomes, the more likely that meaningless patterns will emerge or that important patterns will become buried in the sheer number of potential connections. Noise accumulates along with signal.

• Cover