The Small Data Revolution: AI Isn’t Just For The Big Guys Anymore
With artificial intelligence now all around us, all the time, we must confront a concerning truth: Never before has a technology so ubiquitous been controlled by such a select few.
Of course, it’s a truth that’s easy to overlook because we interact with AI every day. Between the route optimization algorithms that Google Maps uses to speed up your drive to the movies, the personalized shopping suggestions that allow Amazon to find that perfect pair of sneakers and the natural language understanding (NLU) that Apple’s Siri employs to tell you that yes, it will rain tomorrow, a handful tech giants are eliminating a countless number of friction points in our lives as consumers.
Yet in the enterprise, fewer than 15% of companies deploy AI in production, and about the same percentage, according to McKinsey, believe they possess the technological infrastructure needed to support AI initiatives. When narrowing the lens to machine learning (ML), the mathematical techniques that enable AI systems to improve with experience, the gulf between Big Tech and all the rest is even bigger. Of the world’s most digitized companies only 18% utilize ML across multiple workstreams, per McKinsey, and critically, only 9% of these enterprises benefit from NLU in multiple business processes. Unable to communicate with employees on their terms, AI tools designed to reduce friction at work often do just the opposite.
A Difference Of Data
The need to “democratize AI” has become a hot topic in recent years, with experts raising the alarm about Big Tech’s monopoly and calling for greater oversight. Needless to say, most firms are prohibitively short on in-house AI experts: Fewer than 10,000 people on earth have the skills necessary to build advanced AI — a relatively tiny talent pool that usually trickles straight into Silicon Valley. Hiring a team of machine learning specialists to automate internal processes is simply unrealistic for your average organization, since these specialists have the leverage to work almost anywhere and at almost any price.
As a result, many companies are instead implementing off-the-shelf AI tools from third-party vendors, rather than building their own. For such tools to be useful, though, they must still complement the unique systems, terminology and employees of each enterprise that deploys them. But there’s a problem.
Training an ML model requires big data — millions upon millions of data points that ultimately enable the model to make confident predictions. And unlike, say, Google, which has the advantage of 130 trillion webpages to refine its algorithms, a company attempting to automate an internal process might only have 30 relevant examples. AI inequality — at first blush the product of a disparity in resources and expertise — largely comes down to this difference of data.
Similarity Under The Surface
How can we transform the limited data found at most companies into fuel for machine learning? For context, the utility of such “small data” isn’t a new idea — in spite of its insufficiency for ML. What’s not only new but fundamentally game-changing is the power to turn small data into big data using three machine learning techniques: collective learning, transfer learning and meta learning.
To illustrate, take the example of a firm seeking to automate its internal tech support with ML. The firm’s IT team must field thousands of emails that employees write in unstructured, natural language. These emails contain requests for software licenses, for guidance on resetting a password and for such a broad variety of other requests that each one is usually unprecedented in its exact form. This is a perfect problem for ML, but it’s also where most companies get stuck: They don’t have the experts or the resources to build their own ML models, and even if they did, they don’t have nearly enough IT support requests to train those models.
Crucially, though, the syntactic structure of these requests is consistent with what IT teams are seeing across companies in many industries, despite superficial differences in the names of people and software. That’s where collective learning comes in: Today, we can train ML models on millions of employee support requests from many companies and find the commonalities between them. Depending on your domain, this still might not provide the level of performance you need.
Enter transfer learning, which lets companies apply open-source ML models trained in tangential domains — such as models trained using Google’s BERT —to its own datasets. The result? What once appeared to be a one-of-a-kind IT request can suddenly be understood in relation to countless similar sentences.
In the process of transforming small data into big data, we must not lose the specificity that made the small data valuable. Meta learning is an essential technique here. With meta learning, we augment our understanding of a given phrase by evaluating the information surrounding that phrase, such as what time it was written. For instance, the firm above might have received a handful of IT tickets from employees requesting “access to the report,” without naming the exact report needed. But by considering metadata — an employee’s location, department, role at the firm and more — we suddenly have sufficient inputs to train highly predictive models.
Thanks to bigger datasets, bigger budgets, bigger infrastructure and bigger ML teams, Big Tech still has an advantage in the AI arena. The difference now is that nearly any organization can put AI to work with minimal effort — and without hiring a dedicated team of experts to maintain it.
Nastel has created a feed with some new information on COVID-19! Click here.
This article originally appeared on forbes.com To read the full article and see the images, click here.
Nastel Technologies uses machine learning to detect anomalies, behavior and sentiment, accelerate decisions, satisfy customers, innovate continuously. To answer business-centric questions and provide actionable guidance for decision-makers, Nastel’s AutoPilot® for Analytics fuses:
- Advanced predictive anomaly detection, Bayesian Classification and other machine learning algorithms
- Raw information handling and analytics speed
- End-to-end business transaction tracking that spans technologies, tiers, and organizations
- Intuitive, easy-to-use data visualizations and dashboards
Nastel Technologies is the global leader in Integration Infrastructure Management (i2M). It helps companies achieve flawless delivery of digital services powered by integration infrastructure by delivering Middleware Management, Monitoring, Tracking, and Analytics to detect anomalies, accelerate decisions, and enable customers to constantly innovate, to answer business-centric questions, and provide actionable guidance for decision-makers. It is particularly focused on IBM MQ, Apache Kafka, Solace, TIBCO EMS, ACE/IIB and also supports RabbitMQ, ActiveMQ, Blockchain, IOT, DataPower, MFT and many more.
The Nastel i2M Platform provides:
- Secure self-service configuration management with auditing for governance & compliance
- Message management for Application Development, Test, & Support
- Real-time performance monitoring, alerting, and remediation
- Business transaction tracking and IT message tracing
- AIOps and APM
- Automation for CI/CD DevOps
- Analytics for root cause analysis & Management Information (MI)
- Integration with ITSM/SIEM solutions including ServiceNow, Splunk, & AppDynamics