Distinguished engineer. AI/ML at Cisco. Former Chief Data Scientist at Kenna Security. Co-Founder of Dharma Platform, a Forbes 30 Under 30.
The recent AI/ML inflection point in the C-suites of businesses world wide has a specific cause. The buzz around ChatGPT’s rise is actually about the potential that the architecture of Open AI promises. This is the combination of two tried and true machine learning approaches: large language models (LLM) and reinforcement learning from human feedback (RLHF).
LLMs are advanced prediction tools that generate coherent text sequences by analyzing vast amounts of text data. Their ability to produce relevant text varies based on the application and context, highlighting the challenges of unsupervised learning across diverse use cases. In RLHF, a machine learning agent learns optimal actions through environmental interaction and feedback, incorporating both automated and human evaluations to refine its decisions.
A key innovation in GPT models involves pairing LLMs with a secondary model that assesses text quality, using human-ranked outputs to train this model. This combination enhances the LLM’s ability to align with human preferences, showcasing a synergistic approach to improving AI-generated text.
The future of AI/ML is the ability for a new way to interact with knowledge—one that does not require the skills that are already causing a workforce shortage in security. Vendors have access to troves of security data that the average customer or enterprise using their technology does not.
We need to encode that data into LLMs that represent the language of security. Fundamentally, that means training new neural networks that can perform specific security analyses or tasks. We need to use user interaction data to infer human feedback for reinforcement learning. The holy grail of automation in security has escaped vendors because, for the past two decades, security vendors have been busy building tooling to make action possible. Today, we are able to build models of not just what is possible but rather what is optimal for every situation our customers can find themselves in.
Large Language Models
Security is a language. Few practitioners speak it. In fact, in 2013, Dan Geer and Richard Thieme foretold of the dying breed of security generalist—there are too many subfields, too many specializations and domains that need to be taken care of. As a result, a network analyst may not be speaking the language of the vulnerability manager. The analyst needs models capable of representing and translating security events and findings.
The security workforce shortage is real and increasing. Training new analysts is becoming increasingly more difficult. The current XDR zeitgeist is all about upleveling SOC analysts with tooling. The biggest opportunity in security is to train by translation. This means new and novel ways of interacting with all the security data in an organization, by training first a general model (or many models!) aware of all the types of data a security analyst may see and then building a second, more contextual model aware of that individual enterprise’s environment.
Asking that model a question during an investigation and getting back an answer that may take two SQL joins and a pivot table between two vendors is the holy grail. This is only possible with a breadth of training data—assets across config, network, EDR, NDR, threat intelligence and application security. You need these inputs to train a set of models that will be useful.
Reinforcement Learning Through Human Feedback
UX is the core of machine learning. The big opportunity in security is to systematically gather data about end users’ behavior and use that data as the human in the loop. We have years of data about follow-up actions, clicks, investigations and searches across our assets. A clever ETL pipeline would use that data to train a second model that models the preferences of analysts in similar situations: “Should we autopatch this vulnerability, or is there a possibility of downtime?” “Should we quarantine this Windows machine?” These are questions answered dozens of times, just perhaps not at this client’s site.
What’s Next?
Data comes first. There are three approaches that will accelerate the industry’s time to market with these new technologies.
1. Schema Consistency Across Products: Entropy of datasets is the proven weakness for all versions of LLMs. The more inconsistency between structured and unstructured datasets, the harder it is for a model to infer what two entities are the same—and transformer representations of similar entities is the core of LLMs and the reason they are so good at language representation. My recommendation is to formalize data schemas where applicable. This can be done by mandate, top-down or it can be done with machine learning. Having large amounts of entropy is itself a signal, and there is a suite of algorithms that can infer what fields mean in datasets by measuring entropy. Is this the device name or a website? It largely depends on the entropy in the dataset we’re looking at. Doing this data cleanup early means 1000x returns on the accuracy and efficiency of the models we build on top of our data.
2. Risk As An Outcome Measure: Reward functions are the hardest part of reinforcement learning. While we build that internal human feedback capability the industry already has a great proxy for reward functions in security that ChatGPT didn’t have—risk measurement. If we give the RLHF models a reward function of minimize risk models that already exist in production, we can start building useful models ahead of having end users guide the reward functions. Risk becomes the baseline in a very real sense.
3. UX As The Key To Unlocking User Interaction Data: We need to build the muscle of not just allowing for user interactions that meet stories but also having the UX teams across the security industry build and capture user activity in the way that e-commerce sites do. Each click is an advantage in the AI modeling world.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
