In today’s column, I examine a new form of tuning for generative AI and large language models (LLMs) that is referred to as reinforcement learning with metacognitive feedback (RLMF). Those of you who are familiar with the overall construction of AI are probably aware that a longstanding tuning technique is known as RLHF (reinforcement learning from human feedback). This new RLMF is considered a distinctive variation of the customary RLHF and can be used in conjunction with RLHF, or possibly in place of RLHF, though that is not especially advantageous.
I will walk you through the key elements of RLMF. There are upsides and downsides (as is the case with any AI tuning method). For those who are well-versed in the building of AI, you would likely be familiar with a somewhat allied approach that is RAIF (reinforcement learning from AI feedback). I realize all of this might seem like quite an alphabet soup of esoteric names. In any case, RLMF is certainly worth exploring, particularly for those developing AI foundational models and for those who want to know what makes contemporary LLMs do what they do.
Let’s talk about it. This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
Background About LLMs
I’d like to lay some groundwork before we get into the mainstay topic of RLMF.
In brief, generative AI and LLMs are computer-based, pattern-founded models of human language that have a large-scale data structure and do massive-scale pattern-matching via a large volume of data used for initial data training. The data is typically found by extensively scanning the Internet for zillions of stories, news items, essays, blogs, poems, narratives, and the like. The mathematical and computational pattern-matching homes in on how humans write and henceforth generate responses to posed questions by leveraging those identified patterns. It is said to be mimicking the writing of humans.
I think that is sufficient for the moment as a quickie backgrounder. Look at my extensive coverage of the technical underpinnings of generative AI and LLMs at the link here and the link here, just to name a few.
Tuning An LLM Via RLHF
AI makers fine-tune how their AI will respond to users. They typically use the RLHF (reinforcement learning with human feedback) method to shape their AI. The approach is straightforward. Humans are hired to work with an LLM before it is made available to the public. Partially testers and partially a form of human guidance, they rate the answers that the budding LLM provides to them. An upvote means the person favored the response, and a downvote means that the answer was disfavored. They are providing feedback to the AI and giving guidance on how the AI ought to respond to users.
The aim is to allow the AI to mathematically calibrate what is construed as preferred answers versus less preferred answers, often impacting the tone and style of responses more than the content. For example, to get the AI to be polite, the hired humans upvote answers when the AI is well-mannered and downvote answers when the AI is impolite. This becomes a discernible pattern that the AI then rolls into the rest of the pattern-matching apparatus and abides by overall.
For details about the ins and outs of RLHF, see my coverage at the link here.
Advent Of RLAIF
The RLHF approach can be costly and consumes a lot of time and effort to perform. Why so? Because it takes gobs of human labor to do the task. You typically must employ a veritable army of people. In addition, it is necessary to properly train them on how to undertake the job. If you don’t adequately train them, the guidance they give the AI could be disastrous. Imagine if the new hires opted to upvote the AI when it is insulting to users and downvote when the AI is pleasant. The resulting tone of the AI would be abrasive and possibly intolerable.
Is there anything that we could do to reduce the cost and speed up the feedback process?
Sure, we could employ AI. Here’s the deal. We might get an LLM that is already in the marketplace and well accustomed to interacting with users. We will hook up this AI to the budding AI that we are trying to tune. The more seasoned AI does the same thing that the human feedback testers and guides were doing. Voilà, this can be conducted quickly and usually at a much lower cost.
Not everyone is convinced that this method of RLAIF is as even-handed and capable as when using human labor. Some won’t use RLAIF and only use RLHF. Others attempt to blend both. It is a difficult decision and requires careful judgement on how to best proceed.
Metacognition Enters The Picture
You are now aware of the basic RL aspects of tuning AI. Congrats. The next step is to learn about the newest suggested method, known as reinforcement learning with metacognitive feedback or RLMF.
First, let’s get on the table the overarching meaning of metacognition. You might have heard about metacognition as a topic that comes up in psychology and studies of human behavior. Metacognition entails the act of thinking about thinking.
You do this all the time. Suppose you are planning an upcoming vacation. You put a lot of thought into this. At some point, you might begin to realize that you are possibly putting too much thought into it. You are becoming mentally obsessed with the planning task. As such, perhaps you opt to reduce the intensity and take things a little easier with the planning process.
In this instance, you thought about your thinking processes. It was an act of metacognition. You can get carried away with this by trying to apply metacognition to metacognition. That involves thinking about thinking about cognition. Akin to the sci-fi movie Inception, I suppose you can keep going further down in a multitude of recursive layers.
Applying Metacognition To AI
It is possible to get AI to pay attention to how it is doing its mathematical and computational processing. Is that the same as metacognition? Well, kind of, but kind of not. We usually reserve the word “metacognition” for sentient beings and especially for humans. Trying to bandy that same word around when it comes to AI is a bit of a stretch; some would say it is egregious and undue anthropomorphizing of AI.
Sadly, the AI field has embraced many such pre-loaded words. Some claim that AI “thinks” and “reasons” – but those are words we usually reserve for describing human cognition. This is a misappropriation of those valued words. They carry a false aura about what AI consists of. They undermine the significance of those words when it comes to understanding the human brain and mind.
I used to be adamant about not using those words in an AI context. The rest of society decided that those words are fully acceptable for the AI field. I now hold my nose and reluctantly go along with it. Please know that they aren’t the same meaningful words as when describing human cognition, though they certainly have that appearance.
Alright, now that I’ve gotten that off my chest, we are ready to dive into RLMF.
Introducing RLMF
A recent research paper that originated the RLMF moniker is entitled “Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs” by Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan, arXiv, June 30, 2026, and made these salient points (excerpts):
- “Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one’s own cognitive processes.”
- “Yet LLMs exhibit systemic deficiencies in key metacognitive faculties: they hallucinate with high confidence, fail to recognize knowledge boundaries, and misrepresent their internal uncertainty — undermining trustworthiness and reliability.”
- “We introduce reinforcement learning with metacognitive feedback (RLMF), a training paradigm in which the model is rewarded not only for producing strong outputs, but also for accurately judging how well it performed.”
- “RLMF builds upon prior work showing that intrinsic confidence signals can serve as effective RL rewards, but operates at a higher level of abstraction by leveraging the quality of the model’s assessment of its own performance rather than simply outputting confidence.”
- “Concretely, we introduce a novel metacognitive advantage scaling mechanism: during RL training, we use the accuracy of the model’s self-judgments to scale each completion’s advantage, i.e., the relative learning signal that determines how strongly that completion is reinforced compared to alternative sampled completions.”
Let’s step through the essence of RLMF.
Calibrating AI Confidence
One of the loudest complaints about contemporary AI is that it is shaped to act like its answers are impeccable and not to be doubted. There is an air of perfection. When you read a response, the tone and style cause you to believe that the AI must be correct. Any doubts in your mind are to be placed aside.
Perhaps this is partially why AI hallucinations are as common as they are. An AI hallucination is when an LLM makes up an answer out of thin air. There is no factual grounding for the response. Yet the AI pitches the answer to you as though there is zero hesitation that the response is fully factual. For more about AI hallucinations, see my in-depth explanation at the link here.
If we were to tune AI to be more aware of its own “cognition” and be more suspicious of its own answers, the hope is that the AI would be less likely to give answers that look as though they are unimpeachable. The responses would be worded with a softer tone.
For example, instead of the AI saying it is absolutely sure of something, when it really ought not to say that, we might have the AI say this:
- “I’m confident about this answer.”
- “I’m only moderately certain.”
- “I don’t know.”
- “There are several plausible interpretations.”
AI That Self-Monitors
RLMF adds a dimension that entails rewarding the model for accurately evaluating its own performance. In effect, the model is encouraged not only to generate a good answer, but also to know when it has produced a good answer, but also to know when it is uncertain, and when it has likely made a mistake.
An LLM that uses the RLMF would hopefully recognize its own weak reasoning and be better positioned to:
- Reconsider its reasoning.
- Perform additional internal verification.
- Retrieve more information.
- Possibly decline to proffer a wrong or unsure answer.
Rather than blindly proceeding, the AI develops something resembling an internal quality-control mechanism. An added benefit is that the AI might become more computationally efficient when processing prompts and arriving at answers. No more going down dead ends and spinning the wheel on impossible or fictitious responses.
Using Prompts To Do Likewise
Devoted readers might remember that I am a big proponent of using prompts that tell the AI to consider factors involving certainty and uncertainty; see the link here. You tell the AI via your prompt that it is to attach certainty levels to its responses. Furthermore, you instruct the AI to pay attention to those uncertainty levels.
This provides a clearer sense of whether the AI is confident about its answers. Each response that you get from the AI will tell you the AI-determined certainty. If you see that the certainty is rated as low, it is on your shoulders to decide whether the response is worthwhile.
The nice aspect of RLMF is that this same kind of activity is taken care of for you, having occurred when the AI was initially tuned and shaped. The AI maker already put in that elbow grease. Whether that obviates the need to still do the same when entering prompts is not quite settled, but at least the odds are that the AI will be doing a similar job internally anyway.
AI Games Itself
Like any reinforcement learning objective, RLMF introduces opportunities for reward hacking. This consists of the AI finding shortcuts that you don’t want the AI to undertake.
For example:
- LLM always expresses only moderate confidence (trying to avoid getting pinned down).
- LLM might avoid difficult questions — because it will look bad that the AI cannot provide a high-certainty answer.
- LLM produces self-critiques that sound convincing but are uninformative (abiding by the principle that it isn’t supposed to be overly sure).
- LLM learns statistical cues that correlate with correctness rather than evaluating its reasoning.
A famous economist established a belief that when a measure becomes a target, it ceases to be a good measure. That was Charles Goodhart, and the rule is known as Goodhart’s law. When AI self-evaluates and tries to optimize, the optimization process may exploit imperfections in the metric. Not good.
Proof In The Pudding
You are urged to closely read the RLMF paper and decide whether you believe that RLMF is going to catch hold. Will AI makers opt to use RLMF? Time will tell.
RLMF has the same advantage as RLAIF when it comes to avoiding the cost and time associated with seeking human feedback via the RLHF approach. RLAIF tends to seek determination of which response is best, while RLMF goes toward having the AI recognize the certainty or confidence in whatever its response is. As mentioned, one caveat would be to try to ensure that the reward function for RLMF doesn’t overplay, potentially leading to biases and excessive hedging.
Frederick Lenz, a famous spiritual leader and software designer, made this remark: “You are happiest when you are most humble. You are most miserable when you are egotistical.” We don’t need to make sure that AI is happy, but ensuring that its answers are humble seems like a solid practice. Then again, perhaps we do need to ensure that AI is happy, else once AI takes over, we might have a mean and angry AI on our hands.
