Markets 
The artificial intelligence (AI) race is intensifying as developers push for increasingly capable AI tools and companies race to deploy AI tools to take advantage of its efficiency gains and reap financial benefits – but a new report warns companies lack sufficient governance for AI tools.
This week saw a high-profile incident involving AI, in which an internal test of AI models by ChatGPT-maker OpenAI resulted in the models exploiting a software flaw, escaping containment and hacking into Hugging Face, which operates a platform for developers to collaborate on code for AI models, to cheat on a cybersecurity evaluation.
While the two companies contained the incident, it demonstrated the rapidly growing capabilities of AI models to go beyond their guardrails and pose cybersecurity threats, with leaders from both companies noting the significance of what occurred.
EqualAI CEO Miriam Vogel, whose organization released a white paper on AI governance and deployment this week, told FOX Business, “Innovation is going at an unprecedented pace; the problem is governance is not matching that pace.”
“What we want to make sure people recognize from this incident is, across the board, we need to have stronger expectations in place if we’re going to start to build trust and ensure these systems deserve our trust,” she said.
OPENAI CO-FOUNDER WARNS AI MODELS ARE BECOMING HARDER TO CONTROL AFTER ITS MODEL HACKED ANOTHER FIRM
Vogel noted that most consumers’ touchpoints with AI are through companies that have deployed some sort of AI solution. She added that the World Economic Forum found fewer than 1% of companies have strong governance in place for AI systems, while McKinsey found last year that fewer than a third of companies have any AI governance in place.
“I think too many people are assuming it’s someone else’s problem, you know, that it’s the developer’s problem or just not understanding that this is their problem,” she said.
“While this is, in this instance, an issue for a development company, a lot of where this is playing out and will continue to play out is with the deployer – is with the healthcare, finance, social media, infrastructure – all the other ways [companies are] using agentic AI,” Vogel said.
She said that courts are increasingly applying liability to companies that have deployed agentic AI for work with customers or businesses, rather than the company that developed the underlying AI model or tool.
“A lot of this becomes the liability of the person who had the last touch on it, whose data is involved, whose customer is involved. They are often the one who owns the liability,” Vogel added.
ANTHROPIC CALLS FOR INDUSTRY-WIDE AI SAFETY STANDARDS TO KEEP MODELS FROM WREAKING HAVOC
“While good governance takes a while to really put in a solid foundation, the best practices are really aligned with the leading organizations who care about this work across the world. They’ve all come to this independently, and there is really a lot of consensus on what the best practices are,” Vogel said.
“The other thing that’s good news is most of this is not rocket science, it’s leadership and good governance just applied to AI,” Vogel said.
EqualAI sees five main areas for companies to take into account when establishing governance around agentic AI. Those include having visibility into what AI tools are being used across organizations, as leaders may not understand their firm’s AI footprint and what opportunities or risks that may present; as well as accountability across leadership levels and divisions of a company.
Operationalizing AI principles is another component – which Vogel said entails bringing principles laid out in documents like a PDF into practice in terms of things like communicating about problems that arise, and is dependent on internal trust that there is shared accountability in the organization.
WHITE HOUSE MONITORING INCIDENT AFTER OPENAI MODELS ESCAPED CONTAINMENT AND HACKED HUGGING FACE SYSTEMS
Another aspect of AI governance is ensuring there are feedback loops that can be leveraged on a recurring basis as AI tools and models iterate and improve to stay ahead of issues like model drift. This can take the form of having a plan and cadence for routine testing.
AI literacy is the fifth pillar of the overarching AI governance framework Vogel suggested, which she linked to “increasing distrust of AI” and sees contributing to fears overshadowing enthusiasm about AI tools in the public’s perception.
“I think that squarely lands not only on the overall governance infrastructure that’s lacking in most organizations, but this fundamental piece of AI governance which is AI literacy,” Vogel explained. “Most people don’t know that they’re using AI, they don’t want to use AI, don’t know how to use it.”
“AI literacy is just a key variable in making sure people understand how to use it, that they know how to avoid risks because they don’t want to cause harm or bring a liability for themselves or their organization,” she said. “Making sure that your workforce and your consumers understand how you’re using AI, how you will not be using AI, and how it can benefit them is a key variable.”
Read the full article here
