Securing Your Internal Knowledge Amidst Shadow AI

According to Yuval Fernbach, VP and CTO, MLOps at JFrog, “We shouldn’t ask employees to guess which bots are safe. We should provide them with environment-ready tools where the ‘no training’ guarantee is already baked in.” Consequently, organizations are tasked with validating the forms of AI they make available, so that their internal knowledge remains within the organization as opposed to being used to train or fine-tune models from a thirdparty vendor. Various ways of accomplishing this objective include these measures:

♦ Model Registries: These centralized constructs contain all pertinent information about the sanctioned AI assets, including models, bots, MCP servers, and certain APIs. These resources are only added once they’ve been validated against a host of concerns, including, for each asset, “its licensing, its training data provenance, and—most importantly—a legal and technical guarantee that our data will not be used for model training,” Fernbach specified.

♦ Departmental Catalogs: It’s also useful to stratify which AI resources are being deployed according to factors such as business units, roles, projects, individuals, and more. Fernbach likened this information to a bill of materials for software that provides “an inventory of which AI assets are used in which departments, ensuring we meet regional compliance laws like the EU AI Act.”

♦ User Behavior Monitoring: Security, governance, and IT teams can also implement solutions that facilitate user behavior monitoring to ascertain what AI resources are being employed at that particular moment. Some of these mechanisms involve user behavior analytics as well as centralized overviews of which IT resources are currently being employed. According to Fernbach, “The first step to governing unapproved AI is knowing it exists in the first place. In many cases, the same processes used to verify licensing and usage rights can also be used to surface and identify shadow AI.”

Tougher Than Ever

The pervasiveness of advanced ML models, as well as their effectiveness for increasing productivity, has multiplied the difficulty in securing internal knowledge. Organizations cannot afford to forsake the staples of data access governance, which include data discovery, data classification, access control policy authoring and implementation, monitoring and auditing for regulatory compliance, data privacy, and data security.

However, organizations now have more tools to stay vigilant of, and, depending on how progressive they are in implementing intelligent agents and bots, users to be wary of. Finally, there are those data-hungry statistical AI vendors looking to improve their models on what should be proprietary data. According to Fernbach, “Security works best when it’s an ‘enabler.’ If we give people a safe, approved way to do their jobs, the risk of accidental leaks drops significantly.”