A System of Systems … With a Twist

Designing for Governance and Assurance

AI has been around, at least to a limited extent, for a long time. It just needed Moore’s Law to catch up. The same goes for the long list of cognitive ecosystem-enabling technologies that include intelligent agents, machine learning, IoT, edge and peer-to-peer distributed computing, and human–computer interfaces. Now they’re all converging, and we need to steer the ship carefully and purposefully.

Can these things possibly “go rogue”? Keep piling on enough autonomous machine learning and they could conceivably start forming “minds” of their own. We need to get out in front and begin formulating robust systems of governance and assurance. Doing so can be difficult. Cognitive ecosystems are likely to be made up of many diverse systems—some legacy, some new, and some emergent—all trying to work together. As the ecosystem scales in size and complexity, asymmetries can easily creep in. Adding to different governance frameworks, there will likely be wide variances in cognitive styles, cultural perspectives and expertise on the human side, and conflicting architectures and protocols on the machine side. Advocating for standards that balance openness and accountability will help make the myriad interconnections more seamless and transparent.

Ethics in both design and operation demand incorporating responsible AI combined with human oversight. This includes paying close attention to identifying and mitigating possible unintended consequences. These can arise from decisions made in the presence of bias, forced dependencies, missing or corrupted critical knowledge, and cybersecurity vulnerabilities. This requires sound governance frameworks that include clearly defined ownership, accountability, transparency, and inclusiveness.

Opportunities for KM

You’ve probably experienced something similar to the following. You’re late for an appointment in a busy city. The so-called “synchronized” traffic lights are anything but. The light far ahead of you changes quickly, allowing only one or two cars to proceed at a time. The intersecting street, totally empty, has an almost continuous green light. This illustrates how quickly even the simplest “smart systems” can break down. Large cognitive ecosystems, with all their complexity, interdependencies, and risks, are screaming for KM.

KM can help address issues such as these:

♦ Guarding against the adverse effects of over-automation, ensuring that the humans in the system remain fully empowered by promoting tacit knowledge transfer, critical thinking, and upskilling in areas such as human–AI collaboration, systems thinking, and ethical literacy

♦ Maintaining cultural integrity by promoting and facilitating cross-cultural co-creation and collaboration

♦ Promoting social trust by incorporating transparent, ethical, and explainable AI

♦ Fostering strong public relations by minimizing distrust that can arise from excessively automated or hybrid decision making, especially where the stakes are high (e.g., health, justice) and when precipitated by a prevalence of opacity both on the part of overly guarded and evasive humans and “black box” AI systems

♦ Preventing the emergence of “power asymmetries” by democratizing knowledge and decision making in ways that inhibit domination by only a few platform providers or proprietary AI models

Finally, put on your facilitator’s hat and help foster frank and open dialogue across the stakeholder community while assuring equal access. Are there any knowledge graph engineers in the house to help keep track of all of this?

Taking the Plunge

If you’re still worried, perhaps even terrified, about ASI unleashing a “What hath God wrought?” event upon the world, then consider treating this along the lines of similar transformations such as splitting the atom, gene editing, or quantum computing. The genie is out of the bottle. How we employ cognitive ecosystems is up to us.

Yes, there are risks. But the enormous potential benefits cannot be ignored, especially in sectors where complexity, scale, and speed matter, such as energy, health, mobility, education, and disaster management. Once again, KM has an opportunity to play a major role. Are you up to the challenge?

For a more fun and in-depth peek into this brave new world, check out Braden Allenby’s “World Wide Weird: Rise of the Cognitive Ecosystem.”