AI: A reflection of us? Or them?

I recently finished a course about how AI works, how it can be applied at work, the human support it requires, and typical risks and misunderstandings. About as much as your C-level leader needs before they say something dumb like, “Let’s replace 30% of our team with this AI thing.”

As I worked my way through the course materials, these thoughts flew through my mind:

  • Huh, selecting and scoping an AI project is a lot like selecting and scoping a Six Sigma project. I wonder what impact AI is having on Six Sigma – is it enhancing it or replacing it? (Fellow Black Belts, feel free to weigh in in the comments.)
  • Gee, we’ve all been providing data to support the self-driving auto industry by clicking on cars and stoplights and bicycles during Captcha. If only we’d known…
  • Wow, no wonder the tech giants are all building nuclear power plants: I thought Terabytes were big but these guys are using Petabytes of storage every day.

But mostly, I thought about one of my favorite books.

AI plays a huge role in Hellspark by Janet Kagan, published in 1988. But unlike HAL, or The Terminator, or William Gibson, this book presents AI in an unusually positive light.

The book is science fiction – the main character (Tocohl) dashes from planet to planet in her personal spaceship – combined with a murder mystery (someone has murdered a member of an interplanetary team surveying a new planet with the potential to provide unlimited free power).

One of the main themes is how to define a sapient species, a species that is not only sentient (self-aware) but sapient (deserving to be called “people”, something that apparently humans have trouble doing with people who look, love, or speak differently than they do). One of the native species on the new planet may be sapient, requiring those who want that unlimited power source to negotiate for it instead of just seizing it.

Here’s the information we learn about Maggy, the AI in Hellspark.

Technical Spec’s:

  • A “top-class extrapolative computer with a larger memory bank than most.”
  • Tocohl says that Maggy “learns like a kid does, except that she only needs to be told once. Anything you tell her, she stores for later use; her entire program undergoes constant revision.” But “she has no practical experience in conversation” and seeks correction on her grammar and vocabulary.
  • Not able to access all memory actively at once.
  • Easily multi-tasks, at one point, conversing with Tocohl through her sub-vocalizer and smart glasses, holding another conversation with a teammate through a “remote” (networked robot), and still managing the ship.

Location:

  • Main processing hardware is located within Tocohl’s ship.
  • Also connected to Tocohl’s smart glasses, displaying imagery there.
  • Monitors Tocohl’s vital signs and physical health through Tocohl’s “second skin” suit.
  • Adjusts the weave of Tocohl’s skin suit to shift blows during a fight or support broken bones. As well as changing the suit’s color, pattern, and texture, similar to a cephalopod’s skin.
  • Directs a remote robot and “handhelds” (walkie-talkies), all at the same time.
  • When several of these are disabled by the murderer, others continue to operate independently, taking action without Tocohl’s direction or presence.
  • Records real-time audio/video from Tocohl’s glasses, subvocalizer, the robot, the hand-helds and the ship’s sensors. Retains this data and shares it upon request.

Communication:

  • Has access to a ton of facts and information and can collate and share that data with Tocohl almost instantly.
  • Lacks data on is human emotions and motivations, often initiating questions like, “Is that funny?” and “Why would they be angry about that?”
  • Understands when Tocohl uses verbal shorthand such as “Maggy?” to prompt the sharing of information with others in the room, or “Later.” to hold contributions until others are not around.
  • Uses slang from the multi-lingual dictionary, but is still confused by figures of speech. Tocohl muttering that she’ll eat someone alive for something they did, prompts a reminder of the illegalities of cannibalism.
  • Misapplies examples from Fictional books to real-life situations. Tocohl has to explain that “fiction [and especially the low romances that Tocohl reads for recreation] can’t be counted on to give you a pattern for reality.”

Self-Direction:

  • Able to identify that her priorities had shifted without her being aware of the shift, and therefore initiate self-diagnostics.
  • “Can’t argue” but often presents data that conflicts with Tocohl’s requests.
  • Lies when directed to by Tocohl.
  • States that humans are “confusing” (think Spock’s “illogical”).
  • When someone says they will share data later, states that “I would like that.”
  • Although Tocohl is clearly the primary authority, determines when to take direction from other humans when unable to communicate with Tocohl.
  • Conducts real-time risk analysis without direction, labeling a fight in an alley ahead “trouble” and directing Tochol to “Roll!” during a knife attack.

Initiative:

  • Initiates a call for outside assistance, without being directed.
  • Takes initiative to recite ritual words with Tocohl during a ceremony.
  • Learning that the planet contains natural super-conductors, without direction, immediately starts researching potential trade items that might appeal to the potentially-sapient species because that additional power is expensive, of value to Maggy, and not something that Tocohl may want to invest in.
  • Takes the initiative to create a ritual that allows an action that would otherwise violate a cultural norm, in order to save Tocohl’s life.
  • Takes the initiative to tell Tocohl a visual “story” about where the ship is located, when it would not be visible from Tocohl’s on-planet location.
  • Initiates the expression of affectionate feelings through touch.

The main thing I think about this in terms of AI, is how different this is from the “AI” that we have now. Our AI is not true AI. It does not seek correction, does not seek to understand human emotions and motivations (resting instead on a poor imitation based on LLMs – the majority of my data says this, therefore that must be the norm and anything you do outside of that is abnormal or wrong – and does not self-evaluate, recognizing when it has shifted priorities. In fact, it feeds you bad information and then gives a lip-service apology, and then goes ahead and feeds you bad information again.

I wonder how much of this is because our AI is being developed by men, by tech bro’s with a specific set of beliefs about people, norms, and command and control. As opposed to being raised by a woman who is part of a society that adapts to change, brings people together, and displays empathy.

I read a great post by Emily M. Bender and Nanna Inie that pointed out that we should not be calling what we have now Artificial Intelligence, because that causes us to anthropomorphize AI and assume that it reasons like a reasonable human being would. And it doesn’t. It’s a machine, following code written by a (usually male) engineer based on direction from an (almost always) male leader who just wants to make money for himself and his company.

I prefer the AI from Hellspark.

Leave a comment