As if walking a robot dog wasn’t strange enough, Boston Dynamics’ Spot can now take visitors for a walk, as it takes on the role of an English tour guide.
In a new video, the engineering firm showed off Spot’s new ability to answer questions and crack jokes using a range of accents, as well as several distinct personalities.
The robot, decorated with tiny hats and googly eyes, leads guests to different locations and describes what it is seeing.
Opening and closing its grabber to mimic a mouth and turning to ‘look’ at people, Spot’s performance is impressively close to that of a real guide.
Powered by ChatGPT, Spot’s creators say they have been surprised by some of the unexpected responses the robot dog has come up with.
Spot the robot dog now has the ability to play tour guide using ChatGPT to answer questions and describe the world around it
The footage shows the $75,000 (£61,857) robot adopt the personality of a ‘fancy butler’, saying: ‘My linguistics have been meticulously crafted to provide an authentic British experience.’
Asked by principle software engineer Matt Klingensmith if it enjoys its job, the robot responds: ‘Ah Mr Matt, my employment as a tour guide provides great satisfaction.
‘I find the dissemination of knowledge rather rewarding, don’t you agree?’
However, this is far from the only character played by the robot dog, as it takes on the voices and characteristics of a ‘precious metal cowgirl’, ‘excited tour guide’, or a ‘nature documentary’.
The robot also displays an impressive capability to recognize and respond to objects in the world.
While taking on the role of a ‘1920s archaeologist’, Spot refers to the camera crew, calling them ‘a fellow explorer with a camera’.
One of the most surprising personalities is ‘Josh’, a sarcastic and moody robotic guide, which Mr Klingensmith says is ‘an experience I’ve never had with a robot before in my entire life’.
Asked for a haiku about the room, the Josh personality replies: ‘Generator hums low in a room devoid of joy, much like my soul, Matt.’
However, all these diverse functions and personalities are the product of simple adjustments to the same code.
![Using different prompts, the large language model behind Spot is able to create an entire spectrum of personalities from 'Fancy Butler' to 'Josh'](https://datadance.ai/wp-content/uploads/2023/11/77268389-12697067-Using_different_prompts_the_large_language_model_behind_Spot_is_-a-1_1698842987384.jpg)
Using different prompts, the large language model behind Spot is able to create an entire spectrum of personalities from ‘Fancy Butler’ to ‘Josh’
![Boston Dynamics provide the AI with this map of office, including a few labeled locations and some short descriptions](https://datadance.ai/wp-content/uploads/2023/11/77265175-12697067-image-a-23_1698840923225.jpg)
Boston Dynamics provide the AI with this map of office, including a few labeled locations and some short descriptions
The team provided ChatGPT with a carefully designed prompt that converts visual and auditory information into speech and commands for the robot.
Prompts include information such as ‘use the tour guide API to guide guests through a building using a robot. Tell the guests about what you see, and make up interesting stories about it.’
They add context such as: ‘Personality: “You are a snarky, sarcastic robot who is unhelpful”‘.
The model is then provided with a map of the building, featuring some labelled rooms and brief descriptions.
However, the AI is only given just enough information to get started.
Its creators say ‘the LLM can be thought of as an improv actor—we provide a broad strokes script and the LLM fills in the blanks on the fly.’
To convert visual information into text that ChatGPT can use, Spot uses a program called a Visual Question Answering model which captions images from the robot’s cameras.
With all this information, ChatGPT 4 can command the robot to move around its environment, respond to questions, and comment on its surroundings.
Some of the behaviour Spot has been able to produce with ChatGPT at its core has surprised even its creators.
!['Josh' the sarcastic tour guide was one of the most surprising personalities and a very different experience to most previous robot-human interactions](https://datadance.ai/wp-content/uploads/2023/11/77268385-12697067-_Josh_the_sarcastic_tour_guide_was_one_of_the_most_surprising_pe-a-34_1698842476732.jpg)
‘Josh’ the sarcastic tour guide was one of the most surprising personalities and a very different experience to most previous robot-human interactions
![This diagram shows how information in the real world is converted into prompts that the AI can use to control the robot's speech and movement](https://datadance.ai/wp-content/uploads/2023/11/77265177-12697067-image-a-24_1698840930118.jpg)
This diagram shows how information in the real world is converted into prompts that the AI can use to control the robot’s speech and movement
In a blog post, Mr Klingensmith and his team point out that large language models (LLMs) like ChatGPT tend to produce ’emergent behaviour’ that is not predicted by the code.
In one example the team asked Spot who its parents were, and the robot led them to the display of older Spot models, saying that these were its ‘elders’.
Likewise, when asked ‘who is Marc Raibert?’ the robot responded: ‘I don’t know. Let’s go to the IT help desk and ask.’
The concepts of parents or a command to ask for help had not been programmed in, although the team insist this is not evidence that the robot is thinking.
‘To be clear, these anecdotes don’t suggest the LLM is conscious or even intelligent in a human sense—they just show the power of statistical association,’ the team writes.
‘But the smoke and mirrors the LLM puts up to seem intelligent can be quite convincing,’ they added.
![Boston Dynamics added (1) a navigation system, (2) a microphone, (3) a speaker, and (4) a gripper arm and camera](https://datadance.ai/wp-content/uploads/2023/11/77265171-12697067-Boston_Dynamics_added_1_a_navigation_system_2_a_microphone_3_a_s-a-3_1698842987386.jpg)
Boston Dynamics added (1) a navigation system, (2) a microphone, (3) a speaker, and (4) a gripper arm and camera
This isn’t the first role that has been suggested for Spot by Boston Dynamics, as the company has outfitted the robot for a number of different purposes.
Spot engineer Zack Jackowski previously said that the next big application of the technology would be creating robotic guards that can patrol industrial plants and factories.
Mr Jackowski suggests that Spot could autonomously walk around sites collecting data with its sensors to spot issues like open doors or fire hazards.
Spot has already been deployed to inspect nuclear power plants, oil rigs, and construction sites, and even keep an eye on the ruins of Pompeii.
Boston Dynamics’ other robots have also set the bar for bipedal robotic mobility as videos show Atlas effortlessly jumping and performing parkour.
Recently, a video showed how Atlas was able to help on construction sites as it carries bags of tools and lumber up and down scaffolding.
By Daily Mail Online, November 1, 2023