Autonomous Systems

Autonomous systems will eventually replace mechanized and automated systems.

The COVID-19 pandemic brought unprecedented economic disruption due to its global impact. It forced manufacturers to innovate quickly when their human workforce was unavailable. Plans for integrating AI went into overdrive. The challenge was to create AI-powered systems to coordinate with people, freeing them from dangerous and dirty work. Some ways they can assist are:

• Navigating new challenges by offering calculated advice to workers.
• Assisting operators as they conduct complex operations.
• Acting autonomously to complete assigned tasks.

With expertise programmed into AI, and distributed across autonomous or semi-autonomous systems, manufacturers improve productivity, reduce onboarding resources for new hires, and maintain knowledge that future hires can use. When taught various methods for approaching a task, AI can respond accurately and effectively to evolving environments. It draws on the best minds, aggregates their knowledge and offers solutions.

“The autonomous system elevates its counterpart while providing new learnings for future efforts through instant assessments and accessible innovations.”

AI can manage systems in a way that is proactive rather than reactive. They can assist in production scheduling and performance strategizing.

Software is flexible. With it, humans can connect to bits in a flexible way. Currently, autonomous systems are custom-designed by experts for use in specialized applications. These silos are fragile and liable to collapse in favor of horizontal, open-ended systems. What if you could teach a machine how to navigate the real world? In a digital, parallel reality, AI can test different ways an autonomous system can solve a problem. It can even anticipate “black swan” events. However, engineers must ensure the machine’s safety in the real world, and most importantly, explain its decision-making.

“Sense, Plan, Act” is the feedback loop autonomous systems deploy to interact with and operate in an environment. 

An autonomous machine would learn the way people learn, by sensing its environment, collecting information and making decisions.

“Autonomous systems optimize equipment and processes by sensing changes and responding quickly, using AI-powered automation. This rapid learning curve is achieved because, among other reasons, autonomous systems can be infused with expert knowledge, and the system uses simulations to test and practice its strategies before acting in the real world.”

Using a simulated environment, AI would instruct it to experience its dimensions. It receives “rewards” when it makes accurate decisions. It follows the Sense, Plan, Act, feedback loop:

• Sense – In the real world, sensors relay raw data such as temperature, mass, velocity and dimension. For instance, a factory uses sensors to detect differences between size, weight and the composition of objects on a conveyer belt. A drone would measure wind velocity’s impact on a suspension bridge.
• Plan – Like humans, the machine learns from examining the data inputs, and formulates a plan. Experts guide this process. They give the machine rewards when it makes the right choices. With Project Bonsai, a subject expert could advise on a training model to teach the machine.
• Act – With the plan, the machine has the capacity to act. Usually, it will do this with a human operator monitoring it. Once it performs the action, it returns to the Sense mode to collect the results, and feed them back into the loop. The machine is constantly calibrating to optimize resource use, efficiency and effectiveness.

While there is a “sim-gap” between the training environment and the real world, the more experience the system has, the better its decisions become.

Microsoft’s “Project Bonsai” is an interface for teaching autonomous system “brains” how to learn complex tasks.

Autonomous systems have “brains” that can make decisions in real time. This is crucial for machines that must be responsive to existing conditions and deploy solutions. Control systems already exist, but operate with limited parameters. They don’t deal with uncertainty well. Microsoft offers a training system for developers to instruct computers to perform autonomous tasks. Project Bonsai is the platform in Azure that runs simulations, created by a human expert. The user sets up the sensors and controls, and the “brain” practices tasks and acquires the skill the user needs. Microsoft built a model, a small robot that can balance ping-pong balls on a plate supported by a robotic hand. The model’s goal is to keep the ball in the middle of the plate. This model demonstrates how machines can learn just like a human learns.

“Machine teaching democratizes this potential, empowering more people to innovate and closing the gap between what we can imagine and what we can accomplish.”

Currently, there are few machine learning experts, but there are many domain experts. With Project Bonsai on the Azure AI platform, these experts can use the Machine Teaching language Microsoft developed. With it, you can train and use autonomous systems at different levels:

1. Integrate – Simulators that simulate sensors, agents and environments. They leverage Inkling, the Machine Teaching language. This helps define goals, and the actions to attain them.
2. Train – Provide the “brain” with intuitive learning objectives and goals, to do assessments in real-time, and constantly update its auto-versioning controls.
3. Export – The optimized brain can be deployed on-device, at the edge or in the cloud.

Once the goals are determined, developers use Inkling to define the concepts to teach the AI and a curriculum to teach the concepts. Inkling’s concept graphs explain which simulator to use, what the simulator is doing, and tells it which actions to take, depending on the conditions it perceives. The feedback Bonsai gives the developer is based on specific objectives. In defining what and how you want to teach your AI, you can use your subject expertise and break down complex problems into specific goals and objectives.

Microsoft is a software company. It sees opportunities to create the building blocks for autonomous systems at scale. The applications for autonomous systems are extensive and can transform the way people live. They can also solve some of the world’s most intractable problems. But there are hurdles. Engineers must assure users that autonomous systems can be trusted with decision-making. That requires transparency and trust. Eventually these systems will take the drudgery out of human work, leaving people to focus on what they do best: imagine, problem-solve, innovate and experiment.

“Autonomous systems promise to unlock tremendous leaps in productivity and safety. Where will this take us in the future? It’s up to the imagination.”

Project Bonsai allows subject experts to transfer their whiteboard problem-solving into a low-code or no-code workspace. For example, airline baggage handling has historically frustrated travelers. Engineers can instruct Project Bonsai on the skills and knowledge it needs to optimize for weight, capacity, destination, and so on, so that the system “knows what to do and why.” They can import existing algorithms into the program. Project Bonsai then practices millions of scenarios in sim-mode, to prepare it for deployment in the real world. Not only does Project Bonsai add value for designers and operators closest to the equipment, it improves outcomes in the real world, facilitating more trust between consumers and service and product providers.

Industry Leaders need guidance on the many ways AI can improve their operations.

The following is summarized from the e-book: Get started with autonomous systems: A manufacturer’s use case selection guide. 

With so many new developments, industries need guidance about how AI can improve their safety, efficiency and productivity. Questions arise regarding which changes need priority, what kinds of returns on investment to expect and how to implement autonomous systems on a large scale.

“Microsoft Project Bonsai is a low-code, AI development platform that speeds the creation of AI-powered automation to improve production efficiency and reduce downtime.”

Depending on your type of operations, you can optimize autonomous systems when two conditions intersect:

• When it’s difficult to assess an environment; and
• When it’s difficult to choose the best solution for a certain scenario.

Contrast autonomous systems with other systems:

• IoT networks – for simple assessments that need complex decisions.
• Automations – for simple environments that need simple decisions.
• Machine learning – for complex assessments that need simple decisions.

Microsoft has identified four “use case” scenarios to illustrate how autonomous systems can transform your enterprise.

• Manufacturing yield – In food production, consistency is crucial, and waste occurs when output is off-spec. AI-assisted machines that extrude cereal, for instance, improve in-spec output and reduce waste. 
• Chemical processing – Petrochemical processes need highly specialized operators. Still, off-spec output is common. These operators can train AI to calculate “phase transitions, catalytic chemistry and safety parameters.”
• Logistics optimization – In a global economy, moving things around is an art form. AI can manage the flow of products to people when they want them, reduce wasted time and better allocate resources.
• Commercial energy systems – To achieve carbon-neutral energy production, efficiency is the first step. AI can regulate temperature, humidity and air quality in more efficient, sophisticated ways.

Autonomous systems can help humans imagine and build a better world by “scaling human intelligence.”

It is important to take a “people-centered” approach to AI, and understand that it replicates the vast knowledge and skills of the world’s most talented experts. Manufacturers are already grappling with an aging workforce, with 97% expressing concern about the loss of institutional knowledge when their experts retire. AI-powered autonomous systems also optimize working conditions, leading their human counterparts through “industry-first applications.” PepsiCo, for instance, used the Bonsai software to improve its snack food production capabilities. They created an autonomous system for their extruder, which often malfunctioned due to inconsistencies in cornmeal/water ratios, temperature and speed of extrusion. This system was built under consultation with operators and engineers, using a simulation that factored in complex scenarios.

“By taking a people-centric approach to AI, autonomous systems create an opportunity to partner AI alongside humans as they explore new ideas.”

Bell partnered with Microsoft to create intelligent, unmanned light aircraft. They decided to start with AI-assisted landing, which is a difficult problem because AI must identify safe landing sites before executing landing protocols. Engineers used AirSim and Project Bonsai to create simulations for two AI programs – one for identifying landing sites, the other for executing landing. They struggled to overcome certain barriers. For instance, to AI, a field of wheat or an open body of water look like a safe landing site. The program will have to run millions of times to learn all the variables in landing sites that humans can do very quickly and intuitively. While it isn’t cost-effective to run so many simulations, the challenge taught the developers a great deal about how AI learns.

The ultimate goal for autonomous systems is to operate independently, freeing up valuable time for workers to attend to unique or specialized deliverables. The machines are constantly observing, calibrating and discovering new efficiencies, pushing productivity to new levels. Workers, in the meantime, can learn higher-level skills. For operators, trust is imperative in collaborating with their AI-assisted colleagues. They won’t cooperate well if they think the machines are there to replace them. They also won’t tolerate a machine that infringes on their autonomy or privacy. AI needs people to train it and collaborate with it to be successful. It can’t replace humans, but it can augment performance to bring manufacturers’ productivity to new levels.