Tools to scale robotic process automation enhanced by Microsoft

Microsoft has announced significant advances to its Power Automate platform to help scale robotic process automation (RPA) infrastructure. Key advances include new capabilities for understanding business processes, collaborative bot development, and scaling RPA software bots with virtual desktops. Microsoft is a relative latecomer to the RPA playing field, but is growing this capability quickly — thanks to its existing strengths in office productivity apps, Windows integration, and Azure cloud infrastructure.

The field of RPA started as a way to program sophisticated macros for automating repeating tasks like copying and pasting data between two business apps. Gartner has suggested the future of this field, known as hyperautomation, includes finding ways to identify automation opportunities and program automations, then scale the deployment of the automation more efficiently. Microsoft’s newest updates tick the boxes of significant progress on all three of these aspects.

Microsoft’s Power Automate general manager Stephen Siciliano told VentureBeat, “Understanding up-front which processes have the most wasted time and the highest potential for automation will be extremely valuable for them.” His team aims to provide a single end-to-end product that improves the complete hyperautomation lifecycle and is deeply integrated into Microsoft 365 and built into Windows OS.

New process mining improvements

Many enterprises have hundreds or even thousands of processes that could be automated. The initial step in figuring out how to identify automation opportunities at scale lies in automating the process of seeing what procedures enterprises are commonly repeating. One approach called task mining watches over a  shoulder to see how someone clicks and types their way through a process using a sophisticated macro recorder. Earlier this year, Microsoft provided the ability to bootstrap an automated flow from a task mining diagram, making it easier to go from seeing the process to optimizing it.

The second approach, known as process mining, analyzes enterprise application event logs to reverse engineer a process diagram. This is important, especially when a process spans many users and enterprise applications. Microsoft’s new process mining capability takes advantage of existing Power Query connectors for Power BI and Azure Synapse. So although the specific process mining aspect is new, Power Query supports data ingestion from the hundreds of enterprise applications today.  In addition, Microsoft acquired Clear Software last week, which will improve connectivity with enterprise applications like SAP and Oracle.

Microsoft’s process mining and task mining capabilities approach visualization and the understanding of a process from different angles, which are slowly becoming more connected. Process Mining uses event data from systems of record to understand and analyze the process in a company. Task Mining fills in the gaps in the process identification which event data cannot see, what the human does in the middle, using recorders and RPA techniques.

“In the future, we anticipate that the lines between those two will blur more, and the focus will be on the full end-to-end process, no matter how many tasks or systems it touches,” Siciliano explained.

Collaborative bot development

A second significant advancement improves collaborative bot development. The process and task mining capabilities help generate a template for how a bot is supposed to click and type its way through a particular workflow. But humans must then look over these and identify how to design a more efficient process or respond to common problems.

Microsoft has added collaborative development workflows that allow different experts, including bot developers, business process analysts, subject-matter experts, front-line users, and compliance teams, to collaboratively make comments, recommend changes, and revert or accept the recommended changes. This takes advantage of the same commenting infrastructure used in Word.

One concern is that RPA bots could copy data to applications lacking appropriate protections or to physical locations in violations of regulations like GDPR on where data can be stored. New data loss prevention capabilities widen Microsoft’s existing capabilities for labeling and tracking sensitive data into RPA automation.

Scaling deployment

Historically there have been two sets of machines that enterprises have to manage for RPA: the servers that orchestrate the work that needs to happen and the machines the bots run on. Power Automate automatically manages the machines for RPA orchestration in the cloud. However, enterprises now must manage the machines that contain the running bots. Microsoft is previewing the Azure Desktop Starter Kit, which can automate this second aspect as well.

Siciliano said, “This will make it possible for IT to focus less on the raw tasks of setting up and scaling machines, and more on enabling more users in their organization to build out bots.” This could also simplify governance since the IT team can set the exact policies on its use and who uses it.

Better integration between RPA and Azure desktops promises to simplify the processes of setting up and scaling the appropriate machine configurations for RPA deployments.

“We’ve heard from customers today it can take days to weeks to bootstrap and scale infrastructure,” Siciliano added. “This means not only are citizen developers blocked (and thus not being productive) during that time, but also, there’s a lag time of machines sitting idle. All of that goes away once you can scale automatically.”


VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link


What is robotic process automation?

Join Transform 2021 this July 12-16. Register for the AI event of the year.

Robotic process automation (RPA) — technology that automates monotonous, repetitive chores traditionally performed by human workers — is big business. Forrester estimates that RPA and other AI subfields created jobs for 40% of companies in 2019 and that a tenth of startups now employ more digital workers than human ones. According to a McKinsey survey, at least a third of activities could be automated in about 60% of occupations. And in its recent Trends in Workflow Automation report, Salesforce found that 95% of IT leaders are prioritizing workflow automation, with 70% seeing the equivalent of more than four hours of savings per employee each week.

Switching repetitive tasks to RPA functions not only eliminates errors, it also garners significant cost savings. That’s because RPA addresses bottlenecks with workflows, data, and documentation while providing audit trails and reducing compliance expenses and risks. RPA can also boost legacy integration and record digitization and enable data-driven decisions and “path-to-cognitive” technologies, according to Technologent’s Kevin Buckley.

But as RPA expands to increasingly complex domains, the technology itself grows more complicated. This makes it harder for business decision-makers to understand where and when RPA might be appropriate, factoring in their industry and particular challenges.

RPA: What is it?

RPA is the category of software that automates tasks traditionally done by a human, using software robots that follow a set of rules and interact with enterprise systems via user interfaces. These robots can complete repeatable tasks, perform system integrations, and automate transactions from task-level to enterprise-level via scheduled orchestration.

There’s nuance within this definition, however. RPA often begins with what’s called backend task discovery, or process mining. An RPA client pulls log data from existing systems — including desktop, IT, and email apps and workflows — to identify root cause issues through recommendations, KPIs, and more. Task capture is the next step in the onboarding chain. It comes as employees move through a work process they’d like to automate by taking screenshots, using drag-and-drop designers, and pulling data like window names and descriptions together into a process definition document.

Most RPA platforms leverage AI to map tasks to automation opportunities and identify the most frequent patterns from the data, recording metrics from apps, including steps and execution time. Document understanding capabilities allow these platforms to ingest, analyze, and edit PDFs and images, even those with handwriting, checkboxes, signatures, rotated or skewed elements, and low resolutions.

Computer vision algorithms enable RPA software to recognize and interact with on-screen fields and components like Flash and Silverlight. Drawing on AI, optical character recognition, and approximate string matching, RPA robots can “see” virtual desktop interfaces via clients like Citrix, VMWare, Microsoft RDP, and VNC.

Types of robots

Not every RPA robot is created equal. Platforms such as UiPath offer three types: attended, unattended, and hybrid robots.

Attended robots act like a personal assistant that resides on a user’s computer to take a series of user-triggered actions and complete simple, repetitive tasks. By contrast, unattended robots require very little intervention to perform intensive data processing and data management workloads. Hybrid robots, as their name implies, are a combination of attended and unattended robots and deliver user support and backend processing in a single solution.

Choosing which type of RPA robot to deploy depends on the application. Because attended robots are tailored to the requirements of the user, they are a shoo-in for contact centers, field sales, retail, service engineers, and insurance agents. The scalable nature of unattended robots makes them a fit for application, claims, and invoice processing, as well as data and documentation search and retrieval. As for hybrid robots, they tend to work best in end-to-end scenarios like HR management, application processing, service delivery, and customer support and engagement.

Regardless of the bot type, RPA platforms typically leverage scalability to their technological advantage. For instance, startup WorkFusion claims its bots aggregate and share learnings across the bot ecosystem to create network effects from which all of its customers benefit.


RPA software lets customers manage up to thousands — or tens of thousands — of robots from a single dashboard. Customers can view the robots’ tasks and supporting documents, take remedial actions in the event of a bottleneck, and visualize automation complexity and payback costs. Some software offers toolsets developers can use to borrow prebuilt automation activities, integrate third-party components, and share and reuse components. RPA software also typically lets customers import their own machine learning models or choose from a marketplace of prebuilt options and keep tabs on versioning.

In the areas of AI and machine learning, Indico and other RPA providers apply techniques like transfer learning — where a model tailored to one task is used for another, related task — to deploy to unstructured content more effectively. The company’s out-of-the-box models, which were trained on large datasets of documents, ostensibly learn to analyze industry-specific data from just a few hundred training examples.

Connectors also add enormous value in the world of RPA. For example, RPA startup Bizagi integrates with Azure Cognitive Services to automatically recognize new kinds of paper forms and extract data from them. Sources include contracts, claims forms, emails, spreadsheets, purchase orders, and field reports. And Blue Prism offers a library that gives partners and customers the ability to create, share, and deploy plugins for the company’s RPA solutions.


RPA can handle a vast number of different tasks, from contract audits and customer onboarding to commercial underwriting, financial document analysis, mortgage processing, billing form reviews, and insurance claims analysis. That is one reason the overall RPA market is expected to grow by more than 7% annually over the next few years to reach $379.87 million by 2027, up from $182.8 million in 2019.

Early in the pandemic, RPA companies like Automation Anywhere worked with health care centers to implement bots and automate laborious processes. For example, Olive, a Columbus-based health care automation startup, used a combination of computer vision and RPA to support COVID-19 testing operations by simplifying manual data entry. UiPath partnered with a Dublin-based hospital to process COVID-19 testing kits, enabling the hospital’s on-site lab to receive results in minutes and saving the nursing department three hours per day, on average.

Beyond health care, Gryps, an RPA startup focused on the construction industry, is applying machine learning to organize construction project files and documents. For the Javits Convention Center in New York, Gryps’ software automatically ingested over 20,000 documents and 100,000 data points, collated them, and handed them over to the Javits team, with estimates putting the savings at hundreds of hours of staff time.

The number of industries RPA touches continues to grow, with a Deloitte report predicting the technology will achieve “near universal adoption” within the next five years. According to the same report, 78% of organizations that have already implemented RPA — which see an average payback period of around 9 to 12 months — expect to “significantly” increase their investment in the technology over the next three years.

Security challenges

This isn’t to suggest that RPA is without challenges. The credentials enterprises grant to RPA technology are a potential access point for hackers. When dealing with hundreds to thousands of RPA robots with IDs connected to a network, each could become an attack vessel if companies fail to apply identity-centric security practices.

Part of the problem is that many RPA platforms don’t focus on solving security flaws. That’s because they’re optimized to increase productivity and because some security solutions are too costly to deploy and integrate with RPA.

Of course, the first step to solving the RPA security dilemma is recognizing that there is one. Realizing RPA workers have identities gives IT and security teams a head start when it comes to securing RPA technology prior to its implementation. Organizations can extend their identity and governance administration (IGA) to focus on the “why” behind a task, rather than the “how.” Through a strong IGA process, companies adopting RPA can implement a zero trust model to manage all identities — from human to machine and application.

A privileged access management (PAM) setup that can secure and govern RPA systems can also help. PAM systems allow enterprises to secure, control, and audit the credentials and privileges RPA technology uses without compromising the return on investment (ROI).


RPA challenges don’t stop at security. Deloitte reports that 17% of organizations face employee resistance when piloting RPA and that 63% of those organizations struggle to meet time-to-implement expectations. But the RPA’s return on investment often outweighs difficulties in deployment. According to the Everest Group, top performers earn nearly 4 times on their RPA investments, while other enterprises earn nearly double. And Gartner estimates that by 2024, organizations can lower operational costs 30% by combining automation technologies like RPA with redesigned operational processes.

“The first wave of robotic process automation brought the power of technology to users’ desktops in all industries and companies of all sizes. Today, we see a second wave emerging,” WorkFusion CEO Alex Lyashok recently told VentureBeat via email. “Cloud-based, AI-enabled robots [are] bringing intelligent automation to all enterprises.”


VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Repost: Original Source and Author Link


Robotic lab assistant is 1,000 times faster at conducting research

Researchers have developed what they say is a breakthrough robotic lab assistant, able to move around a laboratory and conduct scientific experiments just like a human.

The machine, designed by scientists from the UK’s University of Liverpool, is far from fully autonomous: it needs to be programmed with the location of lab equipment and can’t design its own experiments. But by working seven days a week, 22 hours a day (with two hours to recharge every night), it allows scientists to automate time-consuming and tedious research they wouldn’t otherwise tackle.

In a trial reported in Nature today, the robot’s creators, led by PhD student Benjamin Burger, say it was able to perform experiments 1,000 times faster than a human lab assistant, with that speed-up mostly due to the robot’s ability to work around the clock without breaks.

But Professor Andy Cooper, whose lab developed the robot, tells The Verge that speed is not necessarily the point. The main benefit of a tool like this, he says, is that it allows scientists to explore avenues of research they wouldn’t waste a human’s time on.

“The idea is not to do things we would do faster, but to do bigger, more ambitious things we wouldn’t otherwise try to tackle,” says Cooper.

The robot can even work in the dark, thanks to laser-based LIDAR navigation sensors.
GIF: The Verge via Nature

For its showcase research, the robot was tasked with finding substances that can speed up chemical reactions that create hydrogen from light and water, an area of research useful to many industries, including green energy production. The robot was programmed with the basic parameters of the experiment but used algorithms to decide how to change 10 different variables, such as the concentration and ratio of chemical reagents.

Over an eight-day period, the machine carried out 688 experiments to find how to create more efficient reactions. It mixed samples in glass vials, exposed them to light, and analyzed the results using gas chromatography.

The results of the tests are promising, but Cooper notes he wouldn’t have asked a human to even carry out the research, given how much time it would take and how it might distract them from their studies. “If you asked a human to do it they could lose their whole PhD,” he says. But for a machine, the potential benefits outweigh any loss of time.

The robot itself is not without its expenses, of course. The basic hardware costs between $125,000 and $150,000, says Cooper, and it took three years to develop the controlling software. The machine navigates labs using LIDAR, the same laser-based technology found in self-driving cars. That means it can operate in the dark, and it won’t get confused by changing lighting conditions. It manipulates lab equipment using an industrial arm built by German robotics firm Kuka, though some machines have to be adapted to its use.

Lee Cronin, a professor of chemistry at the University of Glasgow who also uses automated equipment in his work, said the main advance of the research was the robot’s mobility and its ability to use human equipment. But he cautioned that such machines would still be “niche” in the future, as deploying them won’t always make sense in terms of costs.

“I’m not sure robotic assistants like this are going to be useful in a general sense but in repetitive experiments … they could be excellent,” Cronin told The Verge by email.

Cooper says that although the upfront costs are expensive, they’re not unusual compared to lab equipment, which often costs hundreds of thousands of dollars. He also says that while some scientific research can be automated using static machines, the flexibility of a robot that can be reprogrammed to take on a variety of tasks is ultimately more useful.

“The idea was to automate the researcher, rather than the instrument,” says Cooper. “It’s a different paradigm.”

Working 22 hours a day without a break means tackling time-consuming research with ease.
Image: University of Liverpool

Cooper and his colleagues have already formed a spinoff company named Mobotix to commercialize the work, and they plan to have a “more fully commoditized product” ready in roughly 18 months. “We have an idea for a range of products,” he says. “A robot technician, a robot researcher, and a robot scientist, all with different levels of capabilities.”

Although the development of new robotic technology often leads to fears about loss of work through automation, Cooper says students who saw the robot were more likely to imagine how it could help them.

People were skeptical at first, but there was “general amazement when it first started to work,” he says. “Now people are starting to think ‘if I don’t use this hardware I might be at a massive disadvantage.’”

Repost: Original Source and Author Link

Tech News

Roomba robotic vacuum cleaner software fix promised in the coming weeks

The company behind the Roomba robotic vacuum cleaners, iRobot, has announced that the software updates it issued have been causing problems for some of its robotic vacuum cleaners. Models specifically impacted by the software issue are Roomba i7 and s9 robots. The company states that it is currently working on a software upgrade to fix issues owners have complained about.

The issue for owners of the impacted robotic vacuum cleaners is that the update will be rolled out over the next several weeks. Owners of impacted Roomba vacuums say that the recent 3.12.8 firmware has caused navigation issues with the vacuum cleaners. After applying that software update, one user says their Roomba acted “drunk,” spinning around and bumping into furniture.

The owners also said the vacuum cleaned in strange patterns and would get stuck in an empty area along with not being able to return home to its dock. Other users have reported that the updates wiped out environmental maps made by the Roomba vacuums essential to their cleaning function. Impacts from the bad software update have caused a variety of issues, with some taking longer to clean than usual. Units unable to make it back to their docking station are unable to charge, leaving them unusable.

iRobot has been working with users impacted to roll back the update, but even after the update is rolled back some report they still have issues. Some users who were promised help rolling back the software update say they have waited weeks and still haven’t received help. These robotic vacuum cleaners are typically quite expensive, and a software update leaving them unusable understandably angers owners.

Repost: Original Source and Author Link


The Creeping Normalization of Robotic Police Officers


The classic 1987 movie RoboCop is said to be set roughly 20 years from now, but it looks like 2021 will be the beginning of the era of robotic law enforcement. A growing number of police departments around the country are purchasing robots for police work, and as this behavior becomes normalized, major concerns are starting to arise.

The NYPD purchased a robot dog earlier this year that is apparently capable of opening doors. The same kind of robot police dog has been tested out by the Massachusetts State Police. The use of drones by police departments has skyrocketed during the COVID-19 pandemic. Police departments around the country have purchased the weeble-wobble-looking robot Knightscope robot that apparently enjoys running over children’s feet and ignoring people who need help.

“Slowly introducing these robots as fun novelties is absolutely going to normalize a type of policing that is done by robot and by algorithm.”

Matthew Guariglia, a policy analyst at the Electronic Frontier Foundation, tells Digital Trends that the kinds of robots we’re starting to see police departments use are mostly for show, but that doesn’t mean there isn’t cause for concern. He has researched the companies that sell these robots extensively and argues that these robots are mostly a “public relations move” at this point. However, while they’re fairly innocuous right now, even these PR stunt police robots could open the door to something we should worry about.

MA State Police

“We’ve seen from the inside of the companies selling these robots that most of what they’re selling them based on is social media engagement and good press for your department,” Guariglia says. “Very little of it is about the actual policing the robot can do. I think by slowly introducing these robots as fun novelties that you can take selfies with it is absolutely going to normalize a type of policing that is done by robot and by algorithm.”

You might find a robot police dog to be cute, or perhaps think it’s funny when a robotic cop falls into a fountain, but what these machines could be bringing us closer to is much more pernicious. Guariglia says we could soon see a proliferation of different kinds of police robots, and accountability could become a major problem.

#Exclusive Now that’s some #dog! 4 legged, futuristic, first responder is helping NYPD w the most dangerous jobs. Starting at 4, Digidog, wait until you see what it can do! @abc7ny @NYPDnews @nypd_taru @nypd_esu #abc7ny #crime #technology #protect #officers #police #k9 #nyc

— Kemberly Richardson (@kemrichardson7) December 10, 2020

“I worry about when we move out of the stage where police robots are just photo opportunities. We’re going to eventually have to confront the scenario in which robots that police have to make decisions, and when the time comes that a police robot makes the wrong decision — somebody gets hurt or the wrong person gets arrested — police robots are not people,” Guariglia says. “You can’t reprimand them.”

What if the robot falsely identifies them as a criminal and gets them arrested? Who will be held responsible for that? You can’t fire a robot or charge it with a crime.

It would not surprise me at all if we saw robot police dogs at protests that are equipped with pepper spray on their backs.”

Guariglia also notes that these robots can easily be outfitted with all kinds of surveillance technology, and they could become “roving surveillance towers.” He says a robot might be assigned to a high-crime neighborhood to conduct near-constant surveillance and call the police when it suspects it’s identified a criminal, whether it has or not.

Imagine you’re walking down the street and a police robot orders you to stop. It believes you’re wanted for a crime and calls the police on you. The police arrive and take you to jail. You’re released once they figure out that they’ve arrested the wrong person. They blame the robot’s algorithm, and there’s nothing you can do about it. It’s a dystopian future we could be fast approaching.

Furthermore, Guariglia says we might not be too far away from police robots starting to carry weapons, which would further complicate things. He notes that a police robot was already equipped with C4 and used to kill a mass shooter in Dallas in 2016, so it’s not unreasonable to assume we could see police robots equipped with weapons in the not-too-distant future.

“In extenuating circumstances, police are more than willing to impromptu weaponize robots, and we’ve already seen proposals over the years to put tasers or pepper spray on little flying drones, so it would not surprise me at all if we saw robot police dogs at protests that are equipped with pepper spray on their backs,” Guariglia says.

Eyewitness News/abc7NY

One possible solution to this problem would be communities preventing police departments from getting these kinds of robots in the first place. A city could adopt a Community Control Over Police Surveillance (CCOPS) ordinance, which would make it so the police department has to submit a request to the city council outlining what kind of technology it hopes to purchase before being allowed to purchase it. Guariglia says that police departments might be less inclined to purchase this kind of technology if they had to get approval from the public.

“We need more transparency and public control over what police departments spend their money on,” Guariglia says.

We won’t be seeing the kinds of humanoid robots that were featured in RoboCop patrolling our streets any time soon, but police robots are becoming a reality, and what appears harmless now could become very harmful in the near future. These robots could invade our privacy, get people falsely arrested, and even end up injuring or killing people in some circumstances. It may all depend on how the people respond to this impending threat.

Editors’ Choice

Repost: Original Source and Author Link