Perception and Manipulation: The Unsolved Picking Problem in Warehouse Robotics
Even when a robot can navigate reliably to a shelf location, picking arbitrary objects from cluttered bins remains one of the hardest unsolved problems in warehouse robotics. The inaugural Amazon Picking Challenge made this explicit: analysis from Rutgers University (2018) found that competing teams faced severe difficulties in perceiving, recognising, and grasping the wide variety of objects stored on warehouse shelves, with perception and grasp planning being the primary sources of failure.
“The Amazon Picking Challenge goal — to design an autonomous robot to pick items from a warehouse shelf, a task currently performed by human workers — exposed how far robotic capability lags behind human dexterity in unstructured pick environments.”
Deep learning-based perception has advanced considerably since 2018, but integration challenges persist. The NimbRo Picking system from the University of Bonn, which achieved second place in the 2017 Amazon Robotics Challenge, used a transfer-learning perception pipeline and dual-arm coordination. Yet it required a custom turntable capture system to quickly adapt to new items — a workflow that is impractical for warehouses with rapidly changing SKU inventories. Achieving truly general object recognition and grasp planning without per-item training data remains an open research problem, as confirmed by standards bodies including IEEE in its robotics roadmaps.
The DoraPicker system from the University of Hong Kong (2016) used 6D pose estimation to pick general objects from shelves. However, 6D pose estimation is sensitive to occlusion and clutter — conditions that are pervasive in real unstructured warehouses — making this approach unreliable at scale without additional robustness mechanisms.
Handling arbitrarily shaped, large, or irregularly packed objects introduces additional manipulation challenges. Research from the University of Bristol’s Robotics Laboratory (2022) demonstrated collective transport of arbitrarily shaped objects using robot swarms with decentralised decision-making, where robots had no prior knowledge of object shape or size. While promising in simulation, scaling such swarm approaches to real warehouse conditions introduces new control and coordination overhead. Separately, research from the University of Bremen (2022) confirmed that even relatively simple insertion tasks involve stochastic variation that requires search motions, adding time cost to every operation.
The Amazon Picking Challenge (analysed by Rutgers University, 2018) found that perception failures and grasp planning errors were the primary sources of failure for autonomous robotic picking systems attempting to handle the wide variety of objects stored on warehouse shelves.
Explore the full patent landscape for warehouse robot perception and manipulation systems.
Search Patents in PatSnap Eureka →Human-Robot Safety: Infrastructure and Regulatory Gaps That Block Cobot Deployment
The co-presence of humans and robots in unstructured warehouses creates safety and communication challenges that are categorically different from fully automated facilities. Safe human-robot collaboration requires not only physical safety mechanisms but also reliable wireless infrastructure and intent recognition — and the standards governing all three remain immature.
Research from the University of Zagreb (2020) addressed the signal propagation challenges in automated collaborative warehouses, finding that UWB antenna placement on humans and robots must be carefully optimised to ensure safe communication ranges, accounting for rack-induced multipath effects and surface roughness. Physical rack geometry alone creates complex wireless propagation environments that can degrade the collision-avoidance communication on which co-located human-robot safety depends.
VTT Technical Research Centre of Finland (2019) identified that the single most significant barrier to cobot adoption — reported across robot manufacturers, system integrators, end-users, and academics — was a lack of knowledge about potential applications, safety legislation, and ease of use.
Beyond wireless infrastructure, recognising human intentions in real time is essential for safe robot behaviour. The University of Zagreb (2017) presented a Markov Decision Process-based framework for human intention recognition in robotised warehouses, showing that estimating worker goals from observations enables safer and more efficient robot behaviour. Without such intent modelling, robots operating near humans risk either becoming unsafe or introducing excessive conservatism that degrades operational throughput.
VTT Technical Research Centre of Finland (2019) found that lack of knowledge about safety legislation and application cases was the single most significant barrier to cobot deployment across all stakeholder groups. RWTH Aachen (2015) separately confirmed that appropriate safety standards to ensure occupational safety in direct human-robot cooperation are missing and represent a main barrier. This regulatory gap is documented across multiple jurisdictions and remains unresolved as of 2024, as noted by bodies including ISO in its collaborative robot safety standards work.
Fanuc’s active patent for a Collaborative Robot Risk Assessment Guidance Device and Method (2024) addresses this gap by providing structured, tool-guided risk assessment workflows for collaborative robots, enabling systematic identification of hazard candidates and automated risk level evaluation. Yet even with such tools, integrating them into the rapidly changing conditions of an unstructured warehouse — where human activities, cargo types, and traffic patterns shift continuously — remains a significant operational challenge. Research standards organisations such as WIPO track the growing volume of collaborative robot patents as evidence of sustained industry investment in this problem.
Multi-Robot Coordination: When Fleet Size Becomes the Problem
Deploying multiple robots simultaneously in an unstructured warehouse introduces coordination problems that grow non-linearly with fleet size. Routing conflicts, heterogeneous robot capabilities, and battery management must all be solved concurrently — and no single algorithm presently handles all three at production scale.
Research from Sungkyunkwan University (2022) demonstrated that operating multiple mobile robots in an automated warehouse requires handling complex routing, path conflicts, and sparse reward structures in reinforcement learning. The proposed MARL dual-reward framework represented an improvement over baseline approaches but not a complete solution. Related work from the same institution (2021) confirmed that reinforcement learning approaches show promise but require careful tuning for real warehouse topology before deployment.
Static routing algorithms cannot handle multi-robot warehouse situations where robots have different capabilities — a condition that is the norm in real mixed-fleet warehouses — according to research from the University of Western Macedonia (2021). Real warehouse fleets routinely include AMRs, AGVs, and human workers operating simultaneously in the same space.
Routing algorithms must also account for energy consumption and battery management. The University of Western Macedonia (2022) developed a methodology combining Dijkstra’s and Kuhn-Munkres algorithms to optimise pathfinding while explicitly deciding when robots should divert to charging stations. Its complementary work (2021) highlighted that static algorithm designs cannot handle multi-robot situations where robots have different capabilities — a condition that is the norm, not the exception, in real mixed-fleet warehouses. Standards for multi-robot interoperability are an active area of work at organisations including IEEE.
The management of warehouses with robots of heterogeneous types compounds these coordination difficulties. Caja Elastic Dynamic Solutions’ patent (2022) addresses this directly, proposing methods for scheduling and allocating tasks to robots based on task-related properties that differ between robot types. Deep reinforcement learning approaches such as Deep Q-Learning for autonomous warehouse navigation, studied at North South University (2021), show the capacity for robots to navigate and avoid obstacles in simulation, but their authors acknowledge limitations in the transition to real multi-robot physical deployments.
Locus Robotics’ patented Dynamic Item Putaway Management system (2022) represents a commercially deployed partial solution: rather than full autonomy in unstructured environments, the system pairs mobile robots with human operators, dynamically recalculating optimised routes as items are added and removed from the robot’s tote array. This human-in-the-loop architecture is itself a symptom of the fundamental obstacles that prevent fully autonomous unstructured warehouse operation — a design choice catalogued in PatSnap’s innovation intelligence database.
Analyse multi-robot coordination patents and identify white-space opportunities with PatSnap Eureka.
Explore Patent Data in PatSnap Eureka →Organizational and Software Barriers: Why Technical Solutions Alone Are Not Enough
Even where technical solutions exist, organizational readiness and software infrastructure present independent deployment obstacles that are frequently underestimated by engineering teams. Human discomfort, software fragility, and cultural resistance each impose real costs on warehouse robot programmes.
Research from Delft University of Technology (2021) showed that human discomfort — measured in terms of ergonomic and cognitive strain — is an increasingly important objective in collaborative warehouse automation, but that current robotic assignment policies rarely incorporate human factors. A related study from Open Universiteit Netherlands (2021) examining high-volume distribution centres found that resistance to change, organizational culture, communication strategy, and leadership style are critical determinants of whether cobot implementations succeed. Employees were found to be hesitant or resistant when implementation planning was inadequate.
“Resistance to change, organizational culture, communication strategy, and leadership style are critical determinants of whether cobot implementations succeed in high-volume distribution centres — with employees hesitant or resistant when implementation planning is inadequate.”
On the software architecture side, Chalmers University of Technology’s study of 335 real open-source ROS-based robotic systems (2021) found that while ROS is the de facto standard, systems are growing in complexity faster than documented software architecture practices can accommodate, with quality attributes and architectural documentation remaining poorly addressed. This software fragility translates directly into operational unreliability when warehouse robots must handle the dynamic, unpredictable conditions inherent to unstructured environments. The challenge of software quality in complex robotic systems is recognised in guidelines published by NIST on autonomous system performance measurement.
The University of Essex’s holistic overview (2018) concluded that the lack of observable autonomy in deployed systems is a consequence of both the complexity of the problem and the lack of proven reliability of autonomous solutions where high success rates are required. Unstructured warehouses, while less extreme than nuclear or deep-sea environments, share enough uncertainty and unpredictability to make truly high-autonomy deployment technically non-trivial and commercially high-risk for most operators.
Chalmers University of Technology’s study of 335 real open-source ROS-based robotic systems (2021) found that robotic software systems are growing in complexity faster than documented software architecture practices can accommodate. Quality attributes and architectural documentation remain poorly addressed — a gap that directly increases operational unreliability in dynamic warehouse deployments.