Kenneth Chaney, Fernando Cladera, Ziyun Wang, Anthony Bisulco, M. Ani Hsieh, Christopher Korpela, Vijay Kumar, Camillo J. Taylor, and Kostas Daniilidis

Abstract: We present M3ED, the first multi-sensor event camera dataset focused on high-speed dynamic motions in robotics applications. M3ED provides high-quality synchronized and labeled data from multiple platforms, including ground vehicles, legged robots, and aerial robots, operating in challenging conditions such as driving along off-road trails, navigating through dense forests, and performing aggressive flight maneuvers. Our dataset also covers demanding operational scenarios for event cameras, such as scenes with high egomotion and multiple independently moving objects. The sensor suite used to collect M3ED includes highresolution stereo event cameras (1280×720), grayscale imagers, an RGB imager, a high-quality IMU, a 64-beam LiDAR, and RTK localization. This dataset aims to accelerate the development of event-based algorithms and methods for edge cases encountered by autonomous systems in dynamic environments.

The dataset can be found at https://m3ed.io and the code used to pre-process the data is available at https://github.com/daniilidis-group/m3ed.

Status: Accepted to CVPRW 2023.

Paper Access: CVPRW Proceedings.

Y. Wu, T. Z. Jiahao, J. Wang, P. A. Yushkevich, M. A. Hsieh, J. C. Gee

Abstract: Deformable image registration (DIR), aiming to find spatial correspondence between images, is one of the most critical problems in the domain of medical image analysis. In this paper, we present a novel, generic, and accurate diffeomorphic image registration framework that utilizes neural ordinary differential equations (NODEs). We model each voxel as a moving particle and consider the set of all voxels in a 3D image as a high-dimensional dynamical system whose trajectory determines the targeted deformation field. Our method leverages deep neural networks for their expressive power in modeling dynamical systems, and simultaneously optimizes for a dynamical system between the image pairs and the corresponding transformation. Our formulation allows various constraints to be imposed along the transformation to maintain desired regularities. Our experiment results show that our method outperforms the benchmarks under various metrics. Additionally, we demonstrate the feasibility to expand our framework to register multiple image sets using a unified form of transformation,which could possibly serve a wider range of applications.

Status: Accepted to CVPR 2022.

Preprint available on arXiv: https://arxiv.org/abs/2108.03443

L. Shen, X. Yu, and M. A. Hsieh

Abstract: Mobile agents can form communication networks with links that emerge and disappear over time. Information transmission on such networks must pass through a sequence of links that are activated in a certain chronological order. Uncertainties in a link’s activation or deactivation risk violating the chronological order of the formation of links in paths for information transmission. The reasons for these uncertainties can be varied, but their presence prevents the application of existing topology control tools for mitigation. We propose an approach to leverage the pre-stored knowledge of the scheduled links’ events and create time-respecting subpaths accordingly. By estimating the impact of the stochastic timing on each subpath and by measuring how influential the subpaths are, we are able to measure the impact of uncertainties on any link event’s timing on the whole network’s information transmission performance. Reducing the uncertainty in the timing of those links with the highest impact can largely improve the network’s performance. Our method is tested and validated with simulation results.

Status: Accepted to ACC 2022.

Preprint available on arXiv: To come.

G. Knizhnik, P. Li, X. Yu, and M. A. Hsieh

Abstract: We present a flow-based control strategy that enables resource-constrained marine robots to patrol gyre-like flow environments on an orbital trajectory with a periodicity in a given range. The controller does not require a detailed model of the flow field and relies only on the robot’s location relative to the center of the gyre. Instead of precisely tracking a pre-defined trajectory, the robots are tasked to stay in between two bounding trajectories with known periodicity. Furthermore, the proposed strategy leverages the surrounding flow field to minimize control effort. We prove that the proposed strategy enables robots to cycle in the flow satisfying the desired periodicity requirements. Our method is tested and validated both in simulation and in experiments using a low-cost, underactuated, surface swimming robot, i.e. the Modboat.

Status: Accepted to ICRA 2022.

Preprint available on arXiv: https://arxiv.org/abs/2203.00796

T. Z. Jiahao, L. Pan, and M. A. Hsieh

Abstract: Understanding decentralized dynamics from collective behaviors in swarms is crucial for informing robot controller designs in artificial swarms and multiagent robotic systems. However, the complexity in agent-to-agent interactions and the decentralized nature of most swarms pose a significant challenge to the extraction of single-robot control laws from global behavior. In this work, we consider the important task of learning decentralized single-robot controllers based solely on the state observations of a swarm’s trajectory. We present a general framework by adopting knowledge-based neural ordinary differential equations (KNODE) — a hybrid machine learning method capable of combining artificial neural networks with known agent dynamics. Our approach distinguishes itself from most prior works in that we do not require action data for learning. We apply our framework to two different flocking swarms in 2D and 3D respectively, and demonstrate efficient training by leveraging the graphical structure of the swarms’ information network. We further show that the learnt single-robot controllers can not only reproduce flocking behavior in the original swarm but also scale to swarms with more robots.

Status: Accepted to ICRA 2022.

Preprint available on arXiv: https://arxiv.org/abs/2109.04927