I share a common vision for the Internet of Things wherein mobile readers (such as robots or mobile phones) interact with smart objects -- objects with embedded microelectronics that perform computation, sensing, communication, energy storage, and power harvesting. Recent advances in UHF RFID technology directly address this vision. Companies such as AMS and Farsens are developing commercial sensor tags, and other companies are developing their own tags and end-use applications.

As a postdoc, I explored this vision by experimenting with sensor tags that could be affixed-to or embedded-in objects or the environment to yield true embodied intelligence. Using a pair of autonomous mobile robots outfitted with UHF RFID readers, I explored several potential applications where mobile robots interacted with sensor tags to perform tasks such as: soil moisture sensing, remote crop monitoring, infrastructure monitoring, water quality monitoring, livestock monitoring, and remote sensor deployment.

As a NSF/CRA "Computing Innovation" Postdoc Fellow at Duke University, I worked with my long-time mentor (Dr. Matt Reynolds) on cutting-edge wireless backscatter systems: wireless power harvesting and high-speed (5-10 Mbps) wireless data transfer akin to souped-up UHF RFID tags. My work focused on a software-defined radio receiver to decode (in real-time) the high-speed signals reflected (radar-like) by a custom ASIC. This system was designed to capture and telemeter high-fidelity neural signals from a dragonfly in flight -- aka, a real-life "cyborg dragonfly."

This work was done with biology collaborators at HHMI who are studying dragonflies. Why dragonflies? They're capable of extreme aerial maneuvers and are one of the few (only?) insects that capture their prey in mid-flight. We'd love to know how their internal control and sensing mechanisms function.

Personal robots with autonomy, mobility, and manipulation capabilities have the potential to dramatically improve quality of life for many user populations. Unfortunately, unstructured environments present many challenges that hinder robot deployment in ordinary homes. One major challenge is error-prone object recognition using cameras and depth sensors. I sidestepped this difficulty by developing a new robot sensing modality that leverages a small amount of environmental augmentation in the form of long-range UHF RFID tags -- for which I developed robot behaviors and probabilistic inference techniques using inspirations from the radar literature.

Previous research demonstrated the utility of infrastructure tags (affixed to walls) for robot localization; in my work, I specifically focused on tagging everyday objects. Owing to their low-cost and battery-free operation, users can apply UHF RFID tags to hundreds of objects throughout their homes. The tags provide two valuable properties for robots: a unique identifier and receive signal strength indicator (RSSI, the strength of a tag's response). My graduate work explored robot behaviors and radio frequency perception techniques using robot-mounted UHF RFID readers that enable a robot to efficiently discover, locate, and interact with UHF RFID tags applied to objects and people of interest. The behaviors and algorithms explicitly rely on the robot's mobility and manipulation capabilities to provide multiple opportunistic views of the complex electromagnetic landscape inside a home environment.

Finding tagged objects and manipulating them is a foundational capability that could hasten the deployment of robots into real home environments... at least until computer vision and object recognition capabilities substantially improve -- and even then, tags may contain sensors and operate without line of sight to retain their relevance!

Poor medication adherence is a serious problem; many people do not take their medications as prescribed, which undermines the benefits of drug treatments. Research shows that several short-term approaches to medication adherence can be effective, for example: counseling, written information, and personal phone call reminders; however, for long-term medication adherence, no simple intervention (and only some complex ones) actually lead to improvements in health outcomes.

I believe that autonomous mobile robots can improve adherence by delivering the right medication to the right person at the right time without undue caregiver or care-receiver burden. Along with the medication, the robot could also provide the means to take the medication (eg. a glass of water) and significant motivation through established psychological bonds (eg. reciprocity) that are commonly developed between social robots and their owners. UHF RFID sensing is well-matched to the challenges of robotic medication delivery. By tagging medication bottles (e.g. the FDA E-Pedigree program) and having care-receivers carry UHF RFID tags, robots can use the methods I developed to obtain the medication and then discover, approach, and administer it in a timely fashion. Several properties of UHF RFID make it particularly beneficial in this scenario. First, UHF RFID sensing is not subject to the lighting limitations of cameras; the algorithms could locate recipients equally well in both darkness or lighted rooms. Second, UHF RFID has a low false positive rate -- a crucial feature when the consequences of misidentification are dire (as with medication delivery). Coupled with short-range RFID verification and perhaps other forms of identity confirmation (eg. facial recognition), robots can ensure that the right medication is given to the right person.

Contrary to common industry perceptions, backscatter UHF RFID tags operating at 915 MHz can support very high datarates (up to 100Mbps), even under passive (battery-free) conditions. Using the same IC that was developed for the cyborg dragonfly, my colleagues and I at Duke developed battery-free tags that could simultaneously stream video and stereo audio; we also developed tags that could provide multi-channel ECG. These tags illustrate some interesting applications of custom backscatter tags that weren't previously thought possible.

One summer as a young graduate student, Matt Reynolds and I were brainstorming: "What if we used LF RFID (125 kHz, the kind used for building access badges) to make a wirelessly-powered robot swarm, where the robots are effectively power-harvesting tags? The robots could run forever -- that would be cool!" So we built it, complete with bidirectional communications. :)

We extended the system to wirelessly recharge mobile devices when placed on a table or in a specially-modified backpack. Ultimately, we imagined the power harvesting circuitry would be embedded directly in the cellphone or its case. A variation on this same technology is now commercially available in the form of Duracell's PowerMat or the Qi wireless chargers.

I designed and constructed a unique multi-antenna RFID reader (a sensor) embedded in a robot's manipulator that was specially designed to operate with ordinary UHF RFID tags in a short-range, near-field electromagnetic regime -- an atypical mode of operation that allowed our system to interact with long-range (6+ meter) UHF RFID tags at distances less than 1 meter. This unique sensor allowed a robot to identify objects in and around its gripper with near certainty, and it enabled other "pre-touch" capabilities such as servoing to grasp an object.

The future combination of long-range (far-field) and short-range (near-field) UHF RFID sensing has the potential to enable roboticists to jump-start applications by obviating or supplementing false-positive-prone visual object recognition.

An omnidirectional Mecanum base (aka. "Swedish Wheels") allows for more flexible mobile manipulation. However, slipping of the Mecanum wheels results in poor dead-reckoning estimates from wheel encoders alone, limiting the accuracy and overall utility of this type of base. Some of my grad school colleagues and I built a Mecanum base with a downward-facing camera and light ring to provide robust visual odometry estimates. We mounted the system under the robot which allowed it to operate in conditions such as large crowds or low ambient lighting. We demonstrated that the visual odometry estimates were sufficient to generate closed-loop PID (Proportional Integral Derivative) and LQR (Linear Quadratic Regulator) controllers for motion control, and we showed that this system provides high-fidelity odometry estimates for a four-wheeled omnidirectional mobile base -- one in which those same colleagues went on to use for cutting-edge mobile manipulation research (eg. door opening and manipulation in clutter).

Unstructured, human environments present great challenges and opportunities for robotic manipulation and grasping. Robots that reliably grasp household objects with unknown or uncertain properties would be especially useful.

Eschewing complex manipulators with many degrees of freedom (eg. humanoids), we developed a novel "dust pan" end effector that is capable of robustly picking up a diverse array of everyday handheld objects from the floor and other flat surfaces and delivering them to users. This straight-forward, inexpensive, non-prehensile end effector combined a compliant finger with a thin planar component with a leading wedge that slides underneath the object (much like a dustpan plus brush). We empirically validated our design through a set of 1096 trials over which we systematically varied the object location, object type, object configuration, and floor characteristics -- the end effector succeeded in 95% of trials.

We designed a system that used two small piezoelectric sensors placed on either the wrist or ankle. When a user moved their hands or feet, the sounds generated by the movement travel to the piezoelectric pickups via bone conduction. Our system transmitted the signals to a mobile device or computer, where gestures were recognized using hidden Markov models (HMMs) and used to control various apps. This was a rather rudimentary prototype, but I'm excited for the future of functional implants (not just wearables) that can create natural, omnipresent human-machine interfaces. In fact, I have a whole slew of additional ideas in this space, but insufficient bandwidth to explore them at the moment.