Birds have been described as “a wing guided by an eye”, but before this European Research Council funded project we knew surprisingly little of how birds use vision in flight. Current autonomous air vehicles, or drones, use simple visual sensors to detect drift when hovering and computationally intensive algorithms for visual mapping. Neither approach closely reflects how birds use vision to guide their flight, so understanding this mathematically has the potential to unlock new capabilities for autonomous systems.
Our primary objective was to identify the guidance laws used by birds in pursuit, perching, obstacle avoidance, and gap negotiation. A guidance law is a mathematical relationship describing how sensor information is used to command steering, so using this approach to summarise a bird’s behaviour makes our results ready for implementation in air vehicles. We found that falcons intercept targets using the same guidance law as missiles, but that hawks use a different guidance law to enable collision avoidance when pursuing targets through clutter. As proof-of-concept, we demonstrated the same algorithms on small quadrotor drones. Birds use active vision, which means they direct their gaze to enhance the visual information they obtain.
Our second objective was therefore to identify how birds use head and eye movements to collect visual information during goal-directed flight behaviours. We found that birds look at objects differently according to whether they are targets or obstacles, tracking the centre of a perch but the edge of an obstacle. Many autonomous systems also use active vision, so our results from birds are ready for implementation in vehicles, and have also helped us design patterns modifying the appearance of wind turbine blades to prevent bird collisions.
Our third objective was to understand how birds transform steering commands into actions. Birds morph their wings and tail to control flight, so modelling this behaviour requires new approaches to describing flight control. We found that birds control their flight to minimize the distance from a perch at which they stall, offering a new approach to optimizing perching in drones using machine learning. We also identified a set of control inputs describing the coupled motions of the wings and tail, and created a morphing-wing flight simulator capable of simulating goal-directed morphing-wing flight in birds and air vehicles.
Delivering these objectives involved developing many new experimental techniques, including: (i) a state-of-the-art motion capture facility for indoor flight research, with robotic perches, obstacles, and targets; (ii) miniature onboard sensors for outdoor flight research, including lightweight GPS devices with 2 cm accuracy; and (iii) techniques for 3D video reconstruction in the field. In total, we collected quantitative data on over 20,000 flights.
Our lab at the John Krebs Field Station, Wytham, is home to our flying team of five Harris' Hawks – Drogon, Rhaegal, Toothless, Charmander, and Ruby - and our colony of zebra finches used for experiments on learning. We welcome visits from schools and youth groups by prior arrangement.
We opened our lab to the public in 2022 as part of the Oxford Science + Ideas Festival Animal Senses Zone. This builds on a collaboration "Digital Body: Ascent" involving the Parasol Dance Project for disabled and non-disabled young people, the Alexander Whitley Dance Company, and iF-Oxford. See the beautiful video that resulted from this collaboration here.
For a science podcast aimed at school kids, listen here to our Tumble science podcast on peregrine falcons.
This research has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (Grant Agreement No. 682501). We thank our sponsors for making this research possible.
Using dummy targets, we found that falcons use the same guidance law as missiles to intercept prey, but that hawks use a “new” guidance law suited to tail-chasing prey through clutter. We then showed that falcons modify their guidance at different stages of an attack when chasing evasive prey, using computer simulations to evolve attack and evasion strategies. This modelling showed that falcons dive after their prey because this allows them to sustain higher loads for manoeuvring. We then showed how the guidance law that hawks use could be implemented visually by measuring the motion of a target against the visual background. Finally, we showed that hawks target fixed points when attacking swarms, which allows them to identify targets without confusion because targets on a collision course appear stationary against a distant background.
We found that the same guidance laws could also be used to model obstacle avoidance, and found that pigeons and zebra finches make use of brightness cues for gap negotiation. We then extended our analyses by using video rendering to visualize what our birds saw as the avoided obstacles. This allowed us to show that birds look at the edges of obstacles, but at the centres of perches. Finally, we combined our studies of pursuit and obstacle avoidance, by putting the two behaviours in conflict. We used this to identify how hawks avoid obstacles during pursuit, using a simple modification of the guidance law that we had previously found.
We challenged our birds to land on moving perches, and found that hawks swoop up to a perch in a way that makes them better able to control their flight in the critical moments before landing. We analysed how they morphed their wings and tail during manoeuvres, identifying a set of control inputs that can be used to describe morphing-wing flight control in birds and air vehicles. These results have applications not only in the design of new morphing-wing air vehicles, but also in the use of machine learning to accomplish tasks like perching.
This project has brought the study of behavioural biomechanics into the era of big data, enabling the collection of data on a scale not possible previously. Tracking data from over 20,000 flights has enabled us to analyse the dynamics of bird flight in unprecedented detail, over many individuals, and over multiple timescales, allowing us to validate the repeatability of our findings. Unexpectedly, it has even allowed us to analyse how birds learn to improve their flight behaviour over time, offering detailed insight into the optimization of flight behaviour and opening an entire new avenue of research on learning to fly.
The execution of the project coincided with a period of extremely rapid growth in computational techniques, including deep learning which we applied together with other machine learning techniques to model aspects of our data. It also coincided with the rapid development of video rendering engines by the film and gaming industries, which we used to visualize what our birds were seeing in flight. This in turn was made possible by combining state-of-the-art motion capture technologies measuring the gaze direction of the birds with cutting-edge 3D reconstruction tools modelling the interior of the lab. This has again inspired another new avenue of planned research, using the same tools to visualize how birds see the built environment.
The main application of our research that we foresaw at the start of the project was in designing guidance and control algorithms for autonomous vehicles. Whilst our results from birds have indeed been used to design new guidance and control algorithms as envisioned, an unforeseen application of our experiments with moving obstacle has been to inspire new methods of avoiding bird collisions with wind turbine blades. This involves marking turbine blades in ways that are specifically inspired by what we learnt on this project about how birds direct their gaze and avoid obstacles in flight. We expect to begin testing the first blade patterns that we have designed later this year.
Besides its applications in inspiring new guidance and control algorithms for autonomous vehicles, and their potential to mitigate bird collisions with wind turbines, our research has shed new light on how birds use vision to guide and control their flight. This has proven a remarkably tractable problem to study, and the results provide some of the closest quantitative models of complex natural behaviours achieved to date.
Brighton, C.H., Kempton, J.A., France, L.A., KleinHeerenbrink, M., Miñano, S., Taylor, G.K. (2023). Obstacle avoidance in aerial pursuit. bioRxiv [preprint] https://doi.org/10.1101/2023.01.23.525170.
Kempton, J.A., Brighton, C.H., France, L.A., KleinHeerenbrink, M., Miñano, S., Shelton, J., Taylor, G.K. (2023). Visual versus visual-inertial guidance in hawks pursuing terrestrial targets. J. R. Soc. Interface 20, 20230071. https://doi.org/10.1098/rsif.2023.0071.
Borsier, E., Taylor, G.K. (2023). Zebra finches use brightness cues for gap selection in flight. bioRxiv [preprint] https://doi.org/10.1101/2023.02.21.529441.
Harvey, C., de Croon, G., Taylor, G.K., Bomphrey, R.J. (2023). Lessons from natural flight for aviation: then, now, and tomorrow. J. Exp. Biol. 226, jeb245409. https://doi.org/10.1242/jeb.245409.
Miñano, S., Golodetz, S., Cavallari, T., Taylor, G.K. (2023). Through hawks’ eyes: synthetically reconstructing the visual field of a bird in flight. Int. J. Comput. Vis. https://doi.org/10.1007/s11263-022-01733-2.
Peréz-Campanero Antolín, N., Taylor, G.K. (2023). Gap selection and steering during obstacle avoidance in pigeons. J. Exp. Biol. 226, jeb244215. https://doi.org/10.1242/jeb.244215.
Leroy, A., Taylor, G.K. (2022). FlyView: a bio-informed optical flow truth dataset for visual navigation using panoramic stereo vision. In: 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Datasets and Benchmarks Track.
Mohamed, A., Taylor, G.K., Watkins, S., Windsor, S.P. (2022). Opportunistic soaring by birds suggests new opportunities for atmospheric energy harvesting by flying robots. J. R. Soc. Interface https://doi.org/10.1098/rsif.2022.0671.
KleinHeerenbrink, M., France, L.A., Brighton, C.H., Taylor, G.K. (2022). Optimization of avian perching manoeuvres. Nature 607, 91-96. https://doi.org/10.1038/s41586-022-04861-4.
Brighton, C.H., Kloepper, L.N., Harding, C.D., Larkman, L., McGowan, K., Zusi, L., Taylor, G.K. (2022). Raptors avoid the confusion effect by targeting fixed points in dense aerial prey aggregations. Nat. Commun. 13, 4778. https://doi.org/10.1038/s41467-022-32354-5.
Kempton, J.A., Wynn, J., Bond, S., Evry, J., Fayet, A.L., Gillies, N., Guilford, T., Kavelaars, M., Juarez-Martinez, I., Padget, O., Rutz, C., Shoji, A., Syposz, M., Taylor, G.K. (2022). Optimization of dynamic soaring in a flap-gliding seabird affects its large-scale distribution at sea. Sci. Adv. 8, eabo0200. https://doi.org/10.1126/sciadv.abo0200.
Minano, S., Taylor, G.K. (2021). Through hawks' eyes: reconstructing a bird's visual field in flight to study gaze strategy and attention during perching and obstacle avoidance. In: Workshop on Computer Vision for Animal Behavior Tracking and Modeling, CVPR Virtual, 19-25 June 2021. bioRxiv 2021.06.16.446415. https://doi.org/10.1101/2021.08.28.458019.
Brighton, C.H., Chapman, K.E., Fox, N.C., Taylor, G.K. (2021). Attack behaviour in naïve Gyrfalcons is modelled by the same guidance law as in Peregrines, but at a lower guidance gain. J. Exp. Biol. 224(5): jeb238493. https://doi.org/10.1242/jeb.238493.
Brighton, C.H., Zusi, L., McGowan, K., Kinniry, M., Kloepper, L.N., Taylor, G.K. (2021). Birds versus bats: attack strategies of bat-hunting hawks, and the dilution effect of swarming. Behav. Ecol. 32(3): 464-476. https://doi.org/10.1093/beheco/araa145.
Hein, A.M., Altshuler, D.L., Cade, D.E., Liao, J.C., Martin, B.T., Taylor, G.K. (2020). An algorithmic approach to natural behavior. Curr. Biol. 30(11): PR663-R675. https://doi.org/10.1016/j.cub.2020.04.018.
Brighton, C.H., Taylor, G.K. (2019). Hawks steer attacks using a guidance system tuned for close pursuit of erratically maneuvering targets. Nat. Commun. 10, 2462 https://doi.org/10.1038/s41467-019-10454-z.
Taylor, L.A., Taylor, G.K., Lambert, B., Walker, J.A., Biro, D., Portugal, S.J. (2019). Birds invest wingbeats to keep a steady head and reap the ultimate benefits of flying together. PLoS Biol. 17(6), e3000299 https://doi.org/10.1371/journal.pbio.3000299.
Mills, R., Taylor, G.K., Hemelrijk, C.K. (2019). Sexual size dimorphism, prey morphology and catch success in relation to flight mechanics in the peregrine falcon: a simulation study. J. Avian Biol. 50(3), e01979 https://doi.org/10.1111/jav.01979.
Mills, R., Hildenbrandt, H., Taylor, G.K., Hemelrijk, C.K. (2018). Physics-based simulations of aerial attacks by peregrine falcons reveal that stooping at high speed maximizes catch success against agile prey. PLoS Comput. Biol. 14(4), e1006044. https://doi.org/10.1371/journal.pcbi.1006044.
Brighton, C.H., Thomas, A.L.R., Taylor, G.K. (2017). Terminal attack trajectories of peregrine falcons are described by the proportional navigation guidance law of missiles. Proc. Natl. Acad. Sci. USA 114(51) 13495-13500. https://doi.org/10.1073/pnas.1714532114.