News
-
2023/03/23
We are pleased to announce a new HerdHover publication - a Research Methods Guide describing our approach for drone- and image-based observation of animal groups in the wild, published in the Journal of Animal Ecology!
In this paper we introduce our pipeline for extracting the movement and posture of many individual animals simultaneously, along with information on the surrounding environment. The result is high-res trajectories and pose estimates of entire animal groups in real-world geographic coordinates that can be explored in the context of 3D landscape models.
This work was led by Ben Koger and Blair Costelloe, along with collaborators Jake Graving, Iain Couzin, Jeff Kerby and Adwait Deshpande.
Please check out the publication - it’s open access, and all code and data are also publicly available.
For more information, see the press release here, and Ben and Blair’s Twitter threads (here and here).
To cite the paper in your own research:
Koger, B., Deshpande, A., Kerby, J.T., Graving, J.M., Costelloe, B.R. and Couzin, I.D. 2023. Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision. Journal of Animal Ecology.
-
2022/11/03
Herd hover team member Adwait Deshpande recently finished fieldwork in Mozambique, studying wild African buffalo herds using drone-based videos. Following is a short blog about his experience.
“As a part of my postdoc fellowship, I spent 45 days in a private game reserve in Mozambique with the aim of collecting aerial videos of wild African buffaloes to investigate the collective movements of Herds. For me, the expedition was exciting at the same time, challenging.
We had a small team, Luca (field assistant) and I. Although both of us have experience in Southern African bushveld, it was not easy to find buffalo herds in the Bush as we were expecting. Thus, we spent almost all of our field days looking for buffaloes, from Sunrise until sunset, in our bakkie (4*4 Land cruiser). The open loading bay at the back of the vehicle was set up to quickly deploy and land two Mavic 2 drones.
After an initial struggle, we got some idea about buffalo activity and strategies to find them depending on weather and time of the day, which included random searching using vehicles and drones and waiting at the waterhole. Nonetheless, while constantly looking out for aggressive elephants (we were mock charged a few times), we did more than 250 drone flights and got data on a variety of herd types and which captured some exciting behaviors from the air.
The next task is to process the data using the Herd hover pipeline. It would be tricky as the all the areas where we filmed were not very open as we thought they would be. However, it would be possible with the help of the entire Herd hover team.”
Watch this space for more updates!
-
2022/10/15
This September, Herd Hover team member Ben Koger successfully defended his Ph.D. at the University of Konstanz and started a new position at the University of Washington as a Washington Research Foundation postdoctoral scholar. After a busy few years working to design tools that combine imaging and deep learning techniques as part of the Herd Hover project and the wider Max Planck community, Dr. Ben will continue to work on new ways to study collective animal behavior in natural landscapes. He’ll work with Professor Andrew Berdahl at UW’s School of Aquatic and Fishery Sciences to understand how automated imaging techniques, similar to those developed for the Herd Hover project, can be used to study and manage Pacific salmon. In the clear waters of Alaska’s Bristol Bay watershed, the large and bright-red sockeye salmon can be observed from overhead cameras as they migrate up to the nesting ponds in which they reproduce. Ben plans to investigate not only the number of fish that return to each specific pond, but also how fish are able to successfully make this complex journey through novel environments and surrounded by predators and competing conspecifics. Beyond a better understanding of salmon ecology, these fish in Bristol Bay are part of the largest commercial sockeye fishery in the world and so are economically and culturally important well beyond their remote nesting ponds. To familiarize himself with this system, Ben spent four weeks in August at the Alaska Salmon Program research station on Porcupine Island in Lake Iliamna. There, he worked on piloting new techniques and learned about existing research and on-going data collection.
The Herd Hover team wishes Ben the best of luck in his new position, and looks forward to continued collaborations!
-
2022/02/09
New Publication in Nature Communications
Herd Hover team members Blair Costelloe and Iain Couzin have a new open access paper out today in Nature Communications. The piece, “Perspectives in machine learning for wildlife conservation”, reviews various types of sensors that generate big data in wildlife ecology and conservation, and recent efforts to apply machine learning to this data to generate actionable insights. Looking forward, the authors call for close collaboration between computer scientists and ecologists to combine novel machine learning approaches with ecological domain knowledge to tackle big challenges in animal ecology and conservation.
Blair is a lead author on the paper along with Devis Tuia (EPFL), Benjamin Kellenberger (EPFL) and Sara Beery (Caltech). Mike Costelloe contributed to the design and execution of the figures.
-
2019/10/01
The first publication from the Herd Hover project is out today in eLife! The paper introduces DeepPoseKit, a new, easy-to-use, open-source software toolkit for estimating animal posture from videos or images. The software was developed by authors Jake Graving and Daniel Chae, in collaboration with Hemail Naik, Liang Li, Benjamin Koger, Blair Costelloe and Iain Couzin.
In addition to describing the software and its performance, the article provides an accessible overview of pose estimation technology and its applications to a wide range of fields, from neuroscience to ecology. This article is a great place to start if you’re new to pose estimation!
At Herd Hover, we are using DeepPoseKit to track the body posture of zebras and other ungulates in our drone footage. This allows us to determine the orientation of each animal’s body, the position of its head, and its behavioral state many times per second. Such detailed data are impossible to collect on entire groups of freely-moving wild animals using conventional observation methods, so posture tracking is opening some exciting doors for field studies of collective behavior. We believe that this combination of consumer drone technology and posture estimation also has the potential to be a valuable tool in wildlife management, health, and conservation, allowing practitioners to efficiently collect and objectively assess behavioral data with minimal disturbance to animals.
Be sure to check out the paper, as well as the official press release. The code for DeepPoseKit is available on GitHub - give it a try!
-
2019/05/23
Project featured in uni'kon magazine
The Herd Hover project is featured in the most recent issue of uni’kon, the University of Konstanz’s magazine. The issue (in German) is available fo download [here] (https://www.uni-konstanz.de/universitaet/aktuelles-und-medien/uni-kon-broschueren-und-jahresberichte/uni-kon/) and the English version of the article is available [here] (https://www.campus.uni-konstanz.de/en/science/a-new-era-in-the-behaviou-shy-ral-sciences).
The article, written by Carla Avolio, describes current research in the Department of Collective Behavior that is drawing on artificial intelligence to revolutionize the study of animal behavior. In addition to our project, you can also learn about exciting work being done by other group members in the labs, field and Imaging Barn.
With the recent establishment of the Centre for the Advanced Study of Collective Behaviour at the University and the founding of the new Max Planck Institute of Animal Behavior, Konstanz is rapidly emerging as an international hub for research in this field. We are thrilled to be based here, and excited to see what the future brings!
If you’d like to keep up with the latest developments in the Department of Collective Behaviour, be sure to follow [@CollectiveBehav] (https://twitter.com/CollectiveBehav) on Twitter!
-
2019/05/02
We are pleased to announce that the first research paper using Herd Hover data is now available as a preprint on bioArxiv! The paper, titled ‘Fast and robust animal pose estimation,’ presents DeepPoseKit, an open-source software toolkit that allows researchers to extract estimates of animal posture from images or videos. DeepPoseKit is inspired by previous pose-estimation methods, but offers improvements in speed and accuracy, while also emphasizing usability. This work is led by Herd Hover team member Jake Graving and uses our Grevy’s zebra videos as one of the test data sets.
It is particularly exciting that we can now estimate animal postures from drone images. We hope this will open the door to more highly-quantitative field studies of animal behavior and inspire applications for wildlife conservation. We are currently using DeepPoseKit to extract fine-scale behavioral measurements from our drone footage. This allows us to go beyond simply tracking the animals’ positions to really get into the details of individual behaviors and social interactions within our herds.
Jake’s announcement of the preprint on Twitter gained a lot of attention that has already led to some refinements of the manuscript. We are hopeful that a peer-reviewed and published version will be available before long. In the meantime, learn more about the work by reading the preprint. More videos demonstrating the technique are also available on YouTube. The code will be available soon at https://github.com/jgraving/deepposekit.
Preprint citation: Graving JM, Chae D, Naik H, Li L, Koger B, Costelloe BR, Couzin ID. Fast and robust animal pose estimation. bioRxiv 620245. DOI: https://doi.org/10.1101/620245.
-
2019/04/08
Last month Blair and Ben attended the Gordon Research Conference for Animal Movement and the associated early career symposium in Barga, Italy. They both presented posters on the latest developments in the Herd Hover project. Check out the newest poster on Researchgate.
The conference was a fun week of science and Italian food, and full of chances to meet new colleagues and catch up with friends. The Herd Hover project was well-received, and it was cool to see the latest work being done on animal movement. After a quick holiday to Florence, Ben and Blair returned to Konstanz with new connections and new ideas.
-
2019/02/06
The start of 2019 has been busy, with one Herd Hover masters student, Sven Lauke, entering the final stages of his thesis writing, and another, Felicitas Oehler, just getting started! Below, Felicitas introduces her plan for her thesis project, for which she will investigate movement and association patterns within zebra herds, and particularly how these change when the group responds to a disturbance (humans approaching on foot). Check back here for future updates from Felicitas and Sven!
“Sometimes great opportunities just pop up in your life when you are at the right place at the right time. That happened to me when I was looking for a job as a research assistant at the university, and saw Blair’s announcement for the field assistant position at the Mpala Research Center!
With the field assistant position in Blair’s ungulate project I got the amazing opportunity to accompany and assist Blair, Ben and Mike in the field, which means in the savannah in Kenya! Furthermore taking that position meant to follow up the work with a master thesis project in the Couzin Lab in the Herd Hover project with Blair and Ben. And that’s where I am at the moment. One month passed already since I’ve started with my master thesis.
In the thesis I want to analyze different aspects of social and collective anti-predation-behavior of the two zebra species, which have two different social organization systems. I want to focus on subgroup formation in a herd and the changes of group composition before, during and after the scare. Furthermore I would like to take the impact of the vegetation on their movement away from the scare impulse into account. Additionally I could imagine having a closer look at the line formation during and after flight and the underlying coordination, order and pattern of this line formation in zebras.
So far I’m diving in into data processing, annotations and track corrections of the existing drone footage, background reading and learning to analyze data with Python.
Having all these enriching remembers of the animal wildlife in Kenya and the great time working together with the group in my mind I’m very motivated to find out some nice and important facts about zebras and their behavior! Then this knowledge can help us to improve the understanding of the anti-predation behavior of ungulate species and the cooperation and interaction between individuals within herds.”
-
2019/01/24
In December Ben Koger visited the Intelligent Control Lab at the Peking University School of Engineering. The lab, run by Professor Xie Guangming, studies control theory for complex dynamical systems and uses these insights to develop real world robotic applications. The lab uses natural systems as inspiration for much of their work. In particular, they have developed very realistic swimming robotic fish that they use to study the potential energy benefits of schooling. Further information about how real groups in the wild respond to complex environments is very useful for their work.
Ben presented the techniques developed by Herd Hover to collect data about diverse ungulate groups in the wild. One of the challenges in designing real world robotic systems is learning how to efficiently observe the environment and then using that information to make fast correct decisions to solve a task. The ungulate groups we study have evolved to do just that whether it be threat detection and evasion, resource search strategies, or distributed decision making. Understanding how animal groups solve these challenges is good inspiration for roboticists. For us, understanding how an engineered agent might optimally solve these problems is very useful as a starting point to think about how animals might really be behaving.
Thanks so much to the Professor Xie and Zhang Tianhao for showing Ben around.
-
2019/01/17
Blair visited Dartmouth College earlier this month and gave a departmental seminar. Her update below:
“It was great to visit Hanover and get to know folks in the Ecology, Evolution, Ecosystems and Society program. Although it was early in the semester, I got to have some really interesting chats with a number of students and faculty. I was also invited to give a brief guest lecture in Jeff Kerby’s undergrad course on animal movement and landscape ecology, which led to a good class discussion. It was an honor to open the department’s spring seminar series, where I got to share Herd Hover’s methodologies and demonstrate how these will be used to answer questions relating to collective detection and movement behavior in ungulates.
I’m grateful to everyone who made time to attend the talk or meet with me one-on-one, and to Carey Nadell for hosting my visit!”
-
2018/12/20
University of Konstanz VTK course
Since early November, three University of Konstanz masters students have been working with the Herd Hover project as part of their intensive course on collective behaviour. Liliana, Kathy and Nadim have gained experience with all stages of data processing and analysis, including annotating drone footage, reviewing and correcting pre-processed tracks, and planning and executing their own analyses of our zebra data.
Earlier this month they gave a short presentation on their project, in which they classified distinct herd “shapes”, used a Markov Chain analysis to quantify transition probabilities between states, and investigated individual-level behaviors that drive these group-level phenomena. We’ve greatly enjoyed working with them and look forward to reading their final reports at the end of January!
-
2018/09/17
Blair Costelloe attended two conferences this summer, where she presented our tracking and posture estimation tools. In early August she gave a talk at the Animal Behavior Society annual meeting at the University of Wisconsin-Milwaukee. Slides from the talk, entitled “Collective detection in ungulate groups: novel tools for field studies of collective behavior,” will be available online soon.
In mid-August, Blair was in Minneapolis, Minnesota at the biennial meeting of the International Society for Behavioral Ecology where she presented a poster titled “Collective detection in free-ranging African ungulates: generating dynamic behavioral datasets with drones and computer vision”. You can view the poster on Researchgate.
Both the talk and the poster were well-received, and Blair got to speak with a number of other researchers interested in using drones and computer vision in their own projects. We are now putting these tools to work to explore the dynamics of collective behavior in African ungulates!
-
2018/06/22
The project site is live! Stay tuned for updates.