Limited by aircraft flight altitude and camera parameters, it is necessary to obtain wide-angle panoramas quickly by stitching aerial images, which is helpful in rapid disaster investigation, recovery after earthquakes, and battlefield reconnaissance. However, most existing stitching algorithms do not meet practical real-time, robustness, and accuracy requirements simultaneously, especially in the case of a long-distance multi-strip flight. In this paper, we propose a novel image-only real-time UAV image mosaic framework for long-distance multi-strip flights, which does not require any auxiliary information such as GPS or GCPs...Read More
In this paper, we investigate the problem of aligning multiple deployed camera into one united coordinate system for cross-camera information sharing and intercommunication. However, the difficulty is greatly increased when faced with large-scale scene under chaotic camera deployment. To address this problem, we propose a UAV-assisted wide area multi-camera space alignment approach based on spatiotemporal feature map. It employs the great global perception of Unmanned Aerial Vehicles (UAVs) to meet the challenge from wide-range environment. Concretely, we first present a novel spatiotemporal feature map construction approach to ...Read More
Accurate localization of moving objects in dynamic environment for small UAV platform using global averagingXiuchuan Xie， Tao Yang， Yanning Zhang， Bang Liang and Linfeng Liu.
Abstract: In recent years, small unmanned aerial vehicles (UAVs) have rapidly developed and are widely used in disaster relief, traffic monitoring and military surveillance. In order to better perform these tasks, it is necessary to improve the environmental perception ability of UAV in dynamic environment including static and dynamic perception ability. Specifically, 3D reconstruction for static scene and localization for moving object are both required. Simultaneous Localization And Mapping (SLAM) technology has made great progress in static scene structure reconstruction and UAV self motion estimation. However, accurate real-time localization of ...
Synthetic aperture imaging, which has been proved to be an effective approach for occluded object imaging, is one of the challenging problems in the field of computational imaging. Currently most of the related researches focus on fixed synthetic aperture which usually accompanies with mixed observation angle and foreground de-focus blur. But the existence of them is frequently a source of perspective effect decrease and occluded object imaging quality degradation. In order to solve this problem, we propose a novel data-driven variable synthetic aperture imaging based on semantic feedback...Read More
In recent years, UAV technology has developed rapidly. Due to the mobility, low cost, and variable monitoring altitude of UAVs, multiple-object detection and tracking in aerial videos has become a research hotspot in the field of computer vision. However, due to camera motion, small target size, target adhesion, and unpredictable target motion, it is still difficult to detect and track targets of interest in aerial videos, especially in the case of a low frame rate where the target position changes too much. In this paper, we propose a multiple-object-tracking algorithm based on dense-trajectory voting in aerial videos...Read More
With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV...Read More
Tracking multiple people in crowds is a fundamental and essential task in the multimedia field. It is often hindered by difficulties such as dynamic occlusion between objects, cluttered background and abrupt illumination changes. To respond to this need, in this paper, we combine deep and depth to build a stereo tracking system for crowds. The core of the system is the fusion of the advantages of deep learning and depth information, which is exploited to achieve object segmentation and improve the multiobject tracking performance in severe occlusion...Read More
In recent years, unmanned aerial vehicles (UAVs) have rapidly developed, but the illegal use of UAVs by civilians has resulted in disorder and security risks and has increasingly triggered community concern and worry. Therefore, the monitoring and recycling of UAVs in key regions is of great significance. This paper presents a novel panoramic UAV surveillance and autonomous recycling system that is based on an unique structure-free fisheye camera array and has the capability of real-time UAV detection...Read More
Hierarchical Clustering-Aligning Framework Based Fast Large-Scale 3D Reconstruction Using Aerial ImageryXiuchuan Xie, Tao Yang, Dongdong Li, Zhi Li, Yanning Zhang.
With extensive applications of Unmanned Aircraft Vehicle (UAV) in the field of remote sensing, 3D reconstruction using aerial images has been a vibrant area of research. However, fast large-scale 3D reconstruction is a challenging task. For aerial image datasets, large scale means that the number and resolution of images are enormous, which brings significant computational cost to the 3D reconstruction, especially in the process of Structure from Motion (SfM). In this paper, for fast large-scale SfM, we propose a clustering-aligning framework that hierarchically merges partial structures...Read More
Moving target detection plays a primary and pivotal role in avionics visual analysis, which aims to completely and accurately detect moving objects from complex backgrounds. However, due to the relatively small sizes of targets in aerial video, many deep networks that achieve success in normal size object detection are usually accompanied by a high rate of false alarms and missed detections. To address this problem, we propose a novel visual detail augmented mapping approach for small aerial target detection. Concretely, we first present a multi-cue foreground segmentation algorithm including motion and grayscale information to extract potential regions...Read More
With the rapid development of Unmanned Aerial Vehicle (UAV) systems, the autonomous landing of a UAV on a moving Unmanned Ground Vehicle (UGV) has received extensive attention as a key technology. At present, this technology is confronted with such problems as operating in GPS-denied environments, a low accuracy of target location, the poor precision of the relative motion estimation, delayed control responses, slow processing speeds, and poor stability. To address these issues, we present a hybrid camera array-based autonomous landing UAV that can land on a moving UGV in a GPS-denied environment...Read More
Recognizing human actions from varied views is challenging due to huge appearance variations in different views. The key to this problem is to learn discriminant view-invariant representations generalizing well across views. In this paper, we address this problem by learning view-invariant representations hierarchically using a novel method, referred to as Joint Sparse Representation and Distribution Adaptation (JSRDA).. . .Read More
Multi-Object Tracking (MOT) in airborne videos is a challenging problem due to the uncertain airborne vehicle motion, vibrations of the mounted camera, unreliable detections, changes of size, appearance and motion of the moving objects and occlusions caused by the interaction between moving and static objects in the scene. To deal with these problems. . .Read More
With the popularization and wide application of drones in military and civilian fields, the safety of drones must be considered. At present, the failure and drop rates of drones are still much higher than those of manned aircraft. Therefore, it is imperative to improve the research on the safe landing and recovery of drones. However, most drone navigation methods rely on global positioning system (GPS) signals. . .Read More
An infrared sensor is a commonly used imaging device. Unmanned aerial vehicles, the most promising moving platform, each play a vital role in their own field, respectively. However, the two devices are seldom combined in automatic ground vehicle detection tasks. Therefore, how to make full use of them—especially in ground vehicle detection based on aerial imagery–has aroused wide academic concern. . .Read More
Infrared human action recognition has many advantages, i.e., it is insensitive to illumination change, appearance variability, and shadows. Existing methods for infrared action recognition are either based on spatial or local temporal information, however, the global temporal information, which can better describe the movements of body parts across the whole video. . .Read More
As the two most commonly used imaging devices, infrared sensor and visible sensor play a vital and essential role in the field of heterogeneous image matching. Therefore, visible-infrared image matching which aims to search images across them has important application and theoretical significance. However, due to the vastly different imaging principles, how to accurately match between visible and infrared image remains a challenge. . .Read More
This work expands upon state-of-the-art multiscale tracking based on compressive sensing (CT) by increasing the overall tracking accuracy. A pixelwise classification stage is incorporated in the CT-based tracker to obtain a relatively stable appearance model, by distinguishing object pixels from the background. .Read More
Pedestrian detection is among the most frequently-used preprocessing tasks in many surveillance application fields, from low-level people counting to high-level scene understanding. Even though many approaches perform well in the daytime with sufficient illumination, pedestrian detection at night is still a critical and challenging problem for video surveillance systems. .Read More
Cross-domain image matching, which investigates the problem of searching images across different visual domains such as photo, sketch or painting, has attracted intensive attention in computer vision due to its widespread application. Unlike intra-domain matching, cross-domain images appear quite different in various characteristics.Read More
Robust extraction of consensus sets from noisy data is a fundamental problem in robot vision. Existing multimodel estimation algorithms have shown success on large consensus sets estimations. One remaining challenge is to extract small consensus sets in cluttered multimodel data set. In this article, we present an effective multimodel extraction method to solve this challenge.Read More
Occlusion poses as a critical challenge in computer vision for a long time. Recently, the technique of synthetic aperture photography using a camera array has been regarded as a promising way to address the problem of occluded object imaging. Unfortunately, the expensive cost of a standard camera array system with the required calibration procedure still limits the widespread popularity of this technique.Read More
Vision-based mobile robot navigation is a vibrant area of research with numerous algorithms having been developed, the vast majority of which either belong to the scene-oriented simultaneous localization and mapping (SLAM) or fall into the category of robot-oriented lane-detection/trajectory tracking. These methods suffer from high computational cost and require stringent labelling and calibration efforts.Read More
Real-time and high performance occluded object imaging is a big challenge to many computer vision applications. In recent years, camera array synthetic aperture theory proves to be a potential powerful way to solve this problem.Read More
This paper proposes a novel infrared camera array guidance system with capability to track and provide real time position and speed of a fixed-wing Unmanned air vehicle (UAV) during a landing process. The system mainly include three novel parts: (1) Infrared camera array and near infrared laser lamp based cooperative long range optical imaging module;Read More
Fast compressive tracking utilizes a very sparse measurement matrix to capture the appearance model of targets. Such model performs well when the tracked targets are well defined. However, when the targets are low-grain, low-resolution, or small, a single fixed size sparse measurement matrix is not sufficient enough to preserve the image structure of the target.Read More
Vehicle surveillance of a wide area allows us to learn much about the daily activities and traffic information. With the rapid development of
Diverse scene stitching is a challenging task in aerial video surveillance. This paper presents a hybrid stitching method based on the observation that aerial videos captured in real surveillance settings are neither totally ordered nor completely unordered.Read More
Heavy occlusions in cluttered scenes impose significant challenges to many computer vision applications. Recent light field imaging systems provide new see-through capabilities through synthetic aperture imaging (SAI) to overcome the occlusion problem.Read More
With the wide development of UAV (Unmanned Aerial Vehicle) technology, moving target detection for aerial video has become a popular research topic in the computer field. Most of the existing methods are under the registration-detection framework and can only deal with simple background scenes.Read More
The highly efficient and robust stitching of aerial video captured by unmanned aerial vehicles (UAVs) is a challenging problem in the field of robot vision. Existing commercial image stitching systems have seen success with offline stitching tasks, but they cannot guarantee high-speed performance when dealing with online aerial video sequences.Read More
Automatically focusing and seeing occluded moving object in cluttered and complex scene is a significant challenging task for many computer vision applications. In this paper, we present a novel synthetic aperture imaging approach to solve this problem.Read More
Robust detection and tracking of multiple people in cluttered and crowded scenes with severe occlusion is a significant challenge for many computer vision applications. In this paper, we present a novel hybrid synthetic aperture imaging model to solve this problem. The main characteristics of this approach are as follows.Read More
Seeing an object in a cluttered scene with severe occlusion is a significantly challenging task for many computer vision applications. Although camera array synthetic aperture imaging has proven to be an effective way for occluded object imaging, its imaging quality is often significantly decreased by the shadows of the foreground occluder.Read More
Abstract Autofocus is a fundamental and key problem for modern imaging sensor design. Although this problem has been well studied in single camera literature, unfortunately, little research has been done on large-scale camera arrays.Read More
A Novel Multi-Object Detection Method in Complex Scene Using Synthetic Aperture Imaging and vision computingZhao Pei, Yanning Zhang, Tao Yang, X. Zhang, and Y.H. Yang.
This paper proposes a novel multi-object detection method using multiple cameras. Unlike conventional multi-camera object detection methods, our method detects multiple objects using a linear camera array. The array can stream different views of the environment and can be easily reconfigured for a scene compared with the overhead surround configuration.Read More
Automatic counting of passengers is very important for both business and security applications. We present a single-camera-based vision system that is able to count passengers in a highly crowded situation at the entrance of a traffic bus.Read More
The traditional target tracking algorithm usually trains the template with detected samples and updates the template at a fixed frequency. This close-loop mechanism lacks feedback and often makes it impossible to track targets robustly when target appearance or illumination changes.Read More
Real time and robust image registration is the premise and key technology of aerial video stabilization, panorama stitching and ground moving target detection and tracking. This paper presents a novel scene complexity and invariant feature based aerial video registration algorith.Read More
A scene model and statistic learning based method for pedestrian detection in complicated real-world scenes is proposed. A unique characteristic of the algorithm is its ability to train a special cascade classifier dynamically for each individual scene.Read More
This paper presents a novel real-time multiple object tracking algorithm, which contains three parts: region correlation based foreground segmentation, merging-splitting based data association and greedy searching based occluded object localization.Read More
An automatic visual–thermal image sequence registration method based on co-motion was proposed. Different from other methods, co-motion (concurrent motions) statistics feature was adopted to regist heterogeneous image sequences.Read More
Foreground detection is an important research problem in visual surveillance. In this paper, we present a novel multiple layer background model to detect and classify foreground into three classes, moving object, static object and ghost. The background is divided into two layers, reference background and dynamic background.Read More
Multiple pedestrian tracking is regarded as a challenging work due to difficulties of occlusion, abrupt motion and changes in appearance. In this paper, we propose a multi-layer graph based data association framework to address occlusion problem. Our framework is hierarchical with three association layers and each layer has its corresponding association method.Read More
Hyperspectral image (HSI) classification deals with the problem of pixel-wise spectrum labelling. Traditional HSI classification algorithms focus on two major stages: feature extraction and classifier design. Though studied for decades, HSI classification hasn't been perfectly solved. One of the main reasons relies on the fact that features extracted by embedding methods can hardly match an ad hoc classifier.Read More
The compressive sensing trackers, which utilize a very sparse measurement matrix to capture the targets' appearance model, perform well when the tracked targets are well defined. However, such trackers often run into drifting problems due to the fact that the tracking result is a bounding box which also includes background information, especially in the case of occlusion and low contrast situations.Read More
Heavy occlusions in cluttered scenes impose significant challenges to many computer vision applications. Recent light field imaging systems provide new see-through capabilities through synthetic aperture imaging (SAI) to overcome the occlusion problem.
Autonomous Near Ground Quadrone Navigation with Uncalibrated Spherical Images Using Convolutional Neural NetworksLingyan Ran, Yanning Zhang, Tao Yang, Ting Chen.
This paper focuses on the use of spherical cameras for autonomous quadrone navigation tasks. Previous works of literature for navigation mainly lie in two categories: scene-oriented simultaneous localization and mapping and robot-oriented heading fields lane detection and trajectory tracking. Those methods face the challenges of either high computation cost or heavy labelling and calibration requirements.Read More
Continuously tracking and see-through occlusion based on a new hybrid synthetic aperture imaging modelTaoYang, YanningZhang, Xiaomin Tong, XiaoqiangZhang, RuiYu.
Robust detection and tracking of multiple people in cluttered and crowded scenes with severe occlusion is a significant challenging task for many computer vision applications. In this paper, we present a novel hybrid synthetic aperture imaging model to solve this problem.
This work presents a real-time system for multiple object tracking in dynamic scenes. A unique characteristic of the system is its ability to cope with long-duration and complete occlusion without a prior knowledge about the shape or motion of objects. The system produces good segment and tracking results at a frame rate of 15-20 fps for image size of 320x240...
In this paper we describe the analysis component of an indoor, real-time, multi-camera surveillance system. The analysis includes: (1) a novel feature-level foreground segmentation method which achieves efficient and reliable segmentation results even under complex condition...
DOTS (Dynamic Object Tracking System) is an indoor, real-time, multi-camera surveillance system, deployed in a real office setting. DOTS combines video analysis and user interface components to enable security personnel to effectively monitor views of interest and to perform tasks such as tracking a person.
FlyingSword: A Real-time Motion Video Registration, Stabilization, Mosaicing and Moving Object Tracking SystemTaoYang, Yanning Zhang.
Developing a fully automatic, efficient and robust video content analysis system is a subject of great scientific and commercial interest. Intelligent video content analysis with a static camera has been well researched over the past decade, and many excellent algorithms and systems have been proposed in the literature.
A Novel Multi-Planar Homography Constraint Algorithm for Robust Multi-People Location with Severe OcclusionXiaomin Tong, TaoYang, Runping Xi, Dapei Shao, Xiuwei Zhang.
Multi-view approach has been proposed to solve occlusion and lack of visibility in crowded scenes. However, the problem is that too much redundancy information might bring about false alarm. Although researchers have done many efforts on how to use the multi-view information to track people accurately, it is particularly hard to wipe off the false alarm.
This work presents an active learning based method for pedestrian detection in complicated real-world scenes. Through analyzing the distribution of all positive and negative samples under every possible feature, a highly efficient weak classifier selection method is presented. Moreover, a novel boosting architecture is given to get satisfied False Positive Rate (FPR) and False Negative Rate (FNR) with few weak classifiers.