Visual perception is indispensable in numerous applications spanning autonomous vehicles and interdisciplinary research. Today’s visual perception algorithms are often developed under a closed-world paradigm, which assumes the data distribution and categorical labels are fixed a priori. This assumption is unrealistic in the real open world, which contains vast situations that are unpredictable and dynamic. As a result, closed-world visual perception systems appear to be brittle in the open-world. For example, equipped with such visual perception systems, an autonomous vehicle could fail to recognize a never-before-seen overturned truck and cause collision; it could fail to detect a pedestrian in a dark night and cause casualties. In this talk, I will present my solutions to recognizing unknown objects, segmenting and detecting general objects, and improving object detection using multimodal signals. If time allows, I will share my thought about open-world visual perception and learning, and sketch my future research.