This paper introduces a new measure of residential segregation based on individual-level data. We exploit complete census manuscript files to derive a measure of segregation based upon the racial similarity of next-door neighbors. Our measure allows us to analyze segregation consistently and comprehensively for all areas in the United States and allows for a richer view of the variation in segregation across time and space. We show that the fineness of our measure reveals aspects of racial sorting that cannot be captured by traditional segregation indices. Our measure can distinguish between the effects of increasing racial homogeneity of a location and the tendency to segregate within a location given a particular racial composition. Analysis of neighbor-based segregation over time establishes several new facts about segregation. First, segregation doubled nationally from 1880 to 1940. Second, contrary to previous estimates, we find that urban areas in the South were the most segregated in the country and remained so over time. Third, the dramatic increase in segregation in the twentieth century was not driven by urbanization, black migratory patterns, or white flight to suburban areas, but rather resulted from a national increase in racial sorting at the household level. The likelihood that an African American household had a non-African American neighbor declined by more than 15 percentage points (more than a 25% decrease) through the mid-twentieth century. In all areas of the United States — North and South, urban and rural — racial segregation increased dramatically.