Contour completion plays an important role in visual perception, where the goal is to group fragmented low-level edge elements into perceptually coherent and salient contours. Most existing methods for contour completion have focused on pixelwise detection accuracy. In contrast, fewer methods have addressed the global contour closure effect, despite psychological evidences for its importance. This paper proposes a purely contour-based higher order CRF model to achieve contour closure, through local connectedness approximation. This leads to a simplified problem structure, where our higher order inference problem can be transformed into an integer linear program and be solved efficiently. Compared with the methods based on the same bottom-up edge detector, our method achieves a superior contour grouping ability (measured by Rand index), a comparable precision-recall performance, and more visually pleasing results. Our results suggest that contour closure can be effectively achieved in contour domain, in contrast to a popular view that segmentation is essential for this purpose.