To simplify the problem, I start from clustering patterns of facade parts of aerial imagery.
Directly clustering densely sampled patches from aerial imagery will give edge and corner patterns rather than give iconic patches.
The result arouses a basic question: "What is iconic pattern?" If edge and corner are the iconic patterns then clustering densely sampled patches yield a good result. However, what we want is pattern like windows, roof tops, etc, which is not just corner and edge.
Therefore, I decide to find symmetric patterns which are usually the characteristics of some semantics tags, such as windows, and roof tops. The symmetric patterns will then go through k-means clustering for iconic patches.
Here is the aerial image I use:
Work on RGB or Lab space for symmetric detection give a poor signal.
We turn the raw image to sift image in SIFT flow paper:
Then do the simple symmetry detection, just SSD of a patch and its flip version:
Black strips, which mean high symmetry parts, show the vertical axis of building.
Thresholding the SSD map gives the symmetric patches.
Use highly symmetric patches for k-means clustering. Here k is 10.
The distance metric is the Euclidean distance in the low dimension embedding that is random projection of RGB space.
Here are 10 cluster centers:
The average patch in a cluster:
The patch that is closet to cluster center:
The clusters distribution on original image:
The different color pixels stand for the center locations of patches for different clusters. Seems still not satisfactory. The ideal clustering result for me is to label the same cluster to the windows on the same facade.
