Image understanding starts from recognizing the wide variety of patterns of image patches at different locations and resolutions. It is therefore useful to understand statistical properties and construct statistical models of image patches of natural scenes. Natural image patches can be roughly classified into three regimes: stochastic textures, object shapes or textons, and geometric lines and regions. In this talk, I will explain that these three different regimes can be unified by what we call information scaling, i.e., the change of statistical properties of image data over the change of resolution. Moreover, the three regimes of patterns can be modeled within a unified framework of what we call information projection, i.e., iteratively projecting the current model onto a manifold of distributions to obtain an updated model. The talk is based on joint work with Song-Chun Zhu and Cheng-en Guo.