Replace subimage by its average value

Question

I have the input image in, and the mask image lev (which label each pixel as a number from 1 to nlevels, nlevels=4). I need to get image out, which from each region in==lev will be replaced by its average value.

Here is my brute force approach. I'm wondering if there is any better way of doing it.

Mat ACA::global_average(Mat in, const Mat &lev)
{
 // in: CV_32F
 // lev: CV_32S
 int M=in.rows;
 int N=in.cols;
 in.convertTo(in, CV_32FC3);
 // vector holds average value for each category
 vector<float> count(nlevels,0);
 vector<Vec3f> average(nlevels);
 int pixlev;
 for (int i=0; i < M; i++)
 for (int j=0; j < N; j++)
 {
 pixlev = lev.at<int>(i,j);
 assert (pixlev<nlevels);
 average[pixlev] += in.at<Vec3f>(i,j);
 count[pixlev]++;
 }
 for (int i=0; i<nlevels; i++)
 average[i] /=count[i];
 Mat out=Mat::zeros(M,N,CV_32FC3);
 for (int i=0; i<M; i++)
 for (int j=0; j<N; j++)
 {
 pixlev = lev.at<int>(i,j);
 out.at<Vec3f>(i,j)=average[pixlev];
 }
 return out;
}

P/S:

Here is an example.

Input Image:

Original Image Mask image, it is basically a matrix labelling the pixels into one of 4 categories

Mask Image

Output picture is the average of input image, for each category of pixels. For example, at first I find all pixels has category 1, find their average, and replace all those pixels by the average value.

Output picture

• kobejohn: I don't really follow your description, especially what average you are taking. Could you provide a simple input / lev / output example? Also if you could comment your code I guess it will help people help you.
• kobejohn: Thanks for the update. It's clear to me now. I'll check it out when I have time.
• Paolo Brandoli: Apart from few missing {} on the outer for loops, it looks good to me
• kobejohn: I also don't see any particular inefficiencies in it. Did you have some ideas that you weren't able to implement?
• Dzung Nguyen: I'm thinking of using convolution to perform averaging ( and convolution can be calculated through FFT(n logn). But nevermind, this should be good enough as you guys point out.

Answer 1

Algorithm

You will benefit greatly from using OpenCV's built-in functionality rather than performing this operation yourself. OpenCV supports masked operations, which will only apply to pixels where the mask is nonzero.

cv::Mat global_average(const cv::Mat& input, const cv::Mat& levels)
{
 assert(levels.channels() == 1);
 cv::Mat output(input.size(), input.type());
 for (int level = 0; level < nlevels; ++level)
 {
 const cv::Mat mask = levels == level;
 const cv::Scalar avg = cv::mean(input, mask);
 output.setTo(avg, mask);
 }
 return output;
}

Using functionality already present in OpenCV is not only clearer and shorter (by a factor of 3), but also allows the code to be more flexible. You will notice:

• There is no explicit type conversion or array indexing occurring, which is safer.
• This code is also more flexible: you do not need to convert your images to floating-point or integers.
• It can operate on 8-bit images as well, which can give a significant speedup, since most of the time in the algorithm will be spent on memory access. Simply using the raw 8-bit images gives a ~15% speedup in my tests.
• It is also possible that OpenCV functions may benefit from hardware and parallelism optimizations, which could make them faster than raw loops.

Code Style

Whitespace

Put spaces around your assignments and comparisons: int i=0; i<m; is harder to read than int i = 0; i < M;. Also, be consistent. Sometimes you do this already, but most of the time you do not.

Braces

You have nested for loops, but only the inner loop has braces. Both loops should have braces surrounding the body for consistency.

Precondition Checking

Although it is good that you leave comments indicating the data types you expect from in and lev, it is better to verify this in code. Adding assertions about these conditions makes it easier to debug if a wrong parameter is passed:

assert(in.depth() == CV_32F);
assert(lev.depth() == CV_32S);

Use const when you can

Both M and N do not change during the function evaluation. They can be declared as const int to allow the compiler to perform checks that they are never accidentally modified. Additionally, you can reduce the scope of pixlev by declaring it inside each of the loops. Then it too can be made const:

const int pixlev = lev.at<int>(i,j); // or use auto to reduce type duplication