Java – Use warpPerspective() on a series of points given by HoughCircles(), OpenCV

I’m trying to detect the position of the billiard ball on the table from an image taken with perspective angle. I’m using the getPerspectiveTransform() method to find the transformation matrix, and I only want to apply it to the circles I detected with HoughCircles. I tried to change from a fairly large trapezoidal shape to a smaller rectangular shape. I don’t want to convert the image first and then find HoughCircles because the image is too distorted for houghcircles to provide useful results.

Here is my code :

        CvMat mmat = cvCreateMat(3,3,CV_32FC1);
        double srcX1 = 462;
        double srcX2 = 978;
        double srcX3 = 1440;
        double srcX4 = 0;
        double srcY = 241;
        double srcHeight = 772;

double dstX = 56.8;
        double dstY = 33.5;
        double dstWidth = 262.4;
        double dstHeight = 447.3;

CvSeq seq = cvHoughCircles(newGray, circles, CV_HOUGH_GRADIENT, 2.1d, (double)newGray.height()/40, 85d, 65d, 5, 50);

JavaCV.getPerspectiveTransform(new double[]{srcX1, srcY, srcX2,srcY, srcX3, srcHeight, srcX4, srcHeight}, 
                  new double[]{dstX, dstY, dstWidth, dstY, dstWidth, dstHeight, dstX, dstHeight}, mmat);
        cvWarpPerspective(seq, seq, mmat);

for(int j=0; j<seq.total(); j++){
            CvPoint3D32f point = new CvPoint3D32f(cvGetSeqElem(seq, j));

float xyr[] = {point.x(),point.y(),point.z()};
            CvPoint center = new CvPoint(Math.round(xyr[0]), Math.round(xyr[1]));

int radius = Math.round(xyr[2]);
            cvCircle(gray, center, 3, CvScalar.GREEN, -1, 8, 0);
            cvCircle(gray, center, radius, CvScalar.BLUE, 3, 8, 0);
        }

The problem is that I’m getting this error on the warpPerspective() method:

error: (-215) seq->total > 0 && CV_ELEM_SIZE(seq->flags) == seq->elem_size in function cv::Mat cv::cvarrToMat(const CvArr*, bool, bool, int) 

Also, I guess it’s worth mentioning that I’m using JavaCV in case the method calls look different from what you’re used to. Thanks for your help.

Answer:

The

problem with what you want to do (except for the obvious, opencv won’t let you do that) is that the radius can’t really be twisted correctly. As far as I know, when m is the transformation matrix. You can crack the radius by transforming all the points of the original circle and then find the maximum distance between x’y’ and those points (at least if the radius in the distorted image is expected to cover all of them)
By the way, mIJx = m(i,j)*x (just for clarification).

End answer.

Everything I write is based on the C++ version, I’ve never used JavaCV, but as far as I can tell, it’s just a wrapper that calls a native C++ library.

CvSeq is a sequence data structure that behaves like a linked list.
Assertion your application crash is

CV_Assert(seq->total > 0 && CV_ELEM_SIZE(seq->flags) == seq->elem_size);

This means that your SEQ instance is empty (the total is the number of elements in the sequence) or the internal SEQ flag is corrupted in some way.

I recommend that you check the total number of members of CvSeq and the cvHoughCircles call.
All of this happens before the actual implementation of cvWarpPerspective (which is the first line in the implementation, and it only converts your CvSeq to cv::Mat). So it’s not a distortion, but what you did before that.
Anyway, to understand what’s wrong with cvHoughCircles, we need more information about creating newGray and circles.

This is an example I found on the javaCV page (Link).

IplImage gray = cvCreateImage( cvSize( img.width, img.height ), IPL_DEPTH_8U, 1 );
cvCvtColor( img, gray, CV_RGB2GRAY );
 smooth it, otherwise a lot of false circles may be detected
cvSmooth(gray,gray,CV_GAUSSIAN,9,9,2,2);
CvMemStorage circles = CvMemStorage.create();
CvSeq seq = cvHoughCircles(gray, circles.getPointer(), CV_HOUGH_GRADIENT,
                                                2, img.height/4, 100, 100, 0, 0);
for(int i=0; i<seq.total; i++){
        float xyr[] = cvGetSeqElem(seq,i).getFloatArray(0, 3);
        CvPoint center = new CvPoint(Math.round(xyr[0]), Math.round(xyr[1]));

int radius = Math.round(xyr[2]);
        cvCircle(img, center.byValue(), 3, CvScalar.GREEN, -1, 8, 0);
        cvCircle(img, center.byValue(), radius, CvScalar.BLUE, 3, 8, 0);
}

From what I saw in the implementation of cvHoughCircles, the answers are saved in circles buffs, and finally they create CvSeq returns from them, so if you assign circles buffs incorrectly, it won’t work

Edit:

As you can see, the CvSeq instance returned from cvHoughCircles is a list of pip values, which may be the reason for the assertion failure. You cannot convert this CvSeq to cv::Mat.. Because it’s not cv::Mat. To get only the circles returned from cvHoughCircles in the cv::Mat instance, you need to create a new cv::Mat instance and then draw all the circles in CvSeq on it – as shown in the example provided above.
Specific warpage will work (you will have an instance of cv::Mat, which is what the function expects – cv::Mat as the only element in CvSeq).

End editing

here is a C++ reference to CvSeq
If you want to fiddle with the source code instead

cvarrToMat is in matrix.cpp

CV_ELEM_SIZE is in types_c.h

cvWarpPerspective is in imgwarp.cpp

cvHoughCircles is in hough.cpp

Hope this helps.

By the way, your next error could be:
cv::warpPerspective in C++ OpencCv asserts dst.data != src.data

Therefore

cvWarpPerspective(seq, seq, mmat);

It won’t work because your source and target pads reference the same data.
Not all functions in OpenCV (and image processing in general) are valid in-situ (Because there is no in-situ algorithm or because it is slower than other versions, e.g. transpose of n*n pads will work in situ, but n*m where n!=m will be harder in situ and may be slower)
You cannot assume that using an SRC matrix as a DST will work.