# Trouble extracting number in Sudoku grid

I am trying to build a Sudoku grabber using OpenCV. I have so far been able to get the boundary of the puzzle and apply a perspective transform to get a bird's eye view. In the next step, I am having trouble detecting the numbers in the individual boxes. Note: **I do not get a perfect square after the perspective transformation. I have some convexity defects at the top.** I am not overly worried about that.

In each individual box, I am trying to extract the number using cv2.findContours and filtering out the number based on aspect ratio and area. The problem I face during this step is that the contours are so disjoint, I am certain they are going to wrongly recognized by the OCR system which is the next step. I have attached the image after the perspective transform and also the contours which I have obtained in a few cases.

```
import numpy as np
import cv2
from matplotlib import pyplot as plt
thresh = cv2.adaptiveThreshold(warp,255,1,1,11,2)
kernel = np.array([0, 1, 0, 1, 1, 1, 0, 1, 0], dtype = "uint8")
kernel = np.reshape(kernel, (3,3))
thresh = cv2.erode(thresh,kernel,iterations = 1)
thresh = cv2.dilate(thresh,kernel,iterations = 2)
# I assume that the perspective transformed image (PTI) is evenly divided into 81 square blocks
# The size of the PTI is 450x450
for x in xrange(0,450,50):
for y in xrange(0,450,50):
mid_point_y = ((2*y)+50)/2
mid_point_x = ((2*x)+50)/2
number_selected = thresh[mid_point_y-15:mid_point_y+15,mid_point_x-15:mid_point_x+15]
all_contour, hierarchy = cv2.findContours(number_selected.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
if len(all_contour)!=0:
cv2.drawContours(number_selected,all_contour,-1,(0,255,0),3)
plt.figure(1)
plt.imshow(number_selected, cmap = 'gray')
plt.title('Number')
plt.show()
```

Edit: 1) I have added the relevant code 2) I have not described the aspect ratio and area code. I wanted to see why there was a single contour for some numbers and multiple contours for some others.

figure_1.png figure_1_actual.png figure_2.png figure_2_actual.png figure_3.png figure_3_actual.png

I don't really understand what you're doing with findContours to obtain such results. They don't look like contours, but more like a skeleton or the output of a distanceTranform. Could you please post your code?

I have added the code

http://opencvpython.blogspot.com.tr/2...