I am back with more updates on my handwriting recognition project!
In my last post, I talked about developing Scribble-pad for accepting user input that we can process for handwriting recognition. Another way of capturing input can be by a phone camera, so the user can just click a picture of the text. I’ll post about the camera input in a separate post, but today I would like to show you how I did some image processing on the image we saved from the Scribble-pad.
Going for the complete handwritten word seemed too ambitious to me, so I’m going to start by recognizing a letter first!
Logic
First we allow the user to write a letter and save it to a file using our Scribble-pad.
A possible way to recognize the letter could be to compare the saved image with all the other english alphabets and numerals. If it matches to a threshold percentage, it’s more likely to actually be that character. There can be some errors because the style and size of the handwriting of every user would differ. We can do some amount of image processing to detect if the user’s handwriting is too small to compare and enlarge it to a working size, however nothing can be done if the user has bad handwriting. (So, my engine wouldn’t work if you have a really bad handwriting)
Implementation
So following the logic I outlined above, we would need source images of all the characters of english letters and numerals. Where do I get these from? The answer is to generate these images from Qt.
Here’s how I generated the images for upper case characters A-Z. These images were generated as follows,
ascii_counter = 65;
QFont font("Mangal",12, QFont::Bold, false);
for(int i = ascii_counter; i<91; i++)
{
QPixmap pix(20,20);
pix.fill(Qt::white);
QPainter p(&pix);
p.setPen(QColor(0,0,0));
p.setFont(font);
p.drawText(QPoint(3,18), QString(QChar(i)));
pix.toImage().convertToFormat(QImage::Format_Mono);
pix.save(FileRoot+ "img_"+ QString::number(i) +".bmp");
}
Similarly, I generated images for lower case characters a-z and numerals 0-9. I used the image size as 20x20px, just because I feel they were sufficiently good for comparison. Besides, the bigger the resolution, the bigger the size of the images!
Note, I also experimented with the font that is used to generate these images. I am using Mangal, size 12, Bold font here, but we can generate the character set for multiple fonts if we want our engine to be more efficient. For example, Lucida Console handwriting font would be good for running handwriting.
Now, to compare the user scribbled image (let’s load it in desImg) with the images that I generated. Here’s how I’ll do the comparison,
QPixmap recalculateResult()
{
QPainter::CompositionMode mode = QPainter::CompositionMode_Screen;
QPainter painter(&resImg);
painter.setCompositionMode(QPainter::CompositionMode_Source);
painter.fillRect(resImg.rect(), Qt::transparent);
painter.setCompositionMode(QPainter::CompositionMode_SourceOver);
painter.drawImage(0, 0, desImg);
painter.setCompositionMode(mode);
painter.drawImage(0, 0, sourceImg);
painter.setCompositionMode(QPainter::CompositionMode_DestinationOver);
painter.fillRect(resImg.rect(), Qt::white);
painter.end();
return QPixmap::fromImage(resImg);
}
Generated Images
