Image Processing, Computer Vision and AI + IoT + 5G are going to and have been changing our life. They should not be something mysterious but something be applied to daily life.
This is an article inspired by a free side dish + my curious soul.
I will divide my article into 2 sections: the fun part and the technical part.
The Fun Part will show you what this project does and how it looks like. That's it, simple and short :) but if you were as curious as me and would like to know the details of the implementation then keep reading.
In the Technical Part, I will treat it as my technical note to remind me what I did for solving the problems and why I did that.
It was a Sunday afternoon, we had been cooking for ages and were sick of doing so... finally we decided to try the expensive hotpot Haidilao. The Haidilao is famous about its premium services and expensive cost.
I know they offer nail and hand skin-care services to customers in waiting queue and yes I actually got the service :)
我会把本文分成两个部分： 看热闹 和 从技术角度谈如何实现这个小项目。
一个风和日丽的周日， 在家做饭许久的我们，被困在澳洲不能回国已一年有余，生活是如此单调。。。最终我们决定去吃饭，试一试听说了许久但从来没舍得吃的 海～底～捞。都知道海底捞服务一流，排队送美甲服务、手部护理。我也去试一试～！
But what I did not know was that they also offered me a little maze to play to kill the waiting time. What's even better was if I got it solved while I was still in the waiting queue, they would reward me with a free dish 😊!
到了以后 果然如此！但我不知道的是，排队等候的时候我们还拿到了两张 小纸片，上面印有迷宫。解出来就有菜吃～！ 嘿嘿黑，这不是送分题吗～太简单了～
不到10分钟， 我就被莉莉打败了。。。。好吧 咳咳，如果我不能打败你， 我就编个程序来战胜你。 从照片中提取 迷宫，看起来很有趣～！而且这正好是计算机图像处理的领域，也是我的最爱～
A piece of cake~!
Within 10 minutes, Lily beat me.... huh.... okay, if I couldn't beat you, I could make a program to beat you. Extracting a maze from a photo looks like a good fit for Image Processing!
Here is what it looks like~!
简单介绍一下： 点击猪哥亮 选择你拍摄的 迷宫照片。（小窍门： 保持迷宫在照片的正中间，确保迷宫的外壳（不是那条它外面的壳）是最外面一层的。让迷宫纸片尽量摊平，和背景区分开来。）剩下的就交给Haidilao Maze Buster来做吧～
As you can see here, the app takes a photo and extracts the maze inside the photo and solves the maze problem.
If you were going to have your lovely Haidilao Hotpot, then I have also made it available for you :)
>>>Haidilao Maze Buster<<<
视频看起来挺短，做解迷宫的部分也就花了2、3天， 但是为了让它更稳定，并且找到合适的免费hosting服务 我找了估计至少有3-4周。。。。
现在它仍有一些不足，比如说：如果箭头堵住迷宫出入口的话 就无法识别出进出口。不过我已经很满意当前的表现了， 毕竟我还有很多其他事情要做， 没法投入太多时间。具体有哪些限制，下面会提到。
接下来 具体阐述如果实现迷宫提取和 矫正迷宫
值得指出的是白平衡真太重要啦～！我感觉我浪费了2周的时间就是因为没有做这一步。 图像处理的白平衡，我自己的感觉是，就像 AI里面的 Normalization 一样。如果不做这一步，同样的代码，这张图可以，灯光稍微改变一些 就会导致另一张图就不行。
The technical part
The demo video looks pretty short, to implement it from the scratch, it only took me 2 or 3 days, but... to perfect it and make it available online actually cost me 3 or 4 weeks....
After couple of weeks working on it, it's still not perfect but good enough to satisfy my requirements, so don't expect it to be a commercial project which works for each and every scenario, I have to admit that it has its limitations. I will explain what the limitations are in the following paragraphs.
So here is the procedure how I got it implemented:
Now, let's talk about how to extract the maze out of the photo and get it face straight to us
There are certain steps to do:
I cannot stress it enough that how important of white balance is. I can tell you that 2 weeks of my precious time was wasted because I did not do the White Balance. To my understanding, it behaves like Normalization in Computer Vision for AI. Without doing it, your code would be fragile and endless frustration is waiting ahead.
What did I mean by this?
Any changes of lighting, the human eyes cannot event notice the differences, will jeopardize the result and breaks the program. I was scratching my head to try to figure out why it works for one image but not for another.
To give you an idea of how the white balanced image look like:
As you can see above, it does change the look and feel of the image by equalizing the Red, Green and Blue channels.
This is very helpful, as it makes the rest of the code free from being affected by any changes of lighting, so the code is more stable.
I wasn't thinking of using HSV color space at first as RGB is the most popular color space. But by operating in the HSV, I found out that the maze area is always the darkest area.
可以看出，图片的温度好像改变了，这是通过 均衡化 红绿蓝三色通道达到的。 这一步非常重要，它能让接下来的代码不会受到任何由于光改变所带来的影响，所以代码就会更加可靠一些。 我起先并没有想到用HSV颜色空间，因为大部分时间都是在RGB空间下工作。 但通过观察HSV空间下的 迷宫，我发现 迷宫区域总是最暗的一部分。
This is great, as it enables me to easily pick the maze paper out of the whole photo(Mask).
As I mentioned above, by also applying morphological close and open I can erase out the rest image and leave a big block of white area to indicate where the maze is located: