Hi C4H9Li. Yes the C++ template is what is called Tier 2.
As for the icon, I assume you added the icon to your project by right clicking on your project and going Add > Resource > Icon. If so, the icon should be in your res.rc file. Once you've done that, open the res.rc file in the viewer and look at what the icon has been called by VS. The name should look something like IDI_ICON1.
Once you've found this, go to the Core.cpp file in the template and search for the line
which is where the Windows class structure is created. Underneith that, you should see a series of assignments to wcex's components, setting up the window. You want to find where wcex.hIconSm is assigned to and replace that line with the following
wcex.hIconSm = LoadIcon(wcex.hInstance, MAKEINTRESOURCE(IDI_ICON1));
of course with the IDI_ICON1 replaced with whatever name VS has given your icon.
I hope that helps. Let me know if you have any problems with it.