Since you know that the magnetometer is off slightly from due North, check to see if the amount happens to be the magnetic declination for where you are. If so, then there are ways to solve it. You can use this map:
http://joesportsandsurplus.com/shared/media/editor/file/magnetic_declination_chart_canada_2000[1].pdf
Alternatively, your government publishes on the web an online calculator, where you just type in your GPS reading and the current year:
http://www.geomag.nrcan.gc.ca/calc/mdcal-en.php
In either case, if the amount of error you have been detecting is in fact the magnetic declination for where you are, then the magnetometer is OK and you are on the right track. If you are content to using it just for roaming around your own city, just use the magnetic declination as an offset to the reading typing it into the program. You can make it a permanent constant (for a year or so).
If you want the program to be more general so you or others can use it elsewhere, you might be able to somehow tie the GPS reading to the online calculator. Some of the other forum folks might be able to guide you regarding coordinating your app with a webpage.
The problem is when you are in the boonies with no internet access, and that is generally when you actually have a need for a compass. l In that case, you are somewhat on the right track with your theory about taking readings from multiple locations.
You don't need three. You need to take a GPS reading from a place that has some prominent feature that you will be able to see from elsewhere. Then when you are elsewhere but you can still see the feature, you take your current GPS reading.
You can calculate the direction from where you are to the feature using spherical trigonometry. Compare that with whatever bearing the device gives you from your program when you point it at the feature. The difference will be the magnetic declination, and you use that value to adjust all other readings from that location.