If you are using double pane glass, you will achieve an R value of about 2. If you replace the windows with glass with a low-e coating and argon gas fill, you will increase the R value to almost 4. This would increase the interior temperature by an estimated 4-10 degrees.

However, as you mentioned, air infiltration could be an even bigger factor. For our clients, we use testing equipment to measure the air infiltration, then we modify the windows for better air infiltration and test again. This enables us to predict long term energy savings and can also have a dramatic effect on localized cold spots in the building.