Bonneville Power Administration

Link to BPA Home Page
Link to EE Home PageEE - Save A WattEE - Compact FluorescentEE - Energy Star
  BPA Home Triangle Bullet Point EE Home Triangle Bullet Point Reports and Publications Triangle Bullet Point Program Evaluation Index Triangle Bullet Point Residential Triangle Bullet Point Evaluation

This is a spacer graphic
This is a spacer graphic Triangle bullet point Program Evaluation Index



This report supplements Lee et at. (1995), which presents the findings of an impact evaluation of the Manufactured Housing Acquisition Program (MAP). Pacific Northwest Laboratory conducted the evaluation and prepared both reports.

This report presents detailed technical information relevant to the MAP impact evaluation. It is intended to provide the interested reader with enough information to answer most technical questions about the analysis and results.

We used a three-tiered process to analyze the energy consumption of both MAP and baseline homes. The information from each analysis was useful for designing our final analysis.

The first approach was a consumption of annual billing data, followed by a simplified regression analysis to adjust for major home characteristics. We compared the mean annual kWh consumption of MAP and baseline homes, and used the difference to estimate energy savings. We made no adjustments for long-term weather.

This first analysis showed that MAP homes consumed less electricity than the baseline homes used in our analysis, but the differences were less than the pre-program estimates suggested. We identified several factors to examine further. First, non-electric supplemental heating was more common in baseline homes than in MAP homes. Second, in some cases heat pumps were more common in baseline homes than MAP homes and they tended to reduce energy consumption. Third, the average baseline home in our sample was smaller than the average MAP home, thus reducing the difference between electricity use in MAP and baseline homes. Fourth, we found that the distribution of total electricity consumption and consumption per square foot in MAP homes exhibited less variance than in baseline homes.

The second-tier approach was an application of the PRInceton Scorekeeping Method (PRISM). This methodology uses monthly billing data to estimate coefficients that can be used to predict the non-temperature-and temperature-sensitive portions of energy consumption. We used PRISM to estimate electricity consumption for a 'normal' weather year.

We used PRISM to analyze several samples. We applied it to billing data for the entire sample of homes, all baseline homes only, and all MAP homes only. We then screened the sample to eliminate observations that could not be modeled well by PRISM, and repeated the analyses. The results showed that the standard errors of the savings estimates declined, in most cases, after the billing data were screened. Screening the data, however, considerably reduced the sample sizes and this tended to diminish the accuracy and precision of all estimates.

The PRISM approach, however, could not be applied effectively to produce the energy savings estimates needed in this analysis. This was because of the confounding effects of non-electric heat, differences in the efficiency baselines of interest, and other factors not addressed by the PRISM approach.

We used a detailed regression analysis to control for a wide range of possible energy consumption determinants such as occupant demographics, appliance inventories, and weather. This allowed us to estimate energy savings attributable to the MAP features under different conditions.

This approach was applied to all sample homes for which we obtained billing data. Billing-period (usually monthly) data were used.

Our model used appliance inventories to explain total kWh consumption like a conditional demand analysis (CDA), but was formulated around the anticipated thermal-physical relationships. It included several appliances and the effects of demographic and behavioral variables that were found to be significant. Because differences in heating performance were the primary anticipated effect of MAP, we focused on coefficients for different types of heating systems and combinations of systems in formulating the model.

Several potential limitations of the model are discussed. These include potential difficulties modeling the space-heating response to temperature and the effect of ventilation. Statistical and econometric details of the model are presented. The technique for developing confidence intervals with this model is also discussed.

We compared our study and results with the pre-program analysis conducted to predict energy savings. The purpose was to help explain differences between pre-program expectations and observed results, and to inform the analysis process so that predictions of future program energy savings might be improved.

The pre-program estimates of MAP savings were based on analyses using the SUNDAY computer program. They were considerably larger than our estimates. The factors most likely to account for the differences included the following: dissimilarities between assumed and actual home sizes, differences in ventilation rates, uncertainties in internal and solar gain assumptions, occurrence of random vacancies, both intentional and implicit zoning of homes, and inaccuracies in modeling temperature setbacks. Several of these effects combined might explain much of the apparent discrepancy between our findings and pre-program estimates.

We also examined and compared our study to one by Regional Economic Research (RER). It analyzed electricity usage in both MAP and control homes within the service territories of three of the region's investor-owned utilities (IOUs). The study employed both engineering estimation and econometric estimation using a conditional demand model. It produced energy savings estimates that were considerably smaller than ours. We identified three possible biases in the RER model that might partially explain the differences.

We used levelized costs to assess MAP's cost effectiveness. The approach that we used was published by Bonneville. Bonneville's methodology focuses on regional system cost. We also analyzed cost-effectiveness from Bonneville's system cost perspective, based on the cost of MAP to Bonneville. Using our estimated energy savings and incremental total costs, we calculated the levelized costs of energy savings for MAP homes.

It is important to note that Bonneville's methodology does not address the market transformation effects of programs such as MAP. Trying to account for the effects of market transformation considerably complicates the assessment of program cost-effectiveness. Issues of 'free-riders' and 'free-drivers' and how they affect different cost-effectiveness tests merit specific attention.

Need help finding something? Send us a message, or give us a call at 800-622-4519.
NOTICE: This site is owned and operated by the Bonneville Power Administration, United States Department of Energy. Use of this system is monitored by system and Security personnel. Anyone using this system consents to MONITORING of this use by system or security personnel. BPA Privacy Policy