This is the first month that the ADP national employment report is being produced in conjunction with Mark Zandi and Moody’s Analytics. Previously, Macroeconomic Advisors helped ADP generate forecasts of the Bureau of Labor Statistics (BLS) nonfarm payroll employment report. This month’s report was released with the announcement of a partnership with Mark Zandi and Moody’s Analytics and a newer and better methodology for predicting the key BLS jobs number. The ADP report typically is released one or two days prior to the BLS report which is released the first Friday of each month. ADP claims that, since 2001, the correlation between its forecast of the change in payroll employment and the BLS change in payroll employment is .96. If this is true, as I explain below, the ADP report is either very close to being the best possible forecast of the BLS employment change or ADP has been very lucky in its forecasting over the past decade. Mark Zandi and Moody’s Analytics will have a very difficult time improving on the previous track record for ADP.
Since 2001, the average change in BLS monthly payroll employment has been an increase of 7,000 employees per month. There is considerable dispersion in monthly changes in employment, however. The standard deviation of changes in monthly employment has been 241,000 employees per month. The BLS reports that the standard deviation in monthly employment changes due to “sampling variation” is 61,000 employees per month. In other words, even if the BLS drew an equally large and representative sample of employers and generated a second independent count of the monthly change in payroll employment, it would not be surprising if the second calculation differed from the first by 61,000 employees.
By converting these standard deviations into variances, and taking a ratio of sampling variation to total variation, it follows that 6.4% of the month-to-month variation in BLS payroll employment changes is due to “noise” or “sampling variation”, and 93.6% of the variation is due to “true” changes in employment. Even if the BLS drew an equally large and representative sample of employers and generated a second count of the change in payroll employment in a given month, the correlation between the two different BLS employment changes would be .968. (Because a correlation coefficient is simply the square root of the percentage of variation explained, and .968 is the square root of .936).
This means that even if Mark Zandi had access to the same data as the BLS, it is unreasonable to expect a forecast to be correlated more closely than .968 with the employment change reported by the BLS. A forecast of the BLS change in payroll employment can’t be more reliable than the report itself. The BLS acknowledges that their reports are imperfect. Even if the BLS could make repeated independent calculations in the same month the correlation between independent BLS calculations would be only .968. Consequently, if in the future ADP claims that their new model generates forecasts that are correlated .97 or more with the BLS jobs report we should all be dubious.