real estate marketing, advertising, blogging, and technology

Zillow’s Zindex does not include recent home sales, yet it professes to be a reliable indicator of home values.

From Zillow’s May 6, 2008 Press Release:

Home values in the first quarter of 2008 fell 1.6 percent from the fourth quarter and 7.7 percent from the year-ago quarter, marking the most significant year-over-year decline in the past 12 years*. (emphasis added)

From Stan Humpries post at the Zillow Blog:

Conditions continued to worsen in Q1 as U.S. home values continued their slide down with the Zindex posting a 7.7% year-over-year decline. (emphasis added)

Sounds like serious stuff, this Zindex. It’s not. It’s a statistical sleight of hand– a Zillow illusion– a Zillusion, if you will. Why? Because a Zindex is NOT based on actual home sales. That’s right.

Not a single home sale is included in the Zindex as a data point.

So what exactly is in this Zindex stew? Nothing but zestimates and boiled potatoes (those folks who proclaim the Zindex as an authoritative housing value report). For those who don’t know it, zestimates are NOT home values, despite what Mr. Humphries writes or Zillow publishes. Zestimates are Zillow estimates of home market values based on computer calculations which depend in large part on data from the local tax assessor’s office, which data may be outdated, incomplete or outright wrong (neither the computer nor the Zillow stat man visit your home). Zestimates are inherently erroneous– to what degree is unknown— the zestimate’s degree of error is not determined UNTIL a home is actually sold. This is a key point– the actual sale determines the zestimate’s worth as a home value indicator— it is the actual home sale which creates the error rate that Zillow touts as a measure of a zestimate’s accuracy and reliability.

So, if actual sales are the gold standard of value against which zestimates are measured, why are they excluded from the Zindex as data points? The short answer is Zillow believes its zestimates are more reliable than recent actual sales to measure housing value trends. (David G of Zillow has called recent sales data a questionable “legacy approach”). This is probably based on the notion that it is better to have estimates of all homes than actual sales prices of some. While this in an attractive theory when there is a dearth of recent sales, it is nonetheless flawed when one is using zestimates of questionable worth. (It is even more unreliable when sales are robust.) Keep in mind that by Zillow’s own admission, a zestimate is only a starting point for determining a home’s value. So, start by asking yourself how a gaggle of starting points can rise to any level of authority of housing values.

Yet, the Zindex database is wholly comprised of zestimates, these starting points, of unknown error. And actual home sales, the ending points, and the zestimate’s gold standard, are left out. But the Zindex is what Zillowsticians are peddling to the real estate industry and the public as authority on the state of home values. Thanks to newspapers picking up the story (and giving the Zindex credibility), the public may be buying it. I’m not. Now, you can expect a comment from David G that because the Zindex is the median zestimate (middle value), the Zindex somehow overcomes the negative effect of having no actual home sales data points and an unknown error rate. Hooey with a capital H, Mr. G. Actual recent home sales are more valuable than zestimates as data points.

Question Zillow (and David G) will not answer: Does the exclusion of recent sales data points make the Zindex a more reliable report from one that does contain actual sales?

In my opinion, the exclusion of recent home sales makes the Zindex less reliable than one that includes recent home sales.

Tip to Zillow to Improve its Zindex: Since you believe all homes should be represented in your Zindex, you should include recent home sales, in lieu of zestimates, for those homes. (You can send me my fee in cash in a brown paper bag.)

*Since Zillow’s Zindex only came into existence in 2006, the Press Release reference to a 12 year low is misplaced. It’s like comparing zapples to zoranges.

[Image: Waterfall by M.C. Escher]

Update: So as to avoid any misunderstanding, the Zindex data points are exclusively zestimates.  Even if there is a recent sale, the Zindex will not take the sales price as a data point but, instead, take the zestimate. (To the extent there were recent sales, they are crunched by the Zillow machine into the zestimates, which nonetheless remain different than the recent sale prices.)  See the comments below for further clarification and examples.

Related Posts:

Zillow Housing Reports: The Statistical Lie of Estimated Truth

Zillow: Truth by Association

The Power of Print: Perception as Truth (see David G’s comment that “recent sales data” is a questionable “legacy approach”. Oh yeah, well this legacy of recent sales is what tax assessor’s use and, thus, Zillow.)

Technorati Tags: zillow, zindex