Zillow Challenges "Inaccurate" Study as Author Defends Findings

Zillow has challenged a recent study that alleged its Zillow Home Loans (ZHL) products are overpriced and discriminatory, defending itself against flawed methodology and a misleading dataset.

Professor Steven Salop published findings that allege ZHL products are more expensive than the industry norm, but Zillow says the paper is 'misleading' and 'inaccurate', pointing out several flaws in the analysis—not least the fact that it is a draft, non-peer-reviewed paper funded by a direct competitor (CoStar Group).

Meanwhile, an employee at Charles River Associates, a firm for which Salop does consultancy work, says Salop's dataset "should not be used to draw conclusions" because it "lacks various key pricing factors."

Zillow's statement reads:

This report draws inaccurate and misleading conclusions. It was commissioned and paid for by a competitor and relies on selectively chosen data and inadequate controls to produce a distorted view of Zillow Home Loans. Zillow Home Loans is committed to both fair lending and pro-consumer practices. We have a robust fair lending compliance program and Zillow’s fair lending data is validated by third party experts, who also have said this report is not credible. We stand by our business model and have a long track record of helping buyers of all types navigate the home buying process.

Zillow added:

The report relies on Home Mortgage Disclosure Act (HMDA) data, which is not designed for market pricing comparisons or borrower-level impact analysis. As federal regulatory agencies have consistently emphasized over many years, HMDA data is insufficient for drawing conclusions regarding fair lending disparities. The single most important driver of mortgage pricing, borrower FICO score, is not included. The author acknowledges this in the report as a “potentially significant gap,” but never resolves it, leaving a critical gap in the analysis. The study does not define an appropriate peer set, resulting in invalid comparisons between Zillow Home Loans and lenders with fundamentally different business models.

Dr. Marsha Courchane, Practice Leader of Financial Economics at Charles River Associates, questioned the use of the data used in Salop's study, commenting:

"As federal regulatory agencies have consistently emphasized over many years, publicly available Home Mortgage Disclosure Act data used in Mr. Salop’s analysis for CoStar should not be used to draw conclusions regarding fair lending disparities because it lacks various key pricing factors. Independent analysis performed by CRA’s fair lending experts as part of Zillow Home Loan's proactive fair lending compliance monitoring program has consistently found no disparities in average mortgage loan pricing based on borrower race or ethnicity for the years studied by Mr. Salop. CRA’s fair lending analysis is based on proprietary Zillow Home Loans data that includes a comprehensive set of non-discretionary loan-level pricing factors that are commonly used in mortgage loan pricing across the industry."

Professor Salop defended his comments, saying:

"I stand by my analysis, which shows consistent Zillow loan overcharges.  The main focus of the study was Zillow’s overall average price differential, controlling for other important explanatory variables. Zillow’s statement appears to criticize the race and ethnicity results.  But Zillow charges more on average to all borrowers, a point that Zillow that does not seem to contest.

"The study could not address what particular features of Zillow's business model led to it charging borrowers’ higher costs.  The study noted in the introduction that the class action plaintiffs have alleged that deceptive steering is an important feature of Zillow’s business plan.   Zillow may have a different explanation for why it charges more.   But its statement did not explain why it charged more on average to all its borrowers.

"The study was not flawed and it was not based on selectively chosen data.  It used the CFPB’s public HMDA data base that other researchers have used.  This public data base does not disclose the credit score.  However, the data base does include key data points that affect the credit score and have been found to be key determinants of the score.  These data points include debt-to-income ratio, combined loan-to-value ratio, income, age, and property location.  The study uses these as control values to help to capture the effect of the borrower’s credit score.

"To be clear, my paper is public.  I posted it on the SSRN.  It is available for independent peer review. If Zillow wants to show that its loans are not more expensive, it should prove it with data that can likewise be reviewed."

CoStar also defended the study, saying the Group "exercised zero influence over the study’s outcome" and "Zillow’s practices are indefensible."