Exhaustive study yielded no racial or ethnic bias, officials assert
A growing chorus of pundits – including those at the venerable Urban Institute – assert that automated valuation models don’t help in weeding out instances of bias as it relates to appraisals, despite their growing usage. But officials at Veros Real Estate Solutions are quick to disabuse the notion among naysayers by releasing a study showing its AVMs produced no evidence of racial bias.
In a study of 50 Chicago ZIP codes, the AVM at Veros yielded no evidence of such bias, company officials claimed during an interview with Mortgage Professional America. What’s more, the proportion of properties undervalued by 15% or more had no racial correlation whatsoever, officials asserted. Buoyed by the results, the company plans to conduct similar assessments at four more major metros, officials told MPA.
“I think it’s fair to say that when the research literature review was conducted, we found just about everything,” chief economist Eric Fox (pictured above) told MPA. “We had folks who said ‘absolutely it exists, and here is the evidence to prove that.’ There were other people who said ‘this is what we found; we’re not sure if we can attribute it to bias by appraisers but we definitely see some issues. And then the other extreme was ‘it doesn’t exist.’ There wasn’t like 95% that were in one of the three camps and 5% in the other two – it was widespread what we saw.”
Such disparate outcomes effectively served as impetus for the Veros study, Fox suggested: “That was what motivated us to take a step back and say ‘let’s just look at the data. Let’s not talk about stories. Let’s not talk about anecdotal evidence, let’s go with the data’. That’s why we decided to go with the ZIP code-related data and the main statistics we were looking at were the percentage of homes that were undervalued by our automated valuation models. So, we’re not talking about appraisers undervaluing homes, we’re talking about automated valuation models undervaluing and based on that data across different ZIP codes with known differences in ethnicity, race, religion, etc., we found no differences.”
Study co-author Reena Agrawal (pictured below) highlighted a key difference in the Veros study compared to other prevailing literature on appraisal bias: “The research you have seen or read about has to do with appraisers or appraisals,” she said. “But our report has more to do with automated valuation models. It’s the first research that has been done with respect to an automated valuation model. That has not been done before.”
Read more: Multiple efforts underway to weed out appraisal bias
In a May 2022 brief authored by Linna Zhu, Michael Neal and Caitlyn Young, authors wrote: “Automated valuation models (AVMs) represent the promise of greater efficiency and lower costs for the mortgage industry. But research has suggested that AVMs can produce racially disparate outcomes – namely, higher error as a percentage of value in majority-Black neighborhoods – that highlight the importance of technology equity. Potential inequities produced may reflect data omission. But they may also result from racial disparities in model inputs or from the modeling techniques AVMs use.”
The AVM trend does offer good news as posited by the Urban Institute in that the models “…may reduce racial bias in home values appraisers have estimated,” Urban institute authors wrote in their paper. “Hypothetically, reducing human input should reduce racial disparities in property valuations,” the authors wrote. “But a previous paper we wrote suggested that AVMs can produce racially disparate home value outcomes.”
In that previous report – which compared majority-Black and majority-White census tracts in the Atlanta, Memphis and Washington, DC, core-based statistical areas, analysts did not find systematic evidence that AVMs undervalued sales prices in majority-Black neighborhoods or majority-White neighborhoods, according the the paper. Moreover, the absolute AVM error – measured as the absolute-value distance between the AVM estimate and sales price – was greater, on average in majority-White neighborhoods than in majority-Black ones, researchers found.
“But the percentage magnitude of AVM error, which measures the absolute difference as a share of the sales price, was greater in majority-Black neighborhoods,” Urban Institute analysts wrote. “This indicates that the degree of absolute AVM error in majority-Black neighborhoods is magnified by the significantly lower home prices in majority-Black neighborhoods. Yet, even after controlling for property differences, neighborhood conditions and turnover, a neighborhood’s majority race was still a significant determinant of the percentage magnitude of AVM error.”
The Veros findings are decidedly different. To evaluate the performance of VeroVALUE – the name of its proprietary AVM first developed 21 years ago – analysts focused on three measurements: The proportion of properties undervalued by the AVM by more than 15% (P15L), the proportion of properties undervalued by more than 15% (P15H) and the median absolute error (MAE.) Measurements were taken from December 2021 sales data for the same Chicago ZIP codes and VeroVALUE’s estimate of value for those properties immediately before their sale. The difference in these values, or errors, were used for the analyses.
Read next: Housing racial discrimination – dismantling its legacy
The upshot: “This report finds no empirical evidence of racial bias in the VeroVALUE AVM results,” company analysts concluded. “More specifically, P15L is not significantly correlated with the racial composition of ZIP codes. Hence, there is no evidence of systemic undervaluation from the VeroVALUE AVM as the percentage of minority population increases. Further, there is no evidence of systemic overvaluation (P15H) from the veroVALUE AVM as the percentage of White population increases. Additionally, MAE is not significantly explained by the racial composition of communities. That is to say, the AVM does not exhibit racial bias and is no less accurate in minority or disadvantaged communities.”
The conclusion: “This report concludes that the VeroVALUE AVM is a tool that should be leveraged in the property valuation process as an independent determination of value in support of or in concert with other determinations of value and may be effectively used to identify potential bias or discrimination in all valuation products and services.”
Stated simply: “The VeroVALUE AVM is blind to racial, gender or related demographic data – both as it relates to parties involved in a specific transaction and the hundreds or thousands of properties and related data analyzed as part of the AVM determination of value.”
The issue of appraiser bias has been heightened of late, as illustrated by the development of a task force by the Biden administration to study the problem in a meaningful manner. For now, however, the debate over potential bias by non-human appraisers will likely rage. But on one front – the importance of accurate appraisals amid a widening borrowing gap negatively impacting minorities free of bias - is agreed upon. “Deals don’t get done, mortgages don’t get funded, HELOCs and refinance don’t happen if the value doesn’t come in,” Fox said in envisioning a world without accurate appraisals. “Or someone else has to write a check at closing and a lot of borrowers don’t have that luxury. If we’re off by 5%, that’s a big deal.”
