FTC Studies The Accuracy Of Consumer Credit Reports. Plenty Of Errors To Be Fixed By Credit Reporting Agencies

The blog post on Monday discussed the 60 Minutes report about the failures in the dispute process at credit reporting agencies to fix mistakes in consumers' credit reports. Today's post discusses the recent U.S. Federal Trade Commission (FTC) survey, which prompted that news report.

The FTC survey analyzed the accuracy and completeness of consumer credit reports. This was the agency's fifth such report. Section 319 of the 2003 FACT Act requires the FTC to conduct a study of the accuracy and completeness of consumer credit reports.

Major findings from this FTC study:

1. 26% of consumers (262 of 1,001 participants) identified errors on their credit reports that might affect their credit scores. 19% of credit reports (572 of 2,968 reports) had an alleged error reported by participants
2. 20% of consumers had an error that was corrected by a credit reporting agency (CRA) after it was disputed, on at least one of their three credit reports
3. Of the 572 credit reports where an error was submitted, 399 reports (70%) were modified by a credit reporting agency, and 211 (36%) had a credit score changed. Those same 211 credit reports are 7.1% of all credit reports in the study
4. Of the 262 consumers who identified alleged inaccuracies in their credit reports and filed disputes, 206 consumers (80%) had a modification made by a credit reporting agency to their credit report in response to the dispute. Of these, 129 consumers (12.9% of all 1,001 participants) experienced a change in credit score following the dispute process
5. Slightly more than 10% of consumers saw a change in their credit score after the credit reporting agencies fixed errors on their credit reports
6. Approximately 5% of consumers had a maximum credit score change of more than 25 points, while 0.4% of consumers had a maximum score change of more than 100 points

If you skimmed or quickly read the high-level findings or the FTC press release, then you might assume that there is no problem -- and you would be wrong for a several reasons. First, that 20% of consumers found an error in at least one of their credit reports means that about could be as many as 40 million people (20% of the 200 million Americans with credit reports) have at least one error in one of their three credit reports with Experiran, Equifax, or TransUnion. That seems to be a huge error rate.

Second, this error rate is based on a percentage of consumers. Some credit reports had multiple errors in them. So, a more accurate error rate would be based on the number of credit reports with errors compared to the total number of credit reports. Or, an even better error rate would be the average number of errors in a credit report. Third, the report doesn't seems to measure the percentage of error items that credit reporting agencies don't fix which they should have fixed (that's another type of error).

Fourth, that 20% error rate is the number of consumers who reported errors and the credit reporting agencies fixed them. (Explanation below.) A much higher rate of consumers reported errors: 26%. It seems that the real error rate is far higher.

I waded through the 370-page FTC report because credit reports are critical documents. Consumers need them to be accurate do business with lenders, and lenders use these documents constantly. Plus, credit reports contain a lot of important, sensitive, personal information about you, your lifestyle, and the purchases you've made:

"... (1) Identifying information including name, address, birth date, SSN, and previous/alternate names and addresses; (2) Credit account information including information about current and past credit accounts such as mortgages, car loans, credit cards, and installment payments; (3) Public records such as bankruptcies, foreclosures, civil judgments, and tax liens; (4) Collection accounts, which include unpaid debts (such as medical bills) that have been turned over to collection agencies; and (5) Inquiries (subscriber requests to access a consumer credit report)."

When you apply for credit or when a potential lender requests to view your credit report to make a lending decision, a "hard inquiry" results. Too many "hard" inquiries and your credit score can go down. The study identified different types of errors (bold emphasis added):

"... we define a ‘potential error’ as an alleged inaccuracy identified by the participants with the help of the study associate... Lenders often use the credit score associated with a credit report to assess the credit risk of a particular consumer. Therefore, we define a ‘potentially material error’ as an alleged inaccuracy in information that is commonly used to generate credit scores. Information used to generate credit scores include the number of collections accounts, the number of inquiries (hard pulls on a credit file), the number of negative items such as late or missed payments, and other factors. An alleged error is considered potentially material prior to the dispute process simply by its nature as an item used to generate credit scores... We define a ‘confirmed material error’ in several ways, though all rely on a confirmed error being determined as a result of the FCRA dispute process..."

If you are reading this closely, then you realize that credit reports contain errors both in the information used to calculate consumers' credit scores, and in the information not used to calculate credit scores:

"Errors in header information (current/previous address, age, or employment) are not considered in determining a FICO credit score and thus are not defined as material in the context of this study."

In my opinion, this distinction does a disservice to consumers. It tolerates a certain level of sloppiness; that it is okay for credit reporting agencies to get their credit reports mostly correct. Header information elements are no less important than other credit report elements. These header elements could be used to match credit reports for a person with input submitted by lenders and/or within dispute investigations. Second, a credit report is such an important document that it needs to be correct. Period. Credit reports are important because:

• They are used by so many potential lenders
• Consumers pay large sums for credit monitoring services
• Credit reporting agencies share information with data brokers

Errors are errors. Period. They all are important. Fix them all. Decades ago and early in my business career, I learned an important lesson about producing a quality product or service:

"Why spend all this time finding and fixing and fighting when you could prevent the incident in the first place?... It is much less expensive to prevent errors than to rework, scrap, or service them... It is always cheaper to do the job right the first time."

Either the credit reporting agencies haven't learned these lessons about quality, or they intentionally choose not to pursue a goal of zero defects.

To the good, the FTC study looked at error rates among header information from credit reports:

"In cases where a participant identified only an error in header information, the participant was instructed to dispute the error directly with FICO and the participant’s credit report was not redrawn. For the individuals with material errors and header information errors, the outcome for the header information disputes is known. The third most common alleged inaccuracies occur in the data on header information (154 alleged errors on 127 reports, comprising 4.3% of the sample). Note this represents a lower bound of the frequency of header information errors, as reports with errors only in header information are not included. The modification rate for header information is higher than that of other alleged material error types (99 modifications, comprising 64.3% of the disputed header information items)."

In other words, in this study 127 credit reports had 154 alleged errors in the header information, or 1.2 errors on average per credit report. The credit reporting agencies fixed 99 of these 154 alleged errors -- what I would calculate as a 64.3% correction rate for header items. Still, this is still a best-case correction rate, because the above excluded instances where the only error reported by the consumer was in the header information.

The study found that the main types of confirmed material errors (that could affect a consumer's credit score) that were fixed by credit reporting agencies were:

"... errors in the tradeline (consumer accounts) or collections information. The most common alleged inaccuracies occur in the data on tradelines (708 alleged errors on 409 reports, comprising 13.8% of the sample) or collections accounts (502 alleged errors on 223 reports, comprising 7.5% of the sample). The most commonly modified errors are tradeline information errors (395 modifications) and collections information errors (267 modifications)."

The supporting details:

Error Type	# of Alleged Errors	Items Modified #(%)	# Reports with Alleged Errors	Avg. # Alleged Errors / Report	Reports with Errors Modified #(%)
Collections	502	267 (53.2%)	223	2.3	146 (4.9%)
Duplicate Entries	65	30 (46.2%)	39	1.7	27 (0.9%)
Header Information	154	99 (64.3%)	127	1.2	90 (3.0%)
Inquiries	88	48 (54.5%)	48	1.8	34 (1.1%)
Derogatory Public Records	44	25 (56.8%)	35	1.3	20 (0.7%)
Tradeline Information	708	395 (55.8%)	409	1.7	267 (9.0%)
Total	1,561	864 (55.3%)	--	--	--

Note: the report did not provide totals. I calculated that row. Overall, slightly more than half (55.3%) of error items reported by consumers are fixed -- and this chart includes only the material errors that could affect a consumer's credit score. What I found interesting: regardless of the error type, there is consistently more than one error per credit report.

The following chart highlights how often credit reporting agencies co-mingle your information with other persons' information:

Error Type	# of Alleged Errors	# Items "Not Mine" Alleged	Items "Not Mine" Corrected #(%)	# Reports With This Alleged Error	# Reports With "Not Mine" Alleged	Reports With "Not Mine" Corrected #(%)
Collections	502	413	209 (50.6%)	224	190	116 (61.1%)
Inquiries	88	88	48 (53.9%)	48	48	33 (68.8%)
Tradeline Information	708	246	133 (54.1%)	409	144	81 (56.3%)
Total	1,561	747	390 (52.2%)	--	--	--

Again, the report did not calculate the total row. I did. As you can see, credit reporting agencies fixed slightly more than half of errors consumers reported as not theirs. How the researchers calculated the effects on consumers' credit scores from credit report errors:

"After the disputes were filed and completed, the study associate drew new credit reports for the consumer and analyzed whether there were changes to the report in response to the dispute. If there were no changes to the report, the original FICO score is relevant for our calculations and if all the alleged inaccurate items were modified by the CRA, the provisional FICO rescore is the relevant credit score. If only some of the disputed items were changed, the modified report was sent to a FICO analyst for a second rescoring to assess the impact of the modifications. The relevant FICO score at the conclusion of the dispute and rescoring process is then compared to the original FICO score to determine how the credit report inaccuracies affected the consumer credit score."

The reports shared a brief explanation of why credit reporting agencies don't fix errors as consumers who reported errors expect:

"... There are a number of reasons, however, why a CRA may make changes to a credit report that differ from the consumer’s instructions. For example, a consumer may dispute an account balance and instruct the CRA to change the balance to a specific amount (i.e., the consumer alleges what is incorrect and what action by the CRA would set it right). If the CRA cannot confirm the existence of the account with the data furnisher, the account is removed from the consumer’s credit report; in this case the outcome is not what the consumer requested. In addition, a consumer may dispute multiple items on a credit report as inaccurate and the CRA may only modify a subset of the disputed items, thus suggesting that the consumer was correct regarding some of the inaccuracies on the report but not all."

The report shared a brief explanation of why credit reporting agencies may not fix at all any errors reported by consumers:

"... there are some consumers who file disputes and yet the CRA makes no modification to their report. For the purpose of the analysis within this report, these consumers are not defined as having a confirmed material error. It is important to note that these consumers with alleged potentially material errors that are not confirmed through the FCRA dispute process may still have inaccurate items on their credit reports; however, we are unable to verify the inaccuracy within the design of this study..."

So, the 20% error rate (percentage of consumers who reported errors and credit reporting agencies fixed them) in the study is probably the best-case scenario; and the real-world error rate is higher. How? If an error discovered and reported by a consumer cannot be verified via the FCRA dispute process, then the credit reporting agencies does nothing and that error remains in the consumers' credit reports. The 60 Minutes show documented real-world examples where consumers fully documented errors in their credit reports; which the credit reporting agencies proceeded to ignore (sometimes setting a lawsuit later out of court).

This best-case error rate problem is also backed by the research methodology. The research team included members from the University of Missouri, St. Louis (UMSL), the University of Arizona, and the Fair Isaac Corporation (FICO). The research methodology included consumers selected at random:

"... from the population of interest (consumers with credit histories at the three national CRAs). Ultimately, 1,001 study participants reviewed 2,968 credit reports (roughly three per participant) with a study associate who helped them identify potential errors. Study participants were encouraged to use the Fair Credit Reporting Act (“FCRA”) dispute process to challenge potential errors that might have a material effect on the participant’s credit standing (i.e., potentially change the credit score associated with that credit report). When a consumer identified and disputed an error on a credit report, the study associate informed FICO of the disputed items, and FICO generated a provisional FICO score for the report under the assumption that all consumer allegations were correct. After the completion of the FCRA dispute process, study participants were provided with new credit reports and credit scores. Using the provisional FICO score, the new credit reports and credit scores, and the original credit reports and credit scores, we are able to determine the impact on the consumer’s credit score..."

Descriptive information of the study participants:

FICO Credit Score	Age	Education	Race
589 and below: 18.2% 590 - 679: 20.2% 680 - 749: 21.0% 750 - 789: 19.5% 790 and above: 21.2%	18 - 30: 21% 31 - 40: 20% 41 - 50: 15% 51 - 60: 21% 61 and older: 22%	HS diploma or less: 12% Some college: 31% College degree: 30% Graduate study: 26%	White: 78% Black: 12% Other: 9%

The study never looked at credit report accuracy in the regional and smaller credit reporting agencies. So, there are more than three credit reports per person on average, when you include those smaller and regional agencies. More credit reports and probably more errors.

What do I think of this study by the FTC? It highlights several important concepts:

• How you define an "error" matters. In the study, a conservative definition yielded a 9.7% error rate (defined as the as the percentage of consumers) while a more expansive definition yielded a 21% error rate.
• How you define an "error" matters. The study calculated the much-publicized error rate based on the percentage of consumers who reported errors. To me, a better method is to calculate the error rate based on the percentage of credit reports with errors. This lets you proceed to the next level to calculate which which credit reporting agency has the higher (or lower) error rate.
• How you label an "error" matters. While caclulating the percentage of credit reports with errors fixed and/or the percentage of error items fixed by credit reporting agencies, what you label these is important. The study used what I consider to be clumsey labels:  "Percent of All Reports Examined With This Error Modified" and "Percent of Items With Any Allegation of this Type Modified," respectively. Let's call them what they really are: "Report Correction Rate" "Report Item Correction Rate," respectively. Then, we can examine which credit reporting agency does a better job of fixing credit reports. Sadly, the study did not provide this level of detail.
• How you define "investigation" matters: this includes both the FCRA dispute process and what credit reporting agencies actually do (or don't do) to investigate error disputes reported by consumers. The 60 Minutes report mentioned low-wage staff in other countries simply assign code numbers to error reports without performing a substantial, comprehensive investigation -- which most consumers probably expect.
• Which brand of credit score matters: this study used FICO credit scores, while many credit reporting agencies and other retailers sell different brands of credit scores to consumers
• Where you place the "responsibility" matters. The study is consistent with general practice -- for better or worse -- that places the responsibility for finding and reporting errors with consumers. Why aren't the credit reporting agencies held responsible for finding, reporting and fixing errors on their own? Would they find the same errors that consumers found? Or more? Or fewer?

This FTC study is half a loaf at best. Why?

First, it didn't analyze the real problem of actual errors already reported by consumers that were never fixed -- what I call the correction rate. A better study would investigate both error rates and correction rates, by perhaps using an independent third-party to analyze the dispute process and the supporting documents submitted by consumers to credit reporting agencies. This would get at the true heart of the matter: how accurate credit reporting agencies are (or are not) with using the documentation consumers provide. In other words, lets better understand the errors that weren't fixed which should have been fixed by credit reporting agencies.

Second, it is better to define error rates not as a percentage of consumers, but instead based on either the number of credit reports with errors, or the average number of error items in a credit report. Each consumer has at least three credit reports -- one with each of the three major credit reporting agencies: Experian, Equifax, and TransUnion. Some consumers have more credit reports with the smaller, regional credit reporting agencies.

Third, the study perpetuates a current bias that distinguishes between errors used to make credit score decision and errors not used in this calculation. Errors are errors. Period. Credit reports are so important, that they need to be correct. Fourth, the study ignored the smaller and regional credit reporting agencies.

Fourth, the study methodology had 100% of participants review their credit reports. In the real-world, far fewer consumers check their credit reports for accuracy. In its report, the FTC said:

"... In 1992, the Associated Credit Bureaus (later Consumer Data Industry Association, or “CDIA”) commissioned Arthur Andersen & Company to perform a study about credit report accuracy. Using credit applicants who had been denied credit, the Andersen Study found that only 8% requested a copy of their report and 2% of those denied credit disputed information contained in their report. Following the dispute, 3% of the people who received copies of their report had the original decision to deny credit reversed...."

While the report cites other studies, the important point is this: if only 8% or consumers request copies of their credit reports, then it makes sense to pursue ways to engage more consumers with checking their credit reports for accuracy. Business as usual means a lot of errors go unreported and undiscovered. In a truly open market with credit reports, each credit reporting agency would tout its accuracy levels; unlike the current mess. The FTC needs to make it real for consumers by explaining the real-world costs of inaccurate credit reports with real examples of denied credit and loans with higher interest rates.

Fifth, I found the language in the report and study methodology needlessly confusing. It could have been simplified with clearer labels, such as:

• Consumer Dispute Rate: the percentage of consumers that submitted error reports
• Credit Report Dispute Rate: the percentage of credit reports with at least one error reported by consumers
• Credit Report Average Item Dispute Rate: the average number of error items per credit report submitted by consumers
• Gross Credit Report Correction Rate: the percentage of credit reports with (all or some) error items fixed by credit reporting agencies
• Net Credit Report Correction Rate: the percentage of error items in credit reports where all items are fixed by credit reporting agencies
• Gross Item Correction Rate: the average number of error items fixed (all or partial) per credit report
• Net Item Correction Rate: the average number of error items where all items are fixed per credit report

What is your opinion of credit reporting agencies? Of their dispute process? Of the FTC study? Share you thoughts below.

Download the 2013 FTC FACTA report (Adobe PDF, 20.8 Mbytes).