Today I'm coming to you with a topic I've thought a lot about recently, which is economic data and how much we can trust that it's really painting a true picture of what's going on in the economy.
This is particularly on my mind right now because of a few things that have happened.
We've seen major revisions in some important economic indicators, and President Trump recently fired the Bureau of Labor Statistics commissioner who oversees the production of several important economic indicators.
So our concerns about the quality of economic data, as well as its vulnerability to political influences, really warrant it.
I'm Alison Nathan, and this is Goldman Sachs Exchanges.
Each month I speak with investors, policymakers and academics about the most pressing market moving issues for our.
Top of my report from Goldman Sachs Research.
This month.
I first turned to Joseph Briggs, who leads the global economics team in Goldman Sachs Research.
I asked him to dig into why data reliability has become a greater focus recently.
So there's a number of different reasons why economic data quality has shifted into focus.
The first and foremost is that we have seen fairly large revisions to the U.S. payroll survey.
That's brought concerns around how reliable is the data that we're collecting top of mind.
There's also been a number of high-profile issues with funding cuts, particularly around the CPI survey.
And that's brought data quality issues top of mind.
And then President Trump's decision in August to fire the BLS commissioner has brought data quality issues definitely to the forefront.
There's also been a number of global data quality issues that have kept data quality issues top of mind.
The combination of all these things have led to concerns around data quality.
So, when you think about the actual drivers of these concerns about data reliability, what stands out the most?
So there's three things that I think are behind the overall deterioration in data quality.
The first is that the pandemic led to wild swings in economic activity in ways we hadn't seen before.
This led to unusual dynamics, which has made seasonal factors hard to pin down correctly.
And so, if we're looking at sequential data, whether it's activity or inflation, seasonal dynamics have added to data noise.
Second and more of a structural issue, statistical agency funding has been pretty stagnant over the last 10 to 20 years.
It's declined in real terms in the US, and so this has limited statistical agencies' ability to keep up with changing economic dynamics that ideally, you would be capturing as you're updating your measurements.
The third and other, somewhat structural reason is is that we have seen a long-run decline in survey responses.
This accelerated after the pandemic.
It's just a lot harder to get people on the phone to have them tell you how their economy is evolving.
I then asked former BLS Commissioner Erica Groschen what she makes of the particularly large revisions to the recent employment figures and data quality concerns more broadly.
It's important to realize that revisions are not bugs.
They are features.
Right.
It's very easy to produce statistics that you never revise.
You do it once and you don't do it again.
And that's that.
So your choice is you can have it very early and have it be lousy quality, or you can have it very late.
Great quality.
But after you want to need it.
So programs that are designed to have revisions recognize that there is value in timeliness and there's value in accuracy.
And this is a way of providing both.
That can be frustrating to users, but any user is free to ignore the first one or free to ignore the last one.
The BLS gives you a choice.
So with the payroll survey, what's going on.
Well, companies have three months to report in.
For May.
They can report it in at the end of May, they can report it in the end of June and they can report it at the end of July.
And some of them don't report the first time they have a chance to report it in.
They don't because they pay monthly.
And so at the end of May, they still haven't done their payrolls for that pay period.
Then there are all sorts of other reasons why they might not have reported it in.
They might be undergoing a lot of turmoil of some sort or another.
So they're laying off people rapidly or they're hiring rapidly and they just don't have it together.
They're changing their IT system.
Their payroll person is out sick.
Who knows, right?
So the first month, BLS usually gets about two-thirds.
Not too bad.
In the second closing date, it goes up.
And then by the third closing date, they're usually about 94%, 95%.
So the revisions mostly reflect new information that's provided by the late reports.
Now, usually that cancels out.
BLS does this stratified random sample idea.
And if you report late, then BLS just takes the average for the people who reported on time and says that's what's going on in that cell.
Now that implicitly imputes to you the same percentage growth or loss as the people who reported on time.
If the reason that you didn't report is because you're having an extreme event and a lot of other people are suffering that same extreme event at the same time, then the late reports can have a bias to them.
Otherwise, they would more or less cancel out.
In this recent episode, they didn't cancel out.
So what do you make of that?
And how unusual is that?
So there usually are some revisions, but these were large and the two revisions were in the same direction.
So that suggests that there might be something at when you're likeliest to get the biggest revisions and particularly the biggest negative revisions.
Those are often around economic turning points.
So going into recession or slowing dramatically suggests that the companies that have been hit first are more likely to delay their reporting.
So it's not a flaw in the system.
It is what happens when the economy does this.
This is how you see it in the data.
Let me follow up on a couple of things I just kind of stuck in my mind.
One is there has been a proposal that, for example, some of the monthly data goes to quarterly because it would potentially avoid revisions.
Is that a good idea?
The only pressures that I ever heard when I was at BLS about timing was how much faster can you get it to us, not how much more slowly can you get it to us right?
So there's a reason why the most watched and most market moving economic indicators in the world come from the BLS monthly indicators.
Primarily the employment situation year in and year out, month in and year out, and then CPI and the inflation measures.
If people felt that the most important thing was accuracy and were willing to wait for the third month, then they would be free to ignore the first two months of the data.
And you would have evidence of that in the way people traded in the newspapers, in the way policy community reacted to those numbers.
They'd be free to ignore them.
The fact that they don't shows, first of all, that the BLS is right to publish them, because there is good information in them.
And secondly, that the users know this.
And so...
Do, I think we'd be depriving our policymakers and our business leaders and the public of important information that they need to understand the state of the economy.
I think the answer is a very clear yes.
Like Joseph, Erica is concerned about the impact of budget and staffing cuts at the statistical agencies.
So at BLS we're talking about loss of at least 15 of staff probably more like 20 now, and there's still a hiring freeze in place.
So I worry about the effects right now on the agencies.
So there are all sorts of temporary adjustments that BLS in particular is making so that the numbers continue to come out on time, and they all have come out on time so far.
So we have a lot of people being reassigned to things that they didn't do before.
That means there are some risks.
You need to retrain people to do that.
You have people doubling up on assignments.
You have people traveling to places where the staff has been reduced and just substituting for them.
If you look at the BLS website, you will see that one third of its senior leadership positions are vacant.
So you're losing a big swath of very experienced, skilled people.
And there's nobody in those places now.
So that's worrisome.
What this affects is resilience.
So the possibility that there will just be a screw-up of some sort weather-related IT-related whatever, a mistake in data that's not noticed, that kind of thing.
So the resilience, quality control, those things are more at risk than they used to be.
BLS is very good about reporting all of its errata.
I haven't seen any increase in its errata yet, but this could happen down the road.
It definitely cuts back on the capacity to modernize because the agencies, where are they going to get people?
They have to move them from doing things that are not needed immediately to things that are needed immediately.
So their ability to modernize and keep up with things is definitely compromised.
They have already taken the steps of reducing granularity. in a number of the price indices.
And I expect that will happen more and more as they can't collect the data.
And there's probably some reduction in non-response follow-up as well, which would make the sample sizes decrease a bit.
They've really done an amazing job under these enormous strains, but they're not sustainable in the long run.
If nothing changes, going to be looking to reduce some of what it does, eliminate some programs that people depend on cut back in other ways because they have to under these circumstances.
Arthur Laffer, an economist who served on the Economic Policy Advisory Board during the Reagan administration, sees these issues differently.
So are you concerned about economic data quality today?
And if so, where are you most concerned?
I am concerned about data.
Whenever you get something like COVID-19, you're going to get an explosion in data.
It's especially going to be serious when you use seasonal adjustments that are such like the ratio of moving average.
And when you use those types of seasonal adjustments, especially if you aggregate micro, seasonally adjusted to macro data, you're going to get huge problems with data during those types of periods.
And it's exactly what we're having right now.
What is the correct seasonal for March?
Now, when you look at what happened to the economy in 2020 in March, you can see how the collapse.
When you use a ratio to moving average system there, you're going to just get these numbers way out of sight.
Way out of whack.
And that's where we are.
Response rates have deteriorated substantially.
Is there some element of that to this?
We've always had problems with the responses.
I mean especially when you move from communication methods like phone calls or like people going to the census, you know, knocking on doors and all that sort of stuff.
You always have those problems.
They're overcomable.
But there is an argument that there's just been a lot of budget cuts, a lot of staffing shortages at some of these statistical agencies, which has just made it a lot harder.
I don't believe that's true at all.
If you look at the budgets back in the olden days and stuff, I mean, come on, they have plenty of money and they really do have plenty of money, and how they distribute that money may make a difference.
But budget cuts in this kind of outrageously expensive world of government spending is not a good start for blaming that as being why the data aren't good.
So what does this all add up to in terms of how reliable economic data really is today?
Here's what my colleague Joseph has to say.
The evidence is somewhat mixed, at least when we're looking at the global economic data.
There's definitely some things that I can point to that suggest a tangible deterioration in data quality.
Standard errors, which are essentially the statistical measure of uncertainty associated with only having a limited number of people responding to any given survey, for some data releases have increased recently.
The poster child for this is the U.S.
JOLT survey.
It experienced a very sharp contraction in the number of firms that would respond to it.
As a result, the response rate standard error is probably about 80 higher today than it was in 2002 to 2013.
The CPI monthly standard error is probably going to increase, probably about double, due to the cutbacks in collection that are being driven by budgetary concerns.
And third, if we look globally, Australia and UK have both seen increases in standard errors for the retail sales survey.
The decline in response rates that has raised a lot of concerns also doesn't look entirely benign.
If we look at the US and UK, responses have skewed towards lower socioeconomic older individuals.
And so this raises concerns that maybe you're not getting a representative sample, and particularly a sample that you can't easily correct with sample weights to uncover the true economic statistics.
Also, we've seen a divergence between household and payroll survey measures in a number of countries.
The two that really stand out are the US and UK.
Given that household and payroll employment should be measuring the same thing.
When these two measures go a little bit different, it tells you that maybe you're not measuring things as appropriately as you would like.
And so all these things do suggest some moderate degree of data deterioration.
If I look at data revisions however, you know which data revisions are useful to look at, because they tell us whether or not what we're measuring changes from month to month or quarter to quarter as new information comes in.
The revisions don't look like they've increased, notably or certainly not to the extent that the common narrative would suggest.
And so my take overall is that we have seen data quality deteriorate globally, but not to a degree that calls the overall usefulness of economic data into question.
Right.
So some of the recent concern might be a little overblown in your estimation.
Yeah.
So there's things that you can point to where quality has deteriorated, but I don't think that it's a broad-based phenomenon.
But a key concern is whether economic data has become politicized.
I asked Erica whether that concern is overblown or not.
Some people have suggested or are concerned that revisions like this might suggest some political influences.
What does the BLS do to safeguard against that type of bias?
And what would you say to those who worry that the numbers can't be trusted?
Well, remember...
You're talking about a factory-like process where all the numbers come in from the employers and then they are put through an estimation routine that spits out the tables on the other end.
A revision is not a group of people sitting around saying, ah no, I really think it went up, or I want it to go up or down.
There's none of that judgment involved in creating it.
Every single employee of the BLS with one exception, the commissioner.
And the commissioner is appointed for a four-year fixed term, does not see any number before its final.
So when I was commissioner, I saw the numbers that were going to be reported on Friday.
I saw those numbers Tuesday night or Wednesday morning.
And I saw tables that had been spit out by this process.
Even changing what's in a row of those tables takes a month or two because they have to test it and make sure everything still looks right and comes out okay.
And that's not changing the values.
That's just saying, oh, we don't want this row anymore.
We want a different row.
So the commissioner has no way of doing that.
And if the him or herself into that process, they'd have to get access to the underlying data and or the programs?
They don't have that now.
They would have to really change the way everything is done.
And it would slow it down because it's pretty automated.
And what about concerns about the dismissal of the BLS commissioner and the nomination of EJ Antony?
Do you think those concerns are warranted or overblown?
Those concerns are completely unfounded as long as BLS processes don't change.
If that commissioner brought in some new people replaced the career, civil servants who were imbued with the culture of the place, able to change the culture of the place, because there's the imposition of a number of personnel changes that have been proposed, one called schedule policy career, which would convert certain civil servants into positions that could be fired by the president forever.
Anything that the president felt interfered with the president's policies.
And that is a proposal of this administration that they are trying to implement.
That you would expect might gradually change the culture of the agency and be problematic.
It's not in place yet, but it could be.
And there are a couple of other changes that they want to make.
But all of these things erode the protections for civil servants from being fired for reasons that have to do with the administration not feeling that they're supportive enough of the policy.
Those are not in place now, but they are a very real threat unless these statistical agencies are exempted from application of these changes in policy.
But Laufer doesn't see how fudging the data, in his words, could even be used politically and thinks the administration's shakeup of the BLS is warranted.
I don't know what purpose someone would fudge a number to get it to go down a little bit and go up a little bit.
What would be the motivation of someone to do that?
There's no election in sight.
What is the old phrase?
Don't attribute to conspiracy that which can easily be explained by incompetence.
I just have a hard time believing that Some would fudge these data for political purposes on a monthly basis.
People aren't trained to be deceitful, deceptive, to hide things.
How do you view the recent dismissal of the BLS commissioner and President Trump's nomination of EJ?
Antony to the post?
Donald Trump is a very action-taking person.
And when he finds what he believes to be a flaw in a system, he operates on it and he goes after it very quickly, very hard, maybe causing a little collateral damage in the process, but often and very often shaking the can hard and getting the things mixed up so that they settle in a better way in the future.
And while it is not my style of doing business, his way of doing business is quite effective on enabling the system to correct mistakes And I got to tell you I laud him for doing it.
It's not comfortable the time he does it, but I do laud him for doing that.
And I don't mean to say that I thought that the commissioner of the BLS was doing anything wrong.
And for sure, I am not accusing her of having done something politically wrong.
But when you get these numbers and you get a post COVID-19, it's time to shake the tub and reevaluate all the processes we're doing to get the best numbers we possibly can, to make the best decisions we possibly can.
And that's needed from time to time.
There's nothing wrong with a shakeup.
So I think you get in there a good director of the BLS and get them to really try to seek out new ways of looking at things.
And I'm not saying E.J.
Antoni is the only one who would be a good director, but I think he would be.
I think he'd be a fine one.
I know him.
I know him well.
I know his work.
And I tried to hire him for use of data, for accuracy, for my work.
I think he's quite qualified to be able to handle this job.
But what if the data were to become more unreliable?
I spoke to Alberto Cavallo, a professor at Harvard Business School and former member of the BLS's Technical Advisory Committee, to understand just how costly unreliable data is.
Here's what he had to say.
I started my career because I'm from Argentina and my country had gone through a similar type of situation.
Inflation had risen and the president didn't like it.
So they put some pressure on the statistical agency and they eventually fired people at the agency.
And then we started seeing things like some statistics that were no longer being reported, numbers that did not seem to coincide with what the perception was.
And I started my work on using online data to measure inflation as a response to that.
It was a very chaotic period that completely destroyed the trust in official statistics in Argentina.
It lasted a very long time and it brought a lot of costs to the government as well, because people didn't want to lend to Argentina anymore.
There was this feeling that Argentina could not be trusted.
So I find it very troubling.
One story I like to tell is that when people don't have reliable official statistics, they always tend to assume the worst.
So we were measuring inflation independently and we would notice sometimes that the inflation rate were actually deep for a while.
And the official index was always about three times lower than our data.
And it co-moved well over time.
When people saw the official index rising, they tended to believe it and they said things are really bad.
When that number actually came down, they assumed it was a complete lie.
So they didn't react in terms of inflation expectations and other behaviors.
So that's a big cost.
People tend to assume the worst and you end up with this asymmetric type of situation.
So there are lots of problems.
And what's important to remember is that this trust that is so fundamental to statistics is something you can easily lose.
And it takes a very long time to recover.
Argentina went through that cycle.
It took many years for people to start thinking that the official data was credible again.
So I hope this doesn't get worse in the US, but I am concerned.
And there are many costs down the road if this doesn't get fixed.
I'm still hopeful for the U.S. compared to Argentina.
I think the U.S. is starting from a better position.
The statistical agency has still a lot of highly qualified statisticians and you would hope there are more checks and balances institutionally for the VLS to resist some of the pressures that I've seen happening in Argentina in the past.
So I'm more optimistic of the U.S., about that.
But it's clearly very concerning to me and other people who work on measurement and on trying to help institutions like the BLS improve their methods over time.
Let's leave it there.
My thanks to Joseph Briggs, Erika Groschen, Arthur Laffer, and Alberto Caballo.
And thank you for listening to this episode of Goldman Sachs Exchanges.
I'm Alison Nathan. to purchase or sell any securities or financial products.
This material may contain forward-looking statements.
Past performance is not indicative of future results.
Neither Goldman Sachs nor any of its affiliates make any representations or warranties, expressed or implied, as to the accuracy or completeness of the statements or information contained herein, and disclaim any liability whatsoever for reliance on such information for any purpose.
Each name of a third-party organization mentioned is the property of the company to which it relates, is used here strictly for informational and identification purposes only and is not used to imply any ownership or license rights between any such company and Goldman Sachs.
This material should not be copied distributed, published or reproduced in whole or in part, or disclosed by any recipient to any other person without the express written consent of Goldman Sachs.
Disclosures applicable to research with respect to issuers, if any mentioned herein are available through your Goldman Sachs representative or at wwwgscom.
Slash research.
Slash hedge dot html.
Goldman Sachs does not endorse any candidate or any political party.
Copyright 2025 Goldman Sachs.
All rights reserved.
