“These caveats do not mean faster data should be ignored, but that we need to know its weaknesses while marvelling at its speed.” @ChrisGiles_ yes and yes https://www.google.com/amp/s/amp.ft.com/content/366653da-fc7b-4f3d-bf2f-ef95dfc18041 …
since 2016 I worked on private big data project at @federalreserve we transformed card transactions from First Data into retail sales series. 5 Fed economists, an excellent research assistant (now Stanford PhD econ studies), a brilliant Palantir programmer, (now at Philly Fed)
our methods paper shoes hoe we started with garbage data, for statistical purposes (left) and got to close correspondence with national, monthly retail sales from Census Bureau. only then did we use our daily, geographic detail for Fed policy work https://www.federalreserve.gov/econres/feds/files/2019057pap.pdf …
my last forecast in 2018, wrote Tealbook, economic summary for FOMC before each monetary policy meeting, we included real-time analysis on consumer spending from Hurricanes Harvey and Irma. we get our data within 3 days! years of painstaking work, got policymakers fast macro read
not at Fed anymore so don’t know how our data are being used now .. but I’m sure they are. FOMC must make decisions in the moment. they cannot wait for the data dust to settle. why they invested big dollars and years of staff time to tapping into private big data, pays off
another project started same time at Fed in 2016 uses ADP payrolls data to understand what’s happening in the labor market ... their high quality data (after years of with too) now shows wide spread wage cuts, never seen since Great Depression https://www.brookings.edu/bpea-articles/covid-19-and-labor-markets/ …
Fed not alone among government agencies using big data, both private and administrative, see this volume, *Big Data for 21st Century Economic Statistics* https://conference.nber.org/sched/CRIWs19 , Fed’s 2 projects there and many more!!
given countless talks on private big data statistics, last one at @business_econ Tech on ethical and logistical challenges in these government / private sector partnerships ... in the end, we on First Data project helped monetize their data, now sell to Wall Street (and Fed)
I’m not naive, knew from start that making First Data money would be a result, also knew I’d chosen to be a public servant, not a research assistant to Wall Street forecasting shops, ahem. whatever, we created data that FOMC benefits from in this crisis, so proud of that outcome
You can follow @Claudia_Sahm.
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.
Enjoy Threader? Sign up.
Since you’re here...
... we’re asking visitors like you to make a contribution to support this independent project. In these uncertain times, access to information is vital. Threader gets 1,000,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Your financial support will help two developers to keep working on this app. Everyone’s contribution, big or small, is so valuable. Support Threader by becoming premium or by donating on PayPal. Thank you.